WO2019154262A1 - 一种图像分类方法及服务器、用户终端、存储介质 - Google Patents

一种图像分类方法及服务器、用户终端、存储介质 Download PDF

Info

Publication number
WO2019154262A1
WO2019154262A1 PCT/CN2019/074149 CN2019074149W WO2019154262A1 WO 2019154262 A1 WO2019154262 A1 WO 2019154262A1 CN 2019074149 W CN2019074149 W CN 2019074149W WO 2019154262 A1 WO2019154262 A1 WO 2019154262A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
classification
feature
initial
classified
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/074149
Other languages
English (en)
French (fr)
Inventor
陈承
冯晓冰
褚攀
徐浩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Publication of WO2019154262A1 publication Critical patent/WO2019154262A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles

Definitions

  • the present application relates to the field of image processing technologies, and in particular, to an image classification method, a server, a user terminal, and a storage medium.
  • Embodiments of the present invention provide an image classification method, a server, a user terminal, and a storage medium, which can quickly and accurately classify an image.
  • an embodiment of the present invention provides an image classification method, including:
  • the classification result of the image to be classified is determined according to the comparison result.
  • an embodiment of the present invention provides another image classification method, including:
  • the camera component is called to acquire the current preview image
  • the response information includes a classification result of the image to be classified, and the classification result is obtained by the server classifying the image to be classified according to the classification model and the feature extraction model. .
  • the embodiment of the present invention further provides a server, including: a processor and a storage device; the storage device is configured to store a program instruction; the processor, the program instruction is invoked, and is executed to: acquire Classification image; calling the classification model to classify the image to be classified, determining an initial category to which the image to be classified belongs; calling a feature extraction model to determine a target feature vector of the image to be classified; The target feature vector is compared with the candidate classification feature vector configured for the initial category to obtain a comparison result; and the classification result of the image to be classified is determined according to the comparison result.
  • a server including: a processor and a storage device; the storage device is configured to store a program instruction; the processor, the program instruction is invoked, and is executed to: acquire Classification image; calling the classification model to classify the image to be classified, determining an initial category to which the image to be classified belongs; calling a feature extraction model to determine a target feature vector of the image to be classified; The target feature vector is compared with the candidate classification feature vector configured
  • an embodiment of the present invention further provides a user terminal, including: a processor and a storage device; the storage device is configured to store a program instruction, and the processor invokes the program instruction to perform: detecting When the triggering operation for the recognition button is performed, the camera component is called to acquire the current preview image; if it is detected that the current preview image is in a stable state, the current preview image is determined as the image to be classified; and the classification request is sent to the server, The classification request carries the image to be classified; receiving response information returned by the server, the response information includes a classification result of the image to be classified, and the classification result is that the server performs a model according to a classification model and a feature extraction model The image to be classified is obtained after classification and determination.
  • an embodiment of the present invention further provides a computer storage medium, where the computer storage medium stores program instructions, and when the program instructions are executed, is used to implement the foregoing methods.
  • the embodiment of the invention can obtain the image to be classified, and call the classification model to perform initial classification on the classified image, and then call the feature extraction model to determine the target feature vector of the image to be classified, and the target feature vector of the image to be classified is only in the initial classification.
  • the candidate feature vector of the configuration is compared, and the classification result of the image to be classified is determined according to the comparison result, which can effectively improve the correct rate of the image classification, and does not need to be compared with the candidate classification feature vector of the excessive category, thereby improving the operation.
  • Efficiency saving hardware and software resources.
  • FIG. 1 is a schematic diagram of a frame for performing image classification according to an embodiment of the present invention
  • FIG. 2a is a schematic structural diagram of an initial classification model according to an embodiment of the present invention.
  • 2b is a schematic structural diagram of an initial feature classification model according to an embodiment of the present invention.
  • 2c is a schematic diagram of a correspondence between a feature extraction network and a feature representation optimization module according to an embodiment of the present invention
  • 2d is a schematic diagram of training whitening parameters according to an embodiment of the present invention.
  • FIG. 3 is a schematic flow chart of an image classification method according to an embodiment of the present invention.
  • FIG. 4 is a schematic flow chart of another image classification method according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a user interface according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of an image classification apparatus according to an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of a server according to an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of another image classification apparatus according to an embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of a user terminal according to an embodiment of the present invention.
  • the classification model is classified by the classification training optimization classification, and the initial category is determined, and the classification model can be optimized by the special classification training mode, and is pre-configured in the server. It is better to classify images in multiple categories in devices with classification functions.
  • the feature extraction model with feature extraction optimization is completed to extract a more precise and accurate target feature vector from the image to be classified, based on the target feature vector and the candidate classification feature vector of one or more candidate images under the initial category. For comparison, the categories of the images to be classified are finally determined, and the classification results are obtained.
  • the classification model is used for initial classification, and then the finer feature vectors are subdivided in the initial classification, so that the confirmed categories are more accurate, and the data processing amount of the feature comparison is also effectively reduced, thereby saving hardware and software resources. Increased processing efficiency.
  • a general classification model may be adopted, and the general classification model may be used to roughly classify the image, for example, the content of an image may be divided into cats or dogs.
  • the classification model may also adopt a fine-grained image classification model based on strong supervisory information or a fine-grained image classification model based on weak supervisory information. Based on the fine-grained image classification model, more detailed classifications can be performed, for example, subdivided into cat breeds, dog breeds, and the like.
  • the fine-grained image classification model based on strong supervised information means that in order to obtain better classification accuracy in model training optimization, in addition to the category label of the image, additional manual annotation information such as object labeling frame and part labeling point is used. This type of classification model can achieve higher classification accuracy.
  • a fine-grained image classification model based on weak supervisory information means that better local information capture can be achieved without labeling points with parts. Compared with the strong supervised classification accuracy, there is a gap, but the implementation cost is low, which is more suitable for engineering applications.
  • Embodiments of the present invention mainly include a configuration phase of a classification model and a feature extraction model, and an image classification phase based on a classification model and a feature extraction model.
  • the initial classification model and the initial feature extraction model can be established offline based on the configuration server, and the initial classification model and the initial feature extraction model are trained and optimized through a large number of training images, and a better classification capable of classifying the image is obtained.
  • Model and feature extraction models are loaded onto a server for image classification (or some other device for image classification) to facilitate providing the user with a classification service for unknown images online.
  • the user In the image classification stage, the user only needs to capture the image to be classified through the user terminal or obtain the image to be classified by other means (for example, downloading, etc.), and carry the image to be classified in the classification request and send it to the server for image classification.
  • the classification model and the feature extraction model are used by the server to process the image to be classified, determine a classification result of the image to be classified, and return a response information to the classification request to the user terminal, where the response information carries the classification The classification result of the image.
  • FIG. 1 is a schematic diagram of a frame for performing image classification, which includes a user side and a service side, wherein the user terminal on the user side may be a smart phone or a tablet computer.
  • a smart terminal with a camera function and a network function can of course be a terminal such as a personal computer.
  • the online service on the service side can be provided by a server or server group that can receive the classification application of the target application for image classification, and the offline processing service on the service side can be used to design and train the classification model and the feature extraction model. Server or server group to provide.
  • the online and offline services on the service side may also be provided through a dedicated server or group of servers.
  • the target application can be an application with a scan identification module.
  • the initial classification model includes: Input layer, Stem layer, Inception-resnet layer, Reduction layer, Average pooling layer, Dropout layer, Softmax layer.
  • the Input layer is an input layer and can be used for inputting images; the Inception-resnet layer is an implicit layer.
  • the initial classification model has three hidden layers: Inception-resnet-A first hidden layer, Inception- The second implicit layer of resnet-B and the third implicit layer of Inception-resnet-C;
  • the Stem layer is a pre-processing layer, which can be used for pre-processing data input to Inception-resnet-A, which can include data Multiple convolutions and pooling;
  • the Average Pooling layer is an average pooling layer that can be used to reduce the dimensionality of the data output by Inception-resnet-C;
  • the Dropout layer can be used to prevent over-fitting of the initial classification model. Effectively avoid the initial classification model to classify the training images well, but after the deployment, the classification effect is poor for the images that need to be classified.
  • the Softmax layer is the classification calculation layer, and its output can be through Input. The image entered by the layer belongs to the probability of each category.
  • the training model of the plurality of image categories may be input to the initial classification model in advance by using the model generation tool, and the initial classification model is trained.
  • the model generation tool used is a tool of Tensorflow (a deep learning framework for fields such as image recognition)
  • the reverse gradient on the Tensorflow tool is automatically calculated, it can be used in the training optimization process.
  • the training parameters of each of the initial classification models are quickly adjusted according to the input training image, so as to optimize the training of the initial classification model, and obtain a classification model.
  • the category of the training image can be adjusted according to different design goals.
  • the design goal is to create a classification model that can be used to distinguish between a first category (eg, a cat) and a second category (eg, a dog) image.
  • M is a positive integer, such as 10000
  • dog category or “dog” for convenience of description.
  • the image of the dog is used as the initial training model for the dog training image and P (P is a positive integer such as 10000) has been determined as the image of the cat category as the cat training image of the initial classification model.
  • the initial classification model may extract image feature data of the dog training image, and classify the dog training image according to the image feature data. If the output classification result indicates that the category of the dog training image is also a dog, it indicates that the classification network model is successful in classifying the dog training image. Further, after classifying the training images that have been marked as the dog category, if the success rate is greater than the preset success rate threshold (eg, 90%), it is determined that the initial classification model can perform the image of the dog category well. Classification identification, otherwise, the parameters corresponding to each node in the initial classification model can be adjusted, and the M dog training images are again classified by the adjusted classification model.
  • the preset success rate threshold eg, 90%
  • the P-cat training images can be used to train and optimize the initial classification model in the same way. If the final classification success rate for the dog training image and the cat training image meets the preset success rate threshold, then The training of the initial classification model is completed, and the initial classification model of the training completion is taken as the classification model in the embodiment of the present invention. In other embodiments, more different categories can be set, and the initial classification model is optimized by acquiring a large number of different types of training images, so that the final classification model can successfully classify each type of image. The rates are above a certain success rate threshold.
  • the categories of the first category and the second category described above may be categories such as cats or dogs, or may be a more refined category such as a specific cat or dog breed, such as a sheepdog, a Shiba, etc. .
  • each trained training image may be determined as a candidate map, and a category label of the category to which the candidate belongs is set according to the category thereof.
  • the candidate map and the corresponding category label are associated and stored in a database to facilitate subsequent determination of the category to which the candidate map belongs based on the category label.
  • the feature extraction model may be configured according to a feature extraction network and a feature representation optimization module, where the feature extraction network is configured to extract an initial feature vector of the image to be classified based on the neural network, and the feature representation optimization module is used for The initial feature vector is optimized to obtain an N-dimensional target feature vector, N is a positive integer, and the feature extraction model finally outputs an N-dimensional target feature vector with respect to the image, for example, outputting a 2048-dimensional target feature vector.
  • an initial feature classification model and a training image may be acquired, and the initial feature classification model includes an initial feature extraction network based on a neural network. Further, the initial feature classification model may be trained and optimized according to the acquired training image to obtain a feature classification model, and the feature classification model includes a feature extraction network obtained by training and optimizing the initial feature extraction network, and then the feature extraction network may be obtained from the feature extraction network.
  • the feature extraction network is obtained in the classification model, and the feature representation optimization module is generated based on the whitening parameter, and the feature extraction model is generated according to the feature extraction network and the feature representation optimization module.
  • the feature representation optimization module can be implemented based on R-MAC, an image feature extraction method.
  • the feature representation optimization module may be obtained by training optimization of the whitening parameter based on the training image, and the whitening parameter may be composed of a matrix w and an offset matrix b, and the feature representation optimization module may be used for the feature
  • the feature vector of the network output is extracted for optimization, and the optimized feature vector is output.
  • This feature indicates that the processing of the optimization module can be a whitening process.
  • the training optimization process of the whitening parameter includes: firstly, the feature extraction network is called to extract the feature vector from the plurality of training images, and the matrix w and the offset matrix are optimized according to the obtained feature vector. b, which in turn obtains a feature representation optimization module that is generated based on the matrix w and the bias matrix b.
  • a 2048-dimensional feature representation training matrix x is obtained, and a matrix of 2048*2048 and an offset matrix of 2048 are calculated.
  • the training matrix x is represented, so that the 2048-dimensional feature represents the training matrix x in the whitened matrix, and the dimensional data is irrelevant, that is, after knowing the value of a certain dimensional data, guessing or calculating The value of the dimension did not help.
  • FIG. 2d the image on the left side in FIG.
  • 2d is a two-dimensional vector of 1 million*2048 dimensions, and it can be seen that in the image on the left side, the dimensional data is relatively dense, and the values of the dimensional data are between There is a certain correlation.
  • the 2048 dimension refers to the dimension quantity that the feature representation optimization module needs to output. If there are other dimension quantity requirements, for example, it is required to output data of 4096 dimensions or even more dimension quantities, then in the training w and b, the feature representation optimization module is generated. In the process, the above 2048 needs to be adjusted to a value of 4096 dimensions or even more dimensions.
  • the feature classification model and the feature representation optimization module may be generated by training the initial feature classification model and the whitening parameters by using a model generation tool.
  • the model generation tool used can also be a Tensorflow tool. Since the inverse gradient on Tensorflow is automatically calculated, the feature classification model and the feature representation optimization module can be obtained relatively quickly.
  • an initial feature classification model is illustrated, which includes an initial feature extraction network based on a neural network, which may employ a convolutional neural network.
  • the initial feature extraction network includes three convolution layers: a first convolutional layer cfgl[0] block, a second convolutional layer cfgl[1] block, and a third convolutional layer cfgl[2].
  • Block in other embodiments, may also include more convolution layers, for example, after the cfgl [2] block, may also include a fourth convolution layer cfgl [3] block and the like.
  • each convolution layer in the initial feature extraction network convolves the input training image and outputs a feature vector (ie, an initial classification feature vector of the training image) about the input training image.
  • the initial feature extraction network includes a first type of convolution layer and a second type of convolution layer.
  • cfgl[0] block, cfgl[1] block are the first type of convolution layer, and cfgl[0
  • the block sends the convolved data to the cfgl[1] block, and the cfgl[1] block sends the convolved data to the cfgl[2] block.
  • the cfgl[2] block shown in Figure 2b is a second type of convolutional layer, and the cfgl[2] block outputs the convolutionally processed data (ie, the initial classification feature vector of the training image) to facilitate subsequent network layer pairs.
  • the initial classification feature vector is calculated to determine the category of the image.
  • the network layer for calculating the classification feature vector is extracted from the trained feature classification model, and these network layers are used as the feature extraction network, for example, cfgl[0] Block, cfgl[1] block, and cfgl[2] block, cfgl[3] block are used as feature extraction networks.
  • the initial feature classification model shown in FIG. 2b is trained and optimized, and after the feature classification model is obtained, the initial feature extraction network after training optimization can be obtained from the feature classification model (ie, obtained from the feature classification model).
  • Feature extraction network Feature extraction network
  • Figure 2c the connection relationship between the feature extraction network and the feature representation optimization module as shown in Figure 2c.
  • each convolution layer (cfgl[0] block, cfgl[1] block, and cfgl[2] block) in the feature extraction network can input the image.
  • Convolution processing is performed to determine the initial classification feature vector of the input image, and the initial classification feature vector of the input image is sent to the feature representation optimization module through the last convolution layer cfgl[2] block, and then the feature representation optimization module pair is utilized.
  • the initial classification feature vector of the input image is optimized, and the candidate classification feature vector of the input image is determined and output.
  • the optimization process may be to perform multi-dimensional vector identification on the initial classification feature vector of the input image, for example, a 2048-dimensional vector representation, that is, the optimized feature vector (ie, the candidate classification feature vector) may be a 2048-dimensional.
  • the vector, the representation of the 2048-dimensional feature data may be, for example, (0.1, 0.11, 0.15, ..., 0.16). It can be understood that, based on different feature representation optimization modules, vectors of other dimensions can also be obtained. The higher the dimension of the used vector, the more accurate the classification result of the input image is.
  • the classification model may be invoked to perform initial classification on various training images, and the successfully trained training images are stored as candidate images in a database.
  • the candidate graph can be used as an input of the feature extraction network, and the feature extraction model is used to calculate the candidate classification feature vector of all the candidate graphs, and the corresponding candidate classification feature vectors are generated for all the candidate graphs, and the candidate graph and the candidate classification feature are obtained.
  • the vector is stored in association so that when the user's classification request for the image is received, the classification result of the desired classification image can be directly determined based on the candidate classification feature vectors and returned to the user terminal without receiving the relevant image.
  • the feature extraction model is called to calculate a large number of candidate graphs to obtain candidate classification feature vectors, which saves a lot of calculation and query time.
  • a mapping relationship between the candidate graph and the candidate classification feature vector may be established, where column 1 is a storage address of the candidate graph, and column 2 is a candidate classification feature vector.
  • column 1 is a storage address of the candidate graph
  • column 2 is a candidate classification feature vector.
  • the classification model and the feature extraction model after the training optimization can be configured into the corresponding server, and a large number of candidates are prepared.
  • the map and the candidate classification feature vector are stored in association with each other to provide an online image classification service for the user.
  • different candidate classification feature vectors may be configured for different categories according to the category to which the candidate graph belongs. Specifically, when L (L is a positive integer) candidate graphs are stored in a certain category, L candidate classification feature vectors may be directly configured for the category, that is, each candidate graph corresponds to one candidate classification feature vector.
  • the candidate classification feature vector of each candidate picture in the category may be similarly calculated, and each candidate classification feature vector whose similarity is smaller than the first similarity threshold is classified into the same type of feature vector, so that belonging to the same class
  • Each candidate graph corresponding to the candidate classification feature vector is represented by the same candidate classification feature vector, that is, multiple candidate graphs correspond to the same candidate classification feature vector, and then less than L candidate classification feature vectors are configured for the category. In this way, the number of candidate classification feature vectors can be reduced for each category while ensuring image classification accuracy, thereby increasing the operation rate.
  • the candidate map may be various training images adopted in the foregoing, or may be corresponding types of images obtained by the user by downloading or photographing, etc., the images are known in categories, and are extracted by the above-mentioned features.
  • the model extracts candidate classification feature vectors, and these images and corresponding candidate classification feature vectors can be stored in the database for subsequent searching.
  • the classification model and the feature extraction model can be trained and configured by the server.
  • the utility model can also be implemented by a powerful personal computer with rich software and hardware resources. The embodiment of the invention is not specifically limited thereto.
  • FIG. 3 it is a schematic flowchart of image classification according to an embodiment of the present invention.
  • the method in the embodiment of the present invention may be performed by a server or a server group.
  • the method of an embodiment of the invention comprises the following steps.
  • the server may receive the classification request sent by the user terminal, and obtain the carried image to be classified from the classification request.
  • the image to be classified may be obtained by the user using the shooting module of the user terminal, or may be obtained by using other methods, which is not specifically limited in the embodiment of the present invention.
  • S302 Calling the classification model to classify the classified image, and determining an initial category to which the image to be classified belongs.
  • the classification model may be obtained by the development user based on the initial classification model for training optimization, and the classification model has the characteristics of high accuracy of TOP5, that is, the classification model can relatively accurately determine the image to be classified.
  • the category ie, the initial category
  • the classification model can more accurately determine that the initial category to which the image to be classified belongs is a cat, not a dog.
  • the classification model is derived from a large number of training optimizations by the initial classification model.
  • the initial classification model can be as shown in FIG. 2a, and the process of optimizing the initial classification model to obtain a classification model can be referred to the related description above, and details are not described herein again.
  • the classification model may extract feature data of the image to be classified, and calculate a probability that the image to be classified belongs to each candidate category according to the feature data, thereby ranking the probability first.
  • K K is a positive integer
  • candidate categories are determined as the initial category to which the image to be classified belongs.
  • the feature extraction model may be configured according to a feature extraction network for extracting an initial feature vector of the image to be classified based on the neural network, and the feature representation module is configured to perform the initial feature vector.
  • the N-dimensional target feature vector is obtained by optimization, and N is a positive integer.
  • an initial feature classification model and a training image may be obtained, and the initial feature classification model includes an initial feature extraction network based on a neural network. Further, the initial feature classification model may be trained and optimized according to the acquired training image to obtain a feature classification model, and the feature classification model includes a feature extraction network obtained by training and optimizing the initial feature extraction network, and then the feature extraction network may be obtained from the feature extraction network.
  • the feature extraction network is obtained in the classification model, and the feature extraction model is generated according to the feature extraction network and the pre-generated feature representation optimization module.
  • the feature representation optimization module may be obtained by training and optimizing the whitening parameter based on the training image, and the feature representation optimization module may be configured to optimize the feature vector output by the feature extraction network, and output the optimized feature vector.
  • connection relationship between the feature extraction network and the feature representation optimization module may be as shown in FIG. 2c.
  • the feature extraction network may be constructed by a development user based on a neural network, and the feature extraction network may include a first type of convolution layer and a second type of convolution layer, and the first type of convolution layer will be convolutionally processed.
  • the data is output to another convolutional layer in the feature extraction network, and the second type of convolutional layer outputs the convolved processed data to the feature representation optimization module.
  • the feature extraction network may be first processed to process the classified image, and an initial feature vector may be obtained for the image to be classified, and the initial feature vector may be input into the feature representation through the second type of convolution layer.
  • the optimization module further calls the feature representation optimization module to optimize the initial feature vector to obtain an N-dimensional target feature vector for the image to be classified, such as a 2048-dimensional vector, which can represent the to-be-classified to some extent.
  • the image, the target feature vector is an image understood by the computer, which is difficult for the user to understand and difficult to visualize.
  • S304 Compare the target feature vector of the image to be classified with the candidate classification feature vector configured for the initial category to obtain a comparison result.
  • the comparison may be performed by calculating the similarity between each dimension vector in the target feature vector and the candidate classification feature vector, for example, calculating the similarity Hamming distance or the Euclidean distance between each dimension vector. The higher the finally determined similarity, the greater the probability that the image to be classified belongs to the category to which the corresponding candidate classification feature vector belongs.
  • S305 Determine a classification result of the image to be classified according to the comparison result.
  • the initial category may include at least one candidate graph
  • the server or the server group may acquire candidate graphs belonging to the initial category, and according to the candidate graph, invoke the feature extraction model to configure the candidate classification feature vector for the initial category.
  • the candidate classification feature vector configured for the initial category may be that the candidate map is processed by the feature extraction network in the feature extraction model, and the initial classification feature vector under the initial category is obtained, and the feature in the feature extraction model is obtained. Indicates that the optimization module optimizes the initial classification feature vector under the initial category.
  • the candidate classification feature vector may be an N-dimensional feature vector, for example, a vector of 2048 dimensions.
  • Q candidate graphs may be included under the initial category (each candidate graph corresponds to one candidate classification feature vector).
  • the Q candidate classification feature vectors can be directly configured for the initial category, that is, each candidate map corresponds to one candidate classification feature vector.
  • the candidate classification feature vector of each candidate graph in the initial category may be similarly calculated, and the candidate classification feature vectors whose similarity is smaller than the first similarity threshold may be classified into the same class, and the candidate classification feature vector belonging to the same category
  • Each of the corresponding candidate graphs is represented by the same candidate classification feature vector, that is, the plurality of candidate graphs all correspond to the same candidate classification feature vector, and then less than Q candidate classification feature vectors are configured for the initial category. In this way, it is possible to reduce the number of candidate classification feature vectors for the initial category while reducing the accuracy of image classification, thereby reducing the amount of calculation, thereby increasing the operation rate.
  • the initial category may include a first category and a second category
  • the server may perform a similarity calculation on the target feature vector of the image to be classified and the candidate classification feature vector configured for the first category to obtain a first similarity; Performing similarity calculation on the target feature vector and the candidate classification feature vector configured for the second category, obtaining a second similarity, and comparing the first similarity with the second similarity to obtain a comparison result, the comparison result indicating the first A larger value between a similarity and a second similarity. Further, if the comparison result indicates that the first similarity is greater than the second similarity, it may be determined that the image to be classified belongs to the first category.
  • the initial category may also include the third category, the fourth category, or other categories, which are not specifically limited in this embodiment of the present invention.
  • the target feature vector of the image to be classified is (0.23, 0.44, . . . , 0.61), and the initial category is a dog, wherein the first category included in the dog category is Golden Retriever, and the second category is Teddy, corresponding to Golden Retriever.
  • the candidate classification feature vectors are (0.23, 0.44, . . . , 0.67), and the candidate classification feature vectors corresponding to Teddy are (0.23, 0.31, . . . , 0.60), and the server passes the target feature vector (0.23, 0.41, . . . , 0.61)
  • the candidate classification feature vectors (0.23, 0.44, ..., 0.67) corresponding to Golden Retriever are similarly calculated, and the first similarity is 98%.
  • the target eigenvectors (0.23, 0.41, . . . , 0.61)
  • the similarity calculation is performed on the candidate classification feature vectors (0.23, 0.31, . . . , 0.60) corresponding to Teddy, and the second similarity is 90%, and the contrast is determined by comparing the first similarity and the second similarity. As a result, if the first similarity is greater than the second similarity, it can be determined that the image to be classified belongs to Golden Retriever.
  • each of the first category and the second category may include a candidate map associated with the candidate classification feature vector. If the comparison result indicates that the first similarity is greater than the second similarity, the first comparison may be performed according to the first The similarity determination determines the associated image of the image to be classified, and the associated image refers to a candidate graph in which the similarity between the candidate classification feature vector and the target feature vector is the first similarity. For example, if the similarity between the feature vector to be classified and the target feature vector is that the candidate image of the first similarity is the image JPG1 and the image JPG2, then both the image JPG1 and the image JPG2 may be determined as the associated image of the image to be classified.
  • the target feature vector of the image to be classified when the target feature vector of the image to be classified is similar to the candidate classification feature vector configured for the first category or the second category, the target feature vector of the image to be classified may be calculated by the matching algorithm and the first The Euclidean distance or the cosine angle between the candidate classification feature vectors under the category or the second category, thereby obtaining the degree of similarity between the target feature vector and the candidate classification feature vector corresponding to each candidate map.
  • the similarity is calculated by the matching algorithm, the matching can also be performed by sequential traversal comparison, and it is also possible to search by using a ditchs (an open source library for efficient similarity search and dense vector clustering) search engine.
  • a noirs an open source library for efficient similarity search and dense vector clustering
  • the response information for the classification request may be returned to the user terminal, and the response information may include at least one of the classification result and the associated image.
  • FIG. 4 it is a schematic flowchart of another image classification method according to an embodiment of the present invention.
  • the method in the embodiment of the present invention may be performed by a user terminal.
  • the method of the embodiment of the invention comprises the following steps.
  • the user terminal is installed with a target application that has an image classification function entry, and the image classification function entry may include a scan identification module.
  • the trigger operation may be input for the recognition button provided by the scan recognition module, and when the user terminal detects the trigger operation, a scan recognition operation event may be generated, and the camera component may be called according to the event.
  • the current preview image Further, if it is detected that the current preview image is in a stable state, the current preview image in a stable state may be taken as the image to be classified, or directly into the stored image as the image to be classified.
  • the target application may be, for example, an application having an image classification function entry such as a browser, an instant messaging application, or a payment application.
  • the current preview image may be acquired according to a preset time interval.
  • the current preview image may be compared with the preview image acquired by the previous T times, if the similarity is greater than Or equal to the similarity threshold, it can be determined that the current preview image is in a stable state.
  • T is a positive integer greater than 0, and the specific value of T may be adjusted according to different design requirements, which is not specifically limited in the embodiment of the present invention.
  • the target application is a browser.
  • the user terminal After clicking the scan identification button 501 on the browser, the user terminal enters an image acquisition interface, and includes an image display area 502 and a prompt on the image acquisition interface.
  • the "+" button 503 of the image is imported, and the user clicks the button 503 to enter the image selection and confirmation interface.
  • the preview image in a stable state is determined as the image to be classified.
  • S403 Send a classification request to the server, where the classification request carries an image to be classified.
  • the user terminal may generate a classification request carrying the image to be classified, and then send the classification request to the server providing the online classification service.
  • the response information includes a classification result of the image to be classified, and the classification result may be obtained by the server classifying and determining the classification image according to the classification model and the feature extraction model.
  • the server may determine the classification result of the image to be classified according to the classification request, and carry the classification result in the response information and return to the user terminal.
  • the user terminal may display the classification result on the user interface for the user to view.
  • the response information may include an associated image or other information of the image to be classified in addition to the classification result, which is not specifically limited in the embodiment of the present invention.
  • the user terminal may display a user interface, where the user interface displays: one or more of a classification result, an associated image, and description information of the image to be classified,
  • the classification result and the associated image may be carried in the response information, and the description information may be obtained by querying according to the classification result.
  • the description information may be text information and image information associated with the classification result.
  • the classification result indicates that the object included in the image to be classified is the X-type off-road vehicle 01
  • the description information may include a basic introduction of the X-type off-road vehicle 01, such as a performance list, time to market, price, and sale of the X-series off-road.
  • the 4S shop of the car 01 and the like may also include a platform website for selling the X-type off-road vehicle 01, etc., which is not specifically described in the embodiment of the present invention.
  • the user terminal after receiving the response information, can directly search online based on the classification result carried in the response information, thereby obtaining description information of the image to be classified, and displaying the description information on the user interface.
  • the description information of the classification result may be searched online or locally, and the obtained description information is displayed on the user interface.
  • the description information displayed by the user interface may be obtained by querying the classification result, or the response information may be carried by the user information, and the user terminal obtains the response information, and the embodiment of the present invention does not Specifically limited.
  • the embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores program instructions, which are used to implement the corresponding methods described in the foregoing embodiments.
  • FIG. 6 is a schematic structural diagram of an image classification apparatus according to an embodiment of the present invention.
  • the image classification apparatus may be disposed in a server, or may be disposed in some intelligent terminals with rich software and hardware resources, such as some individuals. In the computer.
  • the apparatus includes the following structure.
  • the calling module 602 is configured to invoke the classification model to classify the image to be classified, and determine an initial category to which the image to be classified belongs;
  • the calling module 602 is further configured to invoke a feature extraction model to determine a target feature vector of the image to be classified;
  • a comparison module 603 configured to compare a target feature vector of the image to be classified with a candidate classification feature vector configured for the initial category, to obtain a comparison result;
  • the determining module 604 is configured to determine, according to the comparison result, a classification result of the image to be classified.
  • the feature extraction model is configured according to a feature extraction network and a feature representation optimization module, and the feature extraction network is configured to extract an initial feature vector of the image to be classified based on the neural network, where the feature representation optimization module uses The initial feature vector is optimized to obtain an N-dimensional target feature vector, where N is a positive integer.
  • the apparatus may further include: a training module 605, a generating module 606, wherein: the obtaining module 601 is further configured to acquire an initial feature classification model and a training image, where the initial feature classification model includes a neural network based The initial feature extraction network is configured; the training module 605 is configured to perform training optimization on the initial feature classification model according to the training image acquired by the obtaining module 601, to obtain a feature classification model, where the feature classification model includes The feature extraction network obtained after the initial feature extraction network is trained and optimized; the obtaining module 601 is further configured to acquire the feature extraction network from the feature classification model; and the generating module 606 is configured to generate a feature representation based on the whitening parameter An optimization module; the generating module 606 is further configured to generate a feature extraction model according to the feature extraction network and the feature representation optimization module, where the feature representation optimization module is configured to perform feature vectors output by the feature extraction network Optimized and used to output optimized feature vectors.
  • a training module 605 is configured to perform training optimization on the initial feature classification model according to
  • the device may further include: a configuration module 607, wherein: the obtaining module 601 is further configured to acquire a candidate graph belonging to the initial category; and the configuration module 607 is configured to obtain, according to the acquiring module 601 The candidate map is invoked, and the feature extraction model is invoked to configure a candidate classification feature vector for the initial category.
  • the configuration module 607 may be specifically configured to process the candidate graph by using a feature extraction network in the feature extraction model to obtain an initial classification feature vector in the initial category, and pass the feature.
  • the feature representation optimization module in the extraction model optimizes the initial classification feature vector under the initial category to obtain an N-dimensional candidate classification feature vector under the initial category.
  • the initial category includes a first category and a second category
  • the comparing module 603 may include: a calculating unit 6031, configured to: target the target feature vector of the image to be classified The similarity calculation is performed on the candidate classification feature vector of the category configuration to obtain a first similarity; the similarity calculation is performed on the target feature vector and the candidate classification feature vector configured for the second category to obtain a second similarity; the comparing unit 6032, And comparing the first similarity and the second similarity to obtain a comparison result, where the comparison result indicates a larger value between the first similarity and the second similarity.
  • the determining module 604 may be specifically configured to determine that the image to be classified belongs to the first category if the comparison result indicates that the first similarity is greater than the second similarity.
  • the first category and the second category each include a candidate map associated with the candidate classification feature vector
  • the determining module 604 is further configured to: if the comparison result indicates the first similarity If the second similarity is greater than the first similarity, the associated image of the image to be classified is determined according to the first similarity, where the associated image refers to: a similarity between the candidate classification feature vector and the target feature vector The degree is a candidate graph of the first similarity.
  • each module may refer to the description of related content in the foregoing embodiment corresponding to FIG. 3.
  • FIG. 7 is a schematic structural diagram of a server according to an embodiment of the present invention.
  • the server includes a power supply module and the like, and includes a processor 701, a storage device 702, and a network interface 703.
  • the processor 701, the storage device 702, and the network interface 703 can exchange data, and the processor 701 implements a corresponding image classification function.
  • the storage device 702 may include a volatile memory, such as a random-access memory (RAM); the storage device 702 may also include a non-volatile memory, such as a fast A flash memory, a solid-state drive (SSD) or the like; the storage device 702 may further include a combination of the above-described types of memories.
  • RAM random-access memory
  • SSD solid-state drive
  • the network interface 703 can exchange data with other servers and various user terminals, and the user terminal can send a classification request carrying the image to be classified to the network interface 703, and the network interface 703 outputs the processor to the server. 701 is processed.
  • the processor 701 can be a central processing unit (CPU). In an embodiment, the processor 701 may also be a graphics processing unit (GPU) 701. The processor 701 can also be a combination of a CPU and a GPU. In the server, a plurality of CPUs and GPUs may be included as needed to perform corresponding image processing. In one embodiment, the storage device 702 is configured to store program instructions. The processor 701 can invoke the program instructions to implement various methods as described above in the embodiments of the present invention.
  • the processor 701 of the server invokes a program instruction stored in the storage device 702 to acquire an image to be classified; and invokes a classification model to classify the image to be classified. Determining an initial category to which the image to be classified belongs; calling a feature extraction model to determine a target feature vector of the image to be classified; and selecting a target feature vector of the image to be classified and a candidate classification feature configured for the initial category The vector is compared to obtain a comparison result; and the classification result of the image to be classified is determined according to the comparison result.
  • the feature extraction model is configured according to a feature extraction network and a feature representation optimization module, and the feature extraction network is configured to extract an initial feature vector of the image to be classified based on the neural network, where the feature representation optimization module uses The initial feature vector is optimized to obtain an N-dimensional target feature vector, where N is a positive integer.
  • the processor 701 is further configured to acquire an initial feature classification model, where the initial feature classification model includes an initial feature extraction network based on a neural network; acquire a training image, and according to the training image pair The initial feature classification model performs training optimization to obtain a feature classification model, wherein the feature classification model includes a feature extraction network obtained by training and optimizing the initial feature extraction network; and obtaining the feature classification model from the feature classification model Generating a feature extraction network; generating a feature representation optimization module based on the whitening parameter; generating a feature extraction model according to the feature extraction network and the feature representation optimization module, wherein the feature representation optimization module is configured to extract the feature extraction network
  • the output feature vector is optimized and used to output the optimized feature vector.
  • the processor 701 is further configured to acquire a candidate graph that belongs to the initial category;
  • the feature extraction model is invoked to configure a candidate classification feature vector for the initial category.
  • the processor 701 is further configured to process the candidate graph by using a feature extraction network in the feature extraction model to obtain an initial classification feature vector in the initial category, and
  • the feature representation optimization module in the feature extraction model optimizes the initial classification feature vector under the initial category to obtain an N-dimensional candidate classification feature vector under the initial category.
  • the initial category includes a first category and a second category
  • the processor 701 is further configured to: target feature vectors of the image to be classified and candidate classification features configured for the first category Performing similarity calculation on the vector to obtain a first similarity; performing similarity calculation on the target feature vector and the candidate classification feature vector configured for the second category to obtain a second similarity; and the first similarity and the The second similarity is compared to obtain a comparison result indicating a larger value between the first similarity and the second similarity.
  • the processor 701 is further configured to determine that the image to be classified belongs to the first category if the comparison result indicates that the first similarity is greater than the second similarity.
  • the first category and the second category each include a candidate map associated with the candidate classification feature vector
  • the processor 701 is further configured to: if the comparison result indicates the first If the similarity is greater than the second similarity, the associated image of the image to be classified is determined according to the first similarity, where the associated image refers to: between the candidate classification feature vector and the target feature vector
  • the similarity is a candidate graph of the first similarity.
  • the specific implementation of the processor 701 may refer to the description of related content in the foregoing embodiment corresponding to FIG. 3.
  • FIG. 8 it is a schematic structural diagram of another image classification device according to an embodiment of the present invention.
  • the image classification device may be disposed in a user terminal.
  • the apparatus includes the following structure.
  • a detecting module 801 configured to detect a triggering operation for the identification button
  • the obtaining module 802 is configured to invoke the camera component to acquire a current preview image when the detecting module detects a triggering operation for the identification button;
  • the detecting module 801 is further configured to detect whether the current preview image is in a stable state
  • a determining module 803, configured to determine the current preview image as an image to be classified if the detecting module detects that the current preview image is in a stable state
  • the sending module 804 is configured to send a classification request to the server, where the classification request carries the image to be classified;
  • the receiving module 805 is configured to receive response information returned by the server, where the response information includes a classification result of the image to be classified, and the classification result is that the server compares the image to be classified according to a classification model and a feature extraction model. Obtained after classification.
  • the device may further include: a display module 806, configured to display a user interface after receiving the response information returned from the server, where the user interface displays: a classification result of the image to be classified, and an association Any one or more of the image and the description information, the classification result and the associated image are all carried in the response information, and the description information is obtained by querying according to the classification result.
  • a display module 806 configured to display a user interface after receiving the response information returned from the server, where the user interface displays: a classification result of the image to be classified, and an association Any one or more of the image and the description information, the classification result and the associated image are all carried in the response information, and the description information is obtained by querying according to the classification result.
  • each module may refer to the description of related content in the foregoing embodiment corresponding to FIG. 4.
  • FIG. 9 is a schematic structural diagram of a user terminal according to an embodiment of the present invention.
  • the user terminal in the embodiment of the present invention may include a power supply module and the like, and includes a processor 901, a storage device 902, and a transceiver 903.
  • the processor 901, the storage device 902, and the transceiver 903 can exchange data, and the processor 901 implements a corresponding image classification function.
  • the storage device 902 may include a volatile memory, such as a random-access memory (RAM); the storage device 902 may also include a non-volatile memory, such as a fast A flash memory, a solid-state drive (SSD) or the like; the storage device 902 may further include a combination of the above-described types of memories.
  • RAM random-access memory
  • SSD solid-state drive
  • the transceiver 903 can exchange data with the server and various user terminals, and the server can send the response information carrying the classification result to the transceiver 903, and the transceiver 903 outputs the processor 901 to the user terminal. Process it.
  • the processor 901 can be a central processing unit (CPU). In an embodiment, the processor 901 may also be a graphics processing unit (GPU) 901. The processor 901 can also be a combination of a CPU and a GPU. In the user terminal, a plurality of CPUs and GPUs may be included as needed to perform corresponding image processing. In one embodiment, the storage device 902 is configured to store program instructions. The processor 901 can invoke the program instructions to implement various methods as described above in the embodiments of the present invention.
  • the processor 901 of the user terminal invokes a program instruction stored in the storage device 902, and when detecting a trigger operation for the identification button, invoking the camera component to acquire the current Previewing an image; if it is detected that the current preview image is in a stable state, determining the current preview image as an image to be classified; sending a classification request to the server, the classification request carrying the image to be classified; receiving the image
  • the response information returned by the server, the response information includes a classification result of the image to be classified, and the classification result is obtained by the server classifying the image to be classified according to the classification model and the feature extraction model.
  • the processor 901 is further configured to: after receiving the response information returned from the server, display a user interface, where the user interface displays: a classification result, an associated image, and a description of the image to be classified. Any one or more of the information, the classification result and the associated image are all carried in the response information, and the description information is obtained by querying according to the classification result.
  • the specific implementation of the processor 901 may refer to the description of related content in the foregoing embodiment corresponding to FIG. 4.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明实施例公开了一种图像分类方法及服务器、用户终端、存储介质，其中所述方法包括：获取待分类图像，并调用分类模型对待分类图像进行分类，确定出待分类图像所属的初始类别，调用特征提取模型确定出待分类图像的目标特征向量，进而将待分类图像的目标特征向量与为初始类别配置的候选分类特征向量进行对比，得到对比结果，并根据对比结果确定出待分类图像的分类结果。采用本发明实施例可有效提高图像分类的正确率。

Description

一种图像分类方法及服务器、用户终端、存储介质
本申请要求于2018年2月7日提交中国专利局、申请号为201810124834.1、发明名称为“一种图像分类方法、装置及服务器、用户终端、存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，尤其涉及一种图像分类方法及服务器、用户终端、存储介质。
发明背景
在人们的生活中，经常会遇到各种未知的图像，对于非专业人员而言，根本无法凭借肉眼识别出这些图像所属的类别，很可能会出现“指鹿为马”的尴尬。
随着图像识别技术的发展，使得人们摆脱了传统人眼识别图像的方式，极大地节省了识别图像的时间并提高了识别的效率。但如何更加快速、准确的对各种未知图像进分类，仍是一个研究的热点。
发明内容
本发明实施例提供了一种图像分类方法及服务器、用户终端、存储介质，可快捷准确地对图像进行分类。
一方面，本发明实施例提供了一种图像分类方法，包括：
获取待分类图像；
调用分类模型对所述待分类图像进行分类，确定出所述待分类图像所属的初始类别；
调用特征提取模型确定出所述待分类图像的目标特征向量；
将所述待分类图像的目标特征向量与为所述初始类别配置的候选分类特征向量进行对比，得到对比结果；
根据对比结果确定出所述待分类图像的分类结果。
另一方面，本发明实施例提供了另一种图像分类方法，包括：
检测到针对识别按钮的触发操作时，调用摄像组件获取当前预览图像；
如果检测到所述当前预览图像处于稳定状态，则将所述当前预览图像确定为待分类图像；
向服务器发送分类请求，所述分类请求中携带有所述待分类图像；
接收所述服务器返回的响应信息，所述响应信息包括所述待分类图像的分类结果，所述分类结果是所述服务器根据分类模型和特征提取模型对所述待分类图像进行分类确定后得到的。
相应地，本发明实施例还提供了一种服务器，包括：处理器和存储装置；所述存储装置，用于存储程序指令；所述处理器，调用所述程序指令，用于执行：获取待分类图像；调用分类模型对所述待分类图像进行分类，确定出所述待分类图像所属的初始类别；调用特征提取模型确定出所述待分类图像的目标特征向量；将所述待分类图像的目标特征向量与为所述初始类别配置的候选分类特征向量进行对比，得到对比结果；根据对比结果确定出所述待分类图像的分类结果。
相应地，本发明实施例还提供了一种用户终端，包括：处理器和存储装置；所述存储装置，用于存储程序指令；所述处理器，调用所述程序指令，用于执行：检测到针对识别按钮的触发操作时，调用摄像组件获取当前预览图像；如果检测到所述当前预览图像处于稳定状态，则将所述当前预览图像确定为待分类图像；向服务器发送分类请求，所述分类请求中携 带有所述待分类图像；接收所述服务器返回的响应信息，所述响应信息包括所述待分类图像的分类结果，所述分类结果是所述服务器根据分类模型和特征提取模型对所述待分类图像进行分类确定后得到的。
相应地，本发明实施例还提供了一种计算机存储介质，该计算机存储介质中存储有程序指令，该程序指令被执行时，用于实现上述的各方法。
本发明实施例能够获取待分类图像，并调用分类模型对待分类图像进行初分类，再调用特征提取模型确定出待分类图像的目标特征向量，将待分类图像的目标特征向量仅仅与为初分类中配置的候选分类特征向量进行对比，并根据对比结果确定出待分类图像的分类结果，可有效地提高图像分类的正确率，并且不需要与过多类别的候选分类特征向量进行对比，提高了运算效率，节省了软硬件资源。
附图简要说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例的一种用于进行图像分类的框架示意图；
图2a是本发明实施例的一种初始分类模型的结构示意图；
图2b是本发明实施例的一种初始特征分类模型的结构示意图；
图2c是本发明实施例的特征提取网络与特征表示优化模块之间的对应关系的示意图；
图2d是本发明实施例的训练白化参数的示意图；
图3是本发明实施例的一种图像分类方法的流程示意图；
图4是本发明实施例的另一种图像分类方法的流程示意图；
图5是本发明实施例的一种用户界面的示意图；
图6是本发明实施例的一种图像分类装置的结构示意图；
图7是本发明实施例的一种服务器的结构示意图；
图8是本发明实施例的另一种图像分类装置的结构示意图；
图9是本发明实施例的一种用户终端的结构示意图。
实施本发明的方式
为使本发明的技术方案和优点更加清楚，下面将结合附图对本发明实施方式作进一步地详细描述。
在本发明实施例中，一方面，通过完成了分类训练优化的分类模型对待分类的图像进行分类，确定出初始类别，该分类模型可以通过专用的分类训练方式进行训练优化，并且预先配置在服务器等具有分类功能的设备中能够较好地对图像进行多类别分类。一方面，通过完成了特征提取优化的特征提取模型来从待分类图像中提取更为精细准确的目标特征向量，基于该目标特征向量与初始类别下的一个或者多个候选图像的候选分类特征向量进行比较，最终确定出待分类图像的类别，得到分类结果。这样由分类模型进行初始分类，再通过更精细的特征向量在初始分类中进行细分，使得确认出来的类别更准确，并且特征比对的数据处理量也有效地减少，节省了软硬件资源，提高了处理效率。
在一个实施例中，对于上述的分类模型，可以采用通用分类模型，通过通用分类模型，可以对图像进行较为粗略的分类，例如可以将某个图像的内容分为猫，或者是狗等类别。该分类模型也可以采用基于强监督信息的细粒度图像分类模型或者基于弱监督信息的细粒度图像分类模型。基于细粒度图像分类模型，可以进行更为细化的分类，例如可以细分到猫的品 种、狗的品种等。
基于强监督信息的细粒度图像分类模型是指：在模型训练优化时，为了获得更好的分类精度，除了图像的类别标签外，还使用了物体标注框和部位标注点等额外的人工标注信息，这种类型的分类模型可以实现较高的分类精度。
基于弱监督信息的细粒度图像分类模型是指：在不借助部位标注点的情况下，也可以做到较好的局部信息捕捉。相较于强监督分类精度有差距，但是实现成本低，更适合于工程应用。
本发明实施例主要包括对分类模型和特征提取模型的配置阶段，以及基于分类模型和特征提取模型的图像分类阶段。在配置阶段，可以基于配置服务器来离线建立初始分类模型和初始特征提取模型，并通过大量的训练图像对初始分类模型和初始特征提取模型进行训练优化，得到较优的能够对图像进行分类的分类模型和特征提取模型。将进行训练优化后的分类模型和特征提取模型加载到用于图像分类的服务器(或者其他一些用于进行图像分类的设备)上，以便于在线为用户提供对未知图像的分类服务。在图像分类阶段，用户只需通过用户终端拍摄待分类图像或者通过其他方式(例如下载等方式)获取待分类图像，并将该待分类图像携带在分类请求中发送至用于进行图像分类的服务器，由该服务器调用分类模型和特征提取模型对所述待分类图像进行处理，确定出待分类图像的分类结果，并向用户终端返回对该分类请求的响应信息，该响应信息中携带有对待分类图像的分类结果。
在一个实施例中，图1示出了本发明实施例的一种用于进行图像分类的框架示意图，该框架包括用户侧和服务侧，其中，用户侧的用户终端可以为智能手机、平板电脑等带摄像功能和网络功能的智能终端，当然也可以为个人电脑等终端。服务侧的在线服务可以由一个能够接收目标应用的 分类请求，用于进行图像分类的服务器或者服务器组来提供，服务侧的离线处理服务可以由某些用于设计和训练分类模型和特征提取模型的服务器或者服务器组来提供。在其他实施例中，服务侧的在线服务和离线服务也可以通过一个专用的服务器或者服务器组来提供。在一个实施例中，该目标应用可以为具有扫描识别模块的应用。
如图2a所示，示出了一种初始分类模型。该初始分类模型包括：Input层，Stem层，Inception-resnet层，Reduction层，Average pooling层，Dropout层，Softmax层。其中，Input层为输入层，可以用于输入图像；Inception-resnet层为隐含层，该初始分类模型拥有三个隐含层，分别为：Inception-resnet-A第一隐含层、Inception-resnet-B第二隐含层以及Inception-resnet-C第三隐含层；Stem层为预处理层，可以用于对输入Inception-resnet-A的数据进行预处理，该预处理可以包括数据进行多次卷积和池化；Average Pooling层为平均池化层，可以用于对Inception-resnet-C输出的数据进行降维处理；Dropout层，可以用于防止初始分类模型出现过拟合的情况，有效避免初始分类模型对训练图像能够很好的进行分类，但在部署后对实际需要分类的图像，则分类效果较差的情况；Softmax层为分类计算层，它的输出结果可以是通过Input层输入的图像属于各个类别的概率。
在一个实施例中，在配置上述分类模型时，可以利用模型生成工具预先将多个图像类别的训练图像输入至初始分类模型，对初始分类模型进行训练。在一个实施例中，当使用的模型生成工具为Tensorflow(一种用于图像识别等领域的深度学习框架)工具时，由于Tensorflow工具上的反向梯度是自动计算的，因此可以在训练优化过程中，快速地根据输入的训练图像调整以及生成该初始分类模型中每一个节点对应的参数，从而实现对初始分类模型的训练优化，得到分类模型。其中，该训练图像的类别可以根据不同的设计目标进行调整。
在一个实施例中，设计目标为建立分类模型，该分类模型可以用于对第一类别(例如猫类)和第二类别(例如狗类)图像进行区分。针对这种情况，在对初始分类模型进行训练时，可以预先选取M(M为正整数，如10000)个已经被确定类别为狗(以下为了描述方便，简称为“狗类别”或“狗类”)的图像作为初始分类模型的狗类训练图像和P个(P为正整数，如10000)已经被确定为猫类别的图像作为初始分类模型的猫类训练图像。在一个实施例中，当某一狗类训练图像被输入至初始分类模型后，初始分类模型可以提取该狗类训练图像的图像特征数据，并根据这些图像特征数据对狗类训练图像进行分类，如果输出的分类结果指示该狗类训练图像的类别也为狗，则表明分类网络模型对该狗类训练图像的分类是成功的。进一步地，针对M个已经被标注为狗类别的训练图像进行分类后，如果成功率大于预设成功率阈值(如90％)，则确定该初始分类模型能够很好地对狗类别的图像进行分类识别，否则，则可以调整该初始分类模型中每一个节点对应的参数，并通过调整后的分类模型再次对M个狗类训练图像进行分类。同理，可以采用同样的方式利用P个猫类训练图像对初始分类模型进行训练以及优化，如果最终对狗类训练图像和猫类训练图像的分类成功率均满足预设的成功率阈值，则对初始分类模型的训练完成，并将训练完成的初始分类模型作为本发明实施例中的分类模型。在其他实施例中，还可以设置更多的不同类别，通过获取大量的不同类别的训练图像，来对初始分类模型进行训练优化，使得最终得到的分类模型能够对各个类型的图像进行分类的成功率均高于某一成功率阈值。上述的第一类别和第二类别等类别，可以是诸如猫或者狗等类别，也可以是具体的猫或狗的品种等更细化的类别，例如可以是牧羊犬、柴犬等等细分类别。
在一个实施例中，在初始分类模型的训练优化过程中，可以将每个训练成功的训练图像确定为候选图，并根据其所述的类别为该候选图设置一 个其所属类别的类别标签，将该候选图以及对应的类别标签关联存储至数据库中，以便于后续基于该类别标签确定候选图所属的类别。
在本发明实施例中，上述特征提取模型可以是根据特征提取网络和特征表示优化模块构成的，该特征提取网络用于基于神经网络提取待分类图像的初始特征向量，该特征表示优化模块用于对初始特征向量进行优化得到N维的目标特征向量，N为正整数，该特征提取模型最终输出N维的关于图像的目标特征向量，例如输出2048维目标特征向量。
在一个实施例中，为了生成上述特征提取模型，可以获取初始特征分类模型和训练图像，该初始特征分类模型中包括基于神经网络构成的初始特征提取网络。进一步地，可以根据获取到的训练图像对初始特征分类模型进行训练优化，得到特征分类模型，该特征分类模型中包括对初始特征提取网络进行训练优化后得到的特征提取网络，进而可以从该特征分类模型中获取特征提取网络，基于白化参数生成特征表示优化模块，并根据该特征提取网络和该特征表示优化模块生成特征提取模型。在一个实施例中，该特征表示优化模块可以基于R-MAC(一种图像特征提取方法)来实现。
在一个实施例中，该特征表示优化模块可以是基于训练图像对白化参数进行训练优化后得到的，该白化参数可以由矩阵w和偏置矩阵b构成，该特征表示优化模块可以用于对特征提取网络输出的特征向量进行优化，并输出优化后的特征向量。该特征表示优化模块的处理过程可以是一个白化处理的过程。在一个实施例中，白化参数的训练优化过程包括：可以首先调用特征提取网络从多个训练图像中分别提取出特征向量，并根据获取到的特征向量进行计算来优化得到矩阵w和偏置矩阵b，进而得到特征表示优化模块，该特征表示优化模块基于矩阵w和偏置矩阵b生成。例如，有100万的训练样本，即存在100万个训练图像，针对每一个训练图像会 得到2048维的特征表示训练矩阵x，需要计算出一个2048*2048的矩阵w和2048的偏置矩阵b来白化这个2048维的特征表示训练矩阵x，使得2048维的特征表示训练矩阵x在白化后的矩阵中，维度数据之间不相关，也就是知道某个维度数据的值之后对猜测或者计算别的维度的值没有帮助。具体来说，如图2d所示，图2d中左侧的图像为100万*2048维的二维向量，可以看出左侧的图像中，维度数据之间比较密集，维度数据的值之间存在一定的相关性，通过数学计算的方式，不断计算出矩阵w和偏置矩阵b，最终使得左侧的二维向量通过xw＝x*w+b的计算公式计算后，得到右侧的图像。可以看出，右侧的图像中，各个维度数据之间的相关性低。在理论上是可以直接计算出w和b的，但是往往由于样本数量太大，而是通过训练的方式去趋近一个w和b，使得输入的二维向量经过w和b后，能够从左侧的图像转换为右侧的图像。其中，2048维是指特征表示优化模块需要输出的维度量，如果存在其他维度量的需求，例如需要输出4096维甚至更多维度量的数据，则在训练w和b，生成特征表示优化模块的过程中，上述的2048需要调整至4096维甚至更多维度量的数值。
在一个实施例中，该特征表示优化模块是根据一个转换公式生成的，该转换公式为：xw＝x*w+b。也即当一个初始矩阵x输入特征表示优化模块后，该特征表示优化模块可以根据该转换关系输出优化后的矩阵xw＝x*w+b。同样的，当特征提取网络将特征向量输入至特征表示优化模块后，该特征表示优化模块也可以采用相似的方式，输出优化后的特征向量。采用这样的优化处理方式，可以使优化处理后的特征向量中，特征值所表示的特征之间相关性低，例如需要低于一定的相关性阈值，各特征具有相同或者相近似的方差。
在一个实施例中，上述特征分类模型和特征表示优化模块均可以是利用模型生成工具对初始特征分类模型和白化参数进行训练而生成得到。具 体地，所使用的模型生成工具也可以为Tensorflow工具，由于在Tensorflow上的反向梯度是自动计算的，可以较为快捷地得到特征分类模型和特征表示优化模块。
如图2b所示，示出了一种初始特征分类模型，该初始特征分类模型包括基于神经网络构成的初始特征提取网络，该神经网络可以采用卷积神经网络。从图中可以看出，该初始特征提取网络包括三个卷积层：第一卷积层cfgl[0]block、第二卷积层cfgl[1]block、第三卷积层cfgl[2]block，在其他实施例中，还可以包括更多的卷积层，例如在cfgl[2]block之后，还可以包括第四卷积层cfgl[3]block等。在训练优化的过程中，初始特征提取网络中的每一个卷积层对输入的训练图像进行卷积处理，并输出关于输入的训练图像的特征向量(即训练图像的初始分类特征向量)。其中，初始特征提取网络包括第一类型的卷积层和第二类型的卷积层，图2b中，cfgl[0]block、cfgl[1]block为第一类型的卷积层，cfgl[0]block将卷积处理后的数据发送给cfgl[1]block，而cfgl[1]block则将卷积处理后的数据发送给cfgl[2]block。图2b所示的cfgl[2]block为第二类型的卷积层，cfgl[2]block将卷积处理后的数据(即训练图像的初始分类特征向量)输出，以便于后续的网络层对初始分类特征向量进行计算，确定图像的类别。在对该初始特征分类模型训练结束得到特征分类模型后，从训练得到的特征分类模型中提取出用于计算分类特征向量的网络层，将这些网络层作为特征提取网络，例如将cfgl[0]block、cfgl[1]block以及cfgl[2]block、cfgl[3]block作为特征提取网络。
在本发明实施例中，对图2b示出的初始特征分类模型进行训练优化，得到特征分类模型后，可以从特征分类模型中获取训练优化后的初始特征提取网络(即从特征分类模型中获取特征提取网络)，并建立特征提取网络与特征表示优化模块的连接关系如图2c所示。从图中可以看出，在图 像输入特征提取网络后，特征提取网络中的每一个卷积层(cfgl[0]block、cfgl[1]block以及cfgl[2]block)均可以对输入的图像进行卷积处理，进而确定出输入图像的初始分类特征向量，并通过最后一个卷积层cfgl[2]block将输入图像的初始分类特征向量发送至特征表示优化模块，进而利用特征表示优化模块对输入图像的初始分类特征向量进行优化处理，确定并输出所述输入图像的候选分类特征向量。其中，该优化处理可以为对所述输入图像的初始分类特征向量进行多维度的向量标识，例如2048维的向量表示，也即优化后的特征向量(即候选分类特征向量)可以为一个2048维的向量，该2048维的特征数据的表现形式例如可以是(0.1、0.11、0.15、……、0.16)。可以理解的是，基于不同的特征表示优化模块，还可以得到其它维数的向量，所使用向量的维度越高，对所述输入图像的分类结果也越准确。
在本发明实施例中，可以调用分类模型对各种训练图像进行初始分类，并将分类成功的训练图像作为候选图存储在数据库中。进一步地，可以将候选图作为特征提取网络的输入，调用特征提取模型计算得到全部候选图的候选分类特征向量，为全部候选图生成各自对应的候选分类特征向量，并将候选图和候选分类特征向量进行关联存储，以便于后续在接收到用户关于图像的分类请求时，能够直接基于这些候选分类特征向量确定出所需分类图像的分类结果并返回给用户终端，而不需要在接收到关于图像的分类请求后，再调用特征提取模型对海量的候选图进行计算得到候选分类特征向量，节省大量计算以及查询时间。
在一个实施例中，在将候选图和候选分类特征向量进行关联存储时，可以建立候选图与候选分类特征向量的映射关系列表，列1为候选图的存储地址，列2为候选分类特征向量，当查找到候选分类特征向量时，便可以根据该映射关系列表，快速查找到该候选分类特征向量对应的候选图， 反之亦然。
在通过大量的训练图像完成对分类模型和特征提取模型进行训练以及相关参数的优化后，即可将完成了训练优化后的分类模型和特征提取模型配置到相应的服务器中，并将大量的候选图和候选分类特征向量关联存储在相应的服务器中，为用户提供在线的图像分类服务。
在一个实施例中，将大量的候选图和候选分类特征向量关联存储在相应的服务器时，还可以根据候选图所属的类别，对不同的类别配置不同的候选分类特征向量。具体地，在某一类别下存储有L(L为正整数)个候选图时，可以直接为该类别配置L个候选分类特征向量，也即是每个候选图对应一个候选分类特征向量。或者，也可以对该类别下各个候选图的候选分类特征向量进行相似度计算，并将相似度小于第一相似度阈值的各个候选分类特征向量归为同一类的特征向量，令属于同一类的候选分类特征向量所对应的各个候选图均使用同一个候选分类特征向量来表征，也即多个候选图均对应同一个候选分类特征向量，进而为该类别配置小于L个的候选分类特征向量。采用这样的方式，可以在保证图像分类准确度的同时，减少为各个类别配置候选分类特征向量的数量，进而提高运算速率。在一个实施例中，候选图可以是前述采用的各种训练图像，也可以是用户通过下载或者拍摄等方式获取到的相应类型的图像，这些图像已知类别，并且通过上述提到的特征提取模型从中提取了候选分类特征向量，这些图像和对应的候选分类特征向量均可以存到数据库中，方便后续查找。
可以理解的是，在本发明实施例中描述了可以通过服务器来训练、配置分类模型和特征提取模型，在其他实施例中，也可以通过功能强大、软硬件资源丰富的个人电脑来实现，本发明实施例对此不作具体限定。
再请参见图3，是本发明实施例的一种图像分类的流程示意图，本发明实施例的所述方法可以由一个服务器或服务器组来执行。本发明实施例 的所述方法包括如下步骤。
S301：获取待分类图像。具体实现中，服务器可以接收用户终端发送的分类请求，并从该分类请求中获取携带的待分类图像。该待分类图像可以为用户利用用户终端的拍摄模块获取得到的，也可以为采用其他方式获取得到的，本发明实施例对此不作具体限定。
S302：调用分类模型对待分类图像进行分类，确定出待分类图像所属的初始类别。在一个实施例中，该分类模型可以是开发用户基于初始分类模型进行训练优化得到的，该分类模型有着TOP5准确率高的特点，也即该分类模型可以比较准确的确定出待分类图像所属的类别(即初始类别)，例如，当待分类图像对应的物体为一只波斯猫，那么分类模型则可以比较准确的确定出待分类图像所属的初始类别为猫类，而非狗类。
在一个实施例中，该分类模型由初始分类模型经过大量的训练优化得到。初始分类模型可以如图2a所示，对该初始分类模型进行训练优化，得到分类模型的过程，可以参见上述的相关描述，此处不再赘述。
在一个实施例中，待分类图像输入分类模型后，分类模型可以提取待分类图像的特征数据，并根据这些特征数据计算出待分类图像属于各个候选类别的概率，进而可以将概率排名靠前的K(K为正整数)个候选类别，确定为待分类图像所属的初始类别。
S303：调用特征提取模型确定出待分类图像的目标特征向量。其中，该特征提取模型可以是根据特征提取网络和特征表示优化模块构成的，该特征提取网络用于基于神经网络提取待分类图像的初始特征向量，该特征表示模块用于对该初始特征向量进行优化得到N维的目标特征向量，N为正整数。
在一个实施例中，为了生成上述特征提取模型，可以获取初始特征分类模型和训练图像，该初始特征分类模型中包括基于神经网络构成的初始 特征提取网络。进一步地，可以根据获取到的训练图像对初始特征分类模型进行训练优化，得到特征分类模型，该特征分类模型中包括对初始特征提取网络进行训练优化后得到的特征提取网络，进而可以从该特征分类模型中获取特征提取网络，并根据特征提取网络和预先生成的特征表示优化模块生成特征提取模型。其中，该特征表示优化模块可以是基于训练图像对白化参数进行训练优化得到的，该特征表示优化模块可以用于对特征提取网络输出的特征向量进行优化，并输出优化后的特征向量。
在一个实施例中，上述特征提取网络与特征表示优化模块的连接关系可以如图2c所示。其中，该特征提取网络可以是开发用户基于神经网络构成的，该特征提取网络可包括第一类型的卷积层和第二类型的卷积层，第一类型的卷积层将卷积处理的数据输出给特征提取网络中的另一个卷积层，第二类型的卷积层将卷积处理后的数据输出给特征表示优化模块。在待分类图像输入特征提取网络后，可以首先调用特征提取网络对待分类图像进行处理，可以得到关于待分类图像的一个初始特征向量，并通过第二类型的卷积层将初始特征向量输入特征表示优化模块，进而调用特征表示优化模块对初始特征向量进行优化，得到一个关于待分类图像的N维的目标特征向量，例如一个2048维的向量，该目标特征向量能够在一定程度上表示该待分类图像，该目标特征向量为计算机理解的图像，用户不易理解，不易可视化。
S304：将待分类图像的目标特征向量与为初始类别配置的候选分类特征向量进行对比，得到对比结果。比较的方式可以为计算目标特征向量和候选分类特征向量中，每一维向量之间的相似度，例如计算每一维向量之间的相似度汉明距离或者欧氏距离。最终确定的相似度越高，则待分类图像属于对应的候选分类特征向量所属的类别的概率越大。
S305：根据对比结果确定出待分类图像的分类结果。
在一个实施例中，该初始类别下可以包括至少一个候选图，服务器或者服务器组可以获取属于初始类别的候选图，并根据候选图，调用特征提取模型为初始类别配置候选分类特征向量。
在一个实施例中，为初始类别配置的候选分类特征向量可以是通过特征提取模型中的特征提取网络对候选图进行处理，得到初始类别下的初始分类特征向量，并通过特征提取模型中的特征表示优化模块对初始类别下的初始分类特征向量进行优化得到的。该候选分类特征向量可以为一个N维的特征向量，例如可以为2048维的向量。
初始类别下可以包括Q个候选图(每个候选图对应一个候选分类特征向量)。针对这种情况，在调用特征提取模型为初始类别配置候选分类特征向量时，可以直接为初始类别配置Q个候选分类特征向量，也即是每个候选图对应一个候选分类特征向量。或者，也可以对初始类别下各个候选图的候选分类特征向量进行相似度计算，并将相似度小于第一相似度阈值的候选分类特征向量归为同一类，属于同一类的候选分类特征向量所对应的各个候选图均使用同一个候选分类特征向量来表征，也即多个候选图均对应同一个候选分类特征向量，进而为初始类别配置小于Q个的候选分类特征向量。采用这样的方式，可以在保证图像分类精度的同时，减少为初始类别配置候选分类特征向量的数量，进而减少计算量，以此提高运算速率。
在一个实施例中，该初始类别可以包括第一类别和第二类别，服务器可以对待分类图像的目标特征向量与为第一类别配置的候选分类特征向量进行相似度计算，得到第一相似度；对目标特征向量与为第二类别配置的候选分类特征向量进行相似度计算，得到第二相似度，并将第一相似度和第二相似度进行对比，得到对比结果，该对比结果指示了第一相似度和第二相似度之间的较大值。进一步地，如果对比结果指示第一相似度大于 第二相似度，则可以确定待分类图像属于第一类别。其中，该初始类别还可以包括第三类别、第四类别或者其它类别，本发明实施例对此不作具体限定。
例如，待分类图像的目标特征向量为(0.23、0.44、….、0.61)，初始类别为狗类，其中，狗类中包括的第一类别为金毛，第二类别为泰迪，金毛对应的候选分类特征向量为(0.23、0.44、….、0.67)，泰迪对应的候选分类特征向量为(0.23、0.31、….、0.60)，服务器通过对目标特征向量(0.23、0.41、….、0.61)与金毛对应的候选分类特征向量(0.23、0.44、….、0.67)进行相似度计算，得到第一相似度为98％，进一步地，对目标特征向量(0.23、0.41、….、0.61)与泰迪对应的候选分类特征向量(0.23、0.31、….、0.60)进行相似度计算，得到第二相似度为90％，通过对比第一相似度和第二相似度的大小，确定对比结果为第一相似度大于第二相似度，则可以确定待分类图像属于金毛。
在一个实施例中，上述第一类别下和第二类别下均可以包括与候选分类特征向量关联的候选图，如果对比结果指示上述第一相似度大于上述第二相似度，则可以根据第一相似度确定得到所述待分类图像的关联图像，该关联图像是指：候选分类特征向量与目标特征向量之间的相似度为第一相似度的候选图。例如，待分类特征向量与目标特征向量之间的相似度为第一相似度的候选图为图像JPG1和图像JPG2，那么，则可以将图像JPG1和图像JPG2均确定为待分类图像的关联图像。
在一个实施例中，待分类图像的目标特征向量与为第一类别或者第二类别配置的候选分类特征向量进行相似度计算时，可以通过匹配算法计算该待分类图像的目标特征向量与第一类别下或者第二类别下候选分类特征向量之间的欧氏距离或余弦夹角，进而得到目标特征向量与各个候选图对应的候选分类特征向量之间的相似程度。在通过匹配算法计算相似度 时，还可以通过顺序遍历依次比较的方式进行匹配，当然也可以采用基于faiss(开源的一个用于有效的相似性搜索和稠密矢量聚类的库)搜索引擎来搜索候选分类特征向量并进行匹配。
在一个实施例中，当服务器确定出待分类图像的分类结果后，可以向用户终端返回针对上述分类请求的响应信息，该响应信息可以包括上述分类结果和上述关联图像中的至少一种。
再请参见图4，是本发明实施例的另一种图像分类方法的流程示意图，本发明实施例的所述方法可以由用户终端来执行。本发明实施例的所述方法包括如下步骤。
S401：检测到针对识别按钮的触发操作时，调用摄像组件获取当前预览图像。
S402：如果检测到当前预览图像处于稳定状态，则将当前预览图像确定为待分类图像。
在一个实施例中，用户终端安装有存在图像分类功能入口的目标应用，该图像分类功能入口可以包括扫描识别模块。当用户想要识别某个物体时，可以针对该扫描识别模块所提供的识别按钮输入触发操作，用户终端检测到该触发操作时，即可产生扫描识别操作事件，基于该事件可以调用摄像组件获取当前预览图像。进一步地，如果检测到当前预览图像处于稳定状态，则可以将处于稳定状态的当前预览图像作为待分类图像，或者直接导入已存储的图像作为待分类图像。其中，该目标应用例如可以为浏览器，即时通讯应用、支付应用等具有图像分类功能入口的应用。
在一个实施例中，可以按照预设时间间隔获取当前预览图像，在确定当前预览图像是否处于稳定状态时，可以将当前预览图像与前T次获取的预览图像进行相似度对比，如果相似度大于或者等于相似度阈值，则可以确定当前预览图像处于稳定状态。其中T为大于0的正整数，T的具体值 可以根据不同设计需求进行相应调整，本发明实施例对此不作具体限定。
在一个实施例中，如图5所示，目标应用为浏览器，在点击了浏览器上的扫描识别按钮501后，用户终端进入图像获取界面，在图像获取界面上包括图像显示区域502和提示导入图像的“+”按钮503，用户点击按钮503可以进入图像选择和确认界面。当检测到图像显示区域502所显示的预览图像处于稳定状态时，则将处于稳定状态的预览图像确定为待分类图像。
S403：向服务器发送分类请求，该分类请求中携带有待分类图像。在得到了待分类图像后，用户终端可以生成携带该待分类图像的分类请求，然后将分类请求发送给提供在线分类服务的服务器。
S404：接收服务器返回的响应信息。该响应信息包括待分类图像的分类结果，该分类结果可以是服务器根据分类模型和特征提取模型对待分类图像进行分类确定后得到的。在一个实施例中，服务器接收到该分类请求后，可以响应该分类请求，确定出待分类图像的分类结果，并将该分类结果携带在响应信息中返回至用户终端。进一步地，用户终端接收到服务器返回的携带有分类结果的响应信息时，可以将该分类结果显示在用户界面上，以便于用户查看。其中，该响应信息除了可以携带分类结果以外还可以包括待分类图像的关联图像或者其它信息，本发明实施例对此不作具体限定。
在一个实施例中，用户终端在接收到从服务器返回的响应信息后，可以显示用户界面，该用户界面显示有：待分类图像的分类结果、关联图像和描述信息中的任意一个或者多个，该分类结果和关联图像可以是响应信息中携带的，该描述信息可以是根据分类结果查询得到的。其中，该描述信息可以为与分类结果相关联的文本信息和图像信息。例如，分类结果显示待分类图像中包括的物体为X系越野车01，那么该描述信息则可以包 括X系越野车01的基本介绍，例如性能列表、上市时间、价格、附近售卖该X系越野车01的4S店等等，还可以包括售卖该X系越野车01的平台网址等等，本发明实施例对此不作具体介绍。
在一个实施例中，用户终端在接收到响应信息后，可以直接基于响应信息中携带的分类结果在线搜索，进而获得待分类图像的描述信息，并将该描述信息显示在用户界面上。或者，可以在接收到针对用户界面的触发操作时，再在线搜索或者本地获取该分类结果的描述信息，并将获取到的该描述信息显示在用户界面上。
需要说明的是，上述用户界面显示的描述信息除了可以是通过分类结果查询得到以外，也可以是响应信息携带有描述信息，用户终端从该响应信息中获取得到的，本发明实施例对此不作具体限定。
本发明实施例还提供了一种计算机存储介质，该计算机存储介质中存储有程序指令，该程序指令被执行时，用于实现上述实施例中描述的相应方法。
再请参见图6，是本发明实施例的一种图像分类装置的结构示意图，该图像分类装置可以设置在服务器中，或者也可以设置在一些软硬件资源较为丰富的智能终端中，例如一些个人电脑中。
本发明实施例的所述装置的一个实现方式中，所述装置包括如下结构。
获取模块601，用于获取待分类图像；
调用模块602，用于调用分类模型对所述待分类图像进行分类，确定出所述待分类图像所属的初始类别；
所述调用模块602，还用于调用特征提取模型确定出所述待分类图像的目标特征向量；
对比模块603，用于将所述待分类图像的目标特征向量与为所述初始 类别配置的候选分类特征向量进行对比，得到对比结果；
确定模块604，用于根据对比结果确定出所述待分类图像的分类结果。
在一个实施例中，所述特征提取模型是根据特征提取网络和特征表示优化模块构成的，所述特征提取网络用于基于神经网络提取待分类图像的初始特征向量，所述特征表示优化模块用于对所述初始特征向量进行优化得到N维的目标特征向量，N为正整数。
在一个实施例中，所述装置还可以包括：训练模块605，生成模块606，其中：获取模块601，还用于获取初始特征分类模型和训练图像，所述初始特征分类模型中包括基于神经网络构成的初始特征提取网络；训练模块605，用于根据获取模块601获取到的所述训练图像对所述初始特征分类模型进行训练优化，得到特征分类模型，其中，所述特征分类模型中包括对所述初始特征提取网络进行训练优化后，得到的特征提取网络；获取模块601，还用于从所述特征分类模型中获取所述特征提取网络；生成模块606，用于基于白化参数生成特征表示优化模块；生成模块606，还用于根据所述特征提取网络和所述特征表示优化模块，生成特征提取模型，其中，所述特征表示优化模块用于对所述特征提取网络输出的特征向量进行优化，并用于输出优化后的特征向量。
在一个实施例中，所述装置还可以包括：配置模块607，其中：获取模块601，还用于获取属于所述初始类别的候选图；配置模块607，用于根据获取模块601获取到的所述候选图，调用所述特征提取模型为所述初始类别配置候选分类特征向量。
在一个实施例中，配置模块607，可以具体用于通过所述特征提取模型中的特征提取网络对所述候选图进行处理，得到所述初始类别下的初始分类特征向量，并通过所述特征提取模型中的特征表示优化模块对所述初始类别下的初始分类特征向量进行优化，得到所述初始类别下的N维的候 选分类特征向量。
在一个实施例中，所述初始类别包括第一类别和第二类别，所述对比模块603，可以包括：计算单元6031，用于对所述待分类图像的目标特征向量与为所述第一类别配置的候选分类特征向量进行相似度计算，得到第一相似度；对目标特征向量与为所述第二类别配置的候选分类特征向量进行相似度计算，得到第二相似度；对比单元6032，用于将所述第一相似度和所述第二相似度进行对比，得到对比结果，所述对比结果指示了所述第一相似度和所述第二相似度之间的较大值。
在一个实施例中，确定模块604可以具体用于如果所述对比结果指示所述第一相似度大于所述第二相似度，则确定所述待分类图像属于所述第一类别。
在一个实施例中，所述第一类别下和所述第二类别下均包括与候选分类特征向量关联的候选图，确定模块604还可以用于如果所述对比结果指示所述第一相似度大于所述第二相似度，则根据所述第一相似度确定得到所述待分类图像的关联图像，所述关联图像是指：所述候选分类特征向量与所述目标特征向量之间的相似度为第一相似度的候选图。
在本发明实施例中，上述各个模块的具体实现可参考前述附图3所对应的实施例中相关内容的描述。
再请参见图7，是本发明实施例的一种服务器的结构示意图，本发明实施例的所述服务器包括供电模块等结构，并包括处理器701、存储装置702以及网络接口703。所述处理器701、存储装置702以及网络接口703之间可以交互数据，由处理器701实现相应的图像分类功能。
所述存储装置702可以包括易失性存储器(volatile memory)，例如随机存取存储器(random-access memory，RAM)；存储装置702也可以包括非易失性存储器(non-volatile memory)，例如快闪存储器(flash  memory)，固态硬盘(solid-state drive，SSD)等；所述存储装置702还可以包括上述种类的存储器的组合。
所述网络接口703可以与其他服务器、各种用户终端之间交互数据，用户终端可以将携带待分类图像的分类请求发送给所述网络接口703，由所述网络接口703输出给服务器的处理器701进行处理。
所述处理器701可以是中央处理器(central processing unit，CPU)。在一个实施例中，所述处理器701还可以是图形处理器701(Graphics Processing Unit，GPU)。所述处理器701也可以是由CPU和GPU的组合。在所述服务器中，可以根据需要包括多个CPU和GPU进行相应的图像处理。在一个实施例中，所述存储装置702用于存储程序指令。所述处理器701可以调用所述程序指令，实现如本发明实施例中上述涉及的各种方法。
在第一个可能的实施方式中，所述服务器的所述处理器701，调用所述存储装置702中存储的程序指令，用于获取待分类图像；调用分类模型对所述待分类图像进行分类，确定出所述待分类图像所属的初始类别；调用特征提取模型确定出所述待分类图像的目标特征向量；将所述待分类图像的目标特征向量与为所述初始类别配置的候选分类特征向量进行对比，得到对比结果；根据对比结果确定出所述待分类图像的分类结果。
在一个实施例中，所述特征提取模型是根据特征提取网络和特征表示优化模块构成的，所述特征提取网络用于基于神经网络提取待分类图像的初始特征向量，所述特征表示优化模块用于对所述初始特征向量进行优化得到N维的目标特征向量，N为正整数。
在一个实施例中，所述处理器701，还用于获取初始特征分类模型，所述初始特征分类模型中包括基于神经网络构成的初始特征提取网络；获取训练图像，并根据所述训练图像对所述初始特征分类模型进行训练优化，得到特征分类模型，其中，所述特征分类模型中包括对所述初始特征 提取网络进行训练优化后，得到的特征提取网络；从所述特征分类模型中获取所述特征提取网络；基于白化参数生成特征表示优化模块；根据所述特征提取网络和所述特征表示优化模块，生成特征提取模型，其中，所述特征表示优化模块用于对所述特征提取网络输出的特征向量进行优化，并用于输出优化后的特征向量。
在一个实施例中，所述处理器701，还用于获取属于所述初始类别的候选图；
根据所述候选图，调用所述特征提取模型为所述初始类别配置候选分类特征向量。
在一个实施例中，所述处理器701，还用于通过所述特征提取模型中的特征提取网络对所述候选图进行处理，得到所述初始类别下的初始分类特征向量，并通过所述特征提取模型中的特征表示优化模块对所述初始类别下的初始分类特征向量进行优化，得到所述初始类别下的N维的候选分类特征向量。
在一个实施例中，所述初始类别包括第一类别和第二类别，所述处理器701，还用于对所述待分类图像的目标特征向量与为所述第一类别配置的候选分类特征向量进行相似度计算，得到第一相似度；对目标特征向量与为所述第二类别配置的候选分类特征向量进行相似度计算，得到第二相似度；将所述第一相似度和所述第二相似度进行对比，得到对比结果，所述对比结果指示了所述第一相似度和所述第二相似度之间的较大值。
在一个实施例中，所述处理器701，还用于如果所述对比结果指示所述第一相似度大于所述第二相似度，则确定所述待分类图像属于所述第一类别。
在一个实施例中，所述第一类别下和所述第二类别下均包括与候选分类特征向量关联的候选图，所述处理器701，还用于如果所述对比结果指 示所述第一相似度大于所述第二相似度，则根据所述第一相似度确定得到所述待分类图像的关联图像，所述关联图像是指：所述候选分类特征向量与所述目标特征向量之间的相似度为第一相似度的候选图。
在本发明实施例中，所述处理器701的具体实现可参考前述附图3所对应的实施例中相关内容的描述。
再请参见图8，是本发明实施例的另一种图像分类装置的结构示意图，该图像分类装置可以设置在用户终端中。
本发明实施例的所述装置的一个实现方式中，所述装置包括如下结构。
检测模块801，用于检测针对识别按钮的触发操作；
获取模块802，用于所述检测模块检测到针对识别按钮的触发操作时，调用摄像组件获取当前预览图像；
所述检测模块801，还用于检测所述当前预览图像是否处于稳定状态；
确定模块803，用于如果所述检测模块检测到所述当前预览图像处于稳定状态，则将所述当前预览图像确定为待分类图像；
发送模块804，用于向服务器发送分类请求，所述分类请求中携带有所述待分类图像；
接收模块805，用于接收所述服务器返回的响应信息，所述响应信息包括所述待分类图像的分类结果，所述分类结果是所述服务器根据分类模型和特征提取模型对所述待分类图像进行分类确定后得到的。
在一个实施例中，该装置还可以包括：显示模块806，用于在接收到从服务器返回的响应信息后，显示用户界面，所述用户界面显示有：所述待分类图像的分类结果、关联图像和描述信息中的任意一个或者多个，所述分类结果和所述关联图像均是所述响应信息中携带的，所述描述信息是根据所述分类结果查询得到的。
在本发明实施例中，上述各个模块的具体实现可参考前述附图4所对应的实施例中相关内容的描述。
再请参见图9，是本发明实施例的一种用户终端的结构示意图，本发明实施例的所述用户终端可以包括供电模块等结构，并包括处理器901、存储装置902以及收发器903。所述处理器901、存储装置902以及收发器903之间可以交互数据，由处理器901实现相应的图像分类功能。
所述存储装置902可以包括易失性存储器(volatile memory)，例如随机存取存储器(random-access memory，RAM)；存储装置902也可以包括非易失性存储器(non-volatile memory)，例如快闪存储器(flash memory)，固态硬盘(solid-state drive，SSD)等；所述存储装置902还可以包括上述种类的存储器的组合。
所述收发器903可以与服务器、各种用户终端之间交互数据，服务器可以将携带有分类结果的响应信息发送给所述收发器903，由所述收发器903输出给用户终端的处理器901进行处理。
所述处理器901可以是中央处理器901(central processing unit，CPU)。在一个实施例中，所述处理器901还可以是图形处理器901(Graphics Processing Unit，GPU)。所述处理器901也可以是由CPU和GPU的组合。在所述用户终端中，可以根据需要包括多个CPU和GPU进行相应的图像处理。在一个实施例中，所述存储装置902用于存储程序指令。所述处理器901可以调用所述程序指令，实现如本发明实施例中上述涉及的各种方法。
在第一个可能的实施方式中，所述用户终端的所述处理器901，调用所述存储装置902中存储的程序指令，用于检测到针对识别按钮的触发操作时，调用摄像组件获取当前预览图像；如果检测到所述当前预览图像处于稳定状态，则将所述当前预览图像确定为待分类图像；向服务器发送分 类请求，所述分类请求中携带有所述待分类图像；接收所述服务器返回的响应信息，所述响应信息包括所述待分类图像的分类结果，所述分类结果是所述服务器根据分类模型和特征提取模型对所述待分类图像进行分类确定后得到的。
在一个实施例中，所述处理器901，还用于在接收到从服务器返回的响应信息后，显示用户界面，所述用户界面显示有：所述待分类图像的分类结果、关联图像和描述信息中的任意一个或者多个，所述分类结果和所述关联图像均是所述响应信息中携带的，所述描述信息是根据所述分类结果查询得到的。
在本发明实施例中，所述处理器901的具体实现可参考前述附图4所对应的实施例中相关内容的描述。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)或随机存储记忆体(Random Access Memory，RAM)等。
以上所揭露的仅为本发明的部分实施例而已，当然不能以此来限定本发明之权利范围，本领域普通技术人员可以理解实现上述实施例的全部或部分流程，并依本发明权利要求所作的等同变化，仍属于发明所涵盖的范围。

Claims (20)

  1. 一种图像分类方法，由一台服务器或服务器组执行，所述方法包括：
    获取待分类图像；
    调用分类模型对所述待分类图像进行分类，确定出所述待分类图像所属的初始类别；
    调用特征提取模型确定出所述待分类图像的目标特征向量；
    将所述待分类图像的目标特征向量与为所述初始类别配置的候选分类特征向量进行对比，得到对比结果；
    根据所述对比结果确定出所述待分类图像的分类结果。
  2. 根据权利要求1所述的方法，其中，所述方法还包括：
    根据特征提取网络和特征表示优化模块构成特征提取模型，所述特征提取网络用于基于神经网络提取待分类图像的初始特征向量，所述特征表示优化模块用于对所述初始特征向量进行优化得到N维的目标特征向量，N为正整数。
  3. 根据权利要求1或2所述的方法，其中，所述方法还包括：
    获取初始特征分类模型，所述初始特征分类模型中包括基于神经网络构成的初始特征提取网络；
    获取训练图像，并根据所述训练图像对所述初始特征分类模型进行训练优化，得到特征分类模型，其中，所述特征分类模型中包括对所述初始特征提取网络进行训练优化后，得到的特征提取网络；
    从所述特征分类模型中获取所述特征提取网络；
    基于白化参数生成特征表示优化模块，所述特征表示优化模块用于对所述特征提取网络输出的特征向量进行优化，并用于输出优化后的特征向 量；
    根据所述特征提取网络和所述特征表示优化模块，生成特征提取模型。
  4. 根据权利要求3所述的方法，其中，所述方法还包括：
    获取属于所述初始类别的候选图；
    根据所述候选图，调用所述特征提取模型为所述初始类别配置候选分类特征向量。
  5. 根据权利要求4所述的方法，其中，所述根据所述候选图，调用所述特征提取模型为所述初始类别配置候选分类特征向量包括：
    通过所述特征提取模型中的特征提取网络对所述候选图进行处理，得到所述初始类别下的初始分类特征向量；及
    通过所述特征提取模型中的特征表示优化模块对所述初始类别下的初始分类特征向量进行优化，得到所述初始类别下的N维的候选分类特征向量。
  6. 根据权利要求1或2所述的方法，其中，所述初始类别包括第一类别和第二类别，所述将所述待分类图像的目标特征向量与为所述初始类别配置的候选分类特征向量进行对比，得到对比结果，包括：
    对所述待分类图像的目标特征向量与为所述第一类别配置的候选分类特征向量进行相似度计算，得到第一相似度；
    对目标特征向量与为所述第二类别配置的候选分类特征向量进行相似度计算，得到第二相似度；
    将所述第一相似度和所述第二相似度进行对比，得到对比结果，所述对比结果指示了所述第一相似度和所述第二相似度之间的较大值。
  7. 根据权利要求6所述的方法，其中，所述根据所述对比结果确定出所述待分类图像的分类结果，包括：
    如果所述对比结果指示所述第一相似度大于所述第二相似度，则确定所述待分类图像属于所述第一类别。
  8. 根据权利要求6所述的方法，其中，所述第一类别下和所述第二类别下均包括与候选分类特征向量关联的候选图，所述方法还包括：
    如果所述对比结果指示所述第一相似度大于所述第二相似度，则根据所述第一相似度确定得到所述待分类图像的关联图像，所述关联图像是指：所述候选分类特征向量与所述目标特征向量之间的相似度为第一相似度的候选图。
  9. 一种图像分类方法，由用户终端执行，所述方法包括：
    检测到针对识别按钮的触发操作时，调用摄像组件获取当前预览图像；
    如果检测到所述当前预览图像处于稳定状态，则将所述当前预览图像确定为待分类图像；
    向服务器发送分类请求，所述分类请求中携带有所述待分类图像；
    接收所述服务器返回的响应信息，所述响应信息包括所述待分类图像的分类结果，所述分类结果是所述服务器根据分类模型和特征提取模型对所述待分类图像进行分类确定后得到的。
  10. 根据权利要求9所述的方法，其中，所述方法还包括：
    在接收到从服务器返回的响应信息后，显示用户界面，所述用户界面显示有：所述待分类图像的分类结果、关联图像和描述信息中的任意一个或者多个，所述分类结果和所述关联图像均是所述响应信息中携带的，所述描述信息是根据所述分类结果查询得到的。
  11. 一种服务器，包括处理器和存储装置，所述处理器和存储装置相互连接，其中，所述存储装置用于存储计算机程序，所述计算机程序包括程序指令，所述处理器被配置用于调用所述程序指令，执行如下步骤：
    获取待分类图像；
    调用分类模型对所述待分类图像进行分类，确定出所述待分类图像所属的初始类别；
    调用特征提取模型确定出所述待分类图像的目标特征向量；
    将所述待分类图像的目标特征向量与为所述初始类别配置的候选分类特征向量进行对比，得到对比结果；
    根据所述对比结果确定出所述待分类图像的分类结果。
  12. 根据权利要求11所述的服务器，其中，所述处理器被配置用于调用所述程序指令，进一步执行如下步骤：
    根据特征提取网络和特征表示优化模块构成特征提取模型，所述特征提取网络用于基于神经网络提取待分类图像的初始特征向量，所述特征表示优化模块用于对所述初始特征向量进行优化得到N维的目标特征向量，N为正整数。
  13. 根据权利要求11或12所述的服务器，其中，所述处理器被配置用于调用所述程序指令，进一步执行如下步骤：
    获取初始特征分类模型，所述初始特征分类模型中包括基于神经网络构成的初始特征提取网络；
    获取训练图像，并根据所述训练图像对所述初始特征分类模型进行训练优化，得到特征分类模型，其中，所述特征分类模型中包括对所述初始特征提取网络进行训练优化后，得到的特征提取网络；
    从所述特征分类模型中获取所述特征提取网络；
    基于白化参数生成特征表示优化模块，所述特征表示优化模块用于对所述特征提取网络输出的特征向量进行优化，并用于输出优化后的特征向量；
    根据所述特征提取网络和所述特征表示优化模块，生成特征提取模 型。
  14. 根据权利要求13所述的服务器，其中，所述处理器被配置用于调用所述程序指令，进一步执行如下步骤：
    获取属于所述初始类别的候选图；
    根据所述候选图，调用所述特征提取模型为所述初始类别配置候选分类特征向量。
  15. 根据权利要求14所述的服务器，其中，所述根据所述候选图，调用所述特征提取模型为所述初始类别配置候选分类特征向量包括：
    通过所述特征提取模型中的特征提取网络对所述候选图进行处理，得到所述初始类别下的初始分类特征向量；及
    通过所述特征提取模型中的特征表示优化模块对所述初始类别下的初始分类特征向量进行优化，得到所述初始类别下的N维的候选分类特征向量。
  16. 根据权利要求11或12所述的服务器，其中，所述初始类别包括第一类别和第二类别，所述将所述待分类图像的目标特征向量与为所述初始类别配置的候选分类特征向量进行对比，得到对比结果，包括：
    对所述待分类图像的目标特征向量与为所述第一类别配置的候选分类特征向量进行相似度计算，得到第一相似度；
    对目标特征向量与为所述第二类别配置的候选分类特征向量进行相似度计算，得到第二相似度；
    将所述第一相似度和所述第二相似度进行对比，得到对比结果，所述对比结果指示了所述第一相似度和所述第二相似度之间的较大值。
  17. 根据权利要求16所述的服务器，其中，所述根据所述对比结果确定出所述待分类图像的分类结果，包括：
    如果所述对比结果指示所述第一相似度大于所述第二相似度，则确定 所述待分类图像属于所述第一类别。
  18. 根据权利要求16所述的服务器，其中，所述第一类别下和所述第二类别下均包括与候选分类特征向量关联的候选图，所述处理器被配置用于调用所述程序指令，进一步执行如下步骤：
    如果所述对比结果指示所述第一相似度大于所述第二相似度，则根据所述第一相似度确定得到所述待分类图像的关联图像，所述关联图像是指：所述候选分类特征向量与所述目标特征向量之间的相似度为第一相似度的候选图。
  19. 一种用户终端，包括处理器和存储装置，所述处理器和存储装置相互连接，其中，所述存储装置用于存储计算机程序，所述计算机程序包括程序指令，所述处理器被配置用于调用所述程序指令，执行如权利要求9-10任一项所述的方法。
  20. 一种计算机存储介质，该计算机存储介质中存储有程序指令，该程序指令被执行时，用于实现如权利要求1-8任一项所述的方法或者实现如权利要求9-10任一项所述的方法。
PCT/CN2019/074149 2018-02-07 2019-01-31 一种图像分类方法及服务器、用户终端、存储介质 Ceased WO2019154262A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810124834.1A CN108304882B (zh) 2018-02-07 2018-02-07 一种图像分类方法、装置及服务器、用户终端、存储介质
CN201810124834.1 2018-02-07

Publications (1)

Publication Number Publication Date
WO2019154262A1 true WO2019154262A1 (zh) 2019-08-15

Family

ID=62864852

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/074149 Ceased WO2019154262A1 (zh) 2018-02-07 2019-01-31 一种图像分类方法及服务器、用户终端、存储介质

Country Status (2)

Country Link
CN (1) CN108304882B (zh)
WO (1) WO2019154262A1 (zh)

Cited By (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111046933A (zh) * 2019-12-03 2020-04-21 东软集团股份有限公司 图像分类方法、装置、存储介质及电子设备
CN111061898A (zh) * 2019-12-13 2020-04-24 Oppo（重庆）智能科技有限公司 图像处理方法、装置、计算机设备及存储介质
CN111078914A (zh) * 2019-12-18 2020-04-28 书行科技(北京)有限公司 重复图片检测方法与装置
CN111160218A (zh) * 2019-12-26 2020-05-15 浙江大华技术股份有限公司 一种特征向量比对方法、装置电子设备及存储介质
CN111209423A (zh) * 2020-01-07 2020-05-29 腾讯科技（深圳）有限公司 一种基于电子相册的图像管理方法、装置以及存储介质
CN111222557A (zh) * 2019-12-31 2020-06-02 Oppo广东移动通信有限公司 图像分类方法、装置、存储介质及电子设备
CN111222585A (zh) * 2020-01-15 2020-06-02 深圳前海微众银行股份有限公司 数据处理方法、装置、设备及介质
CN111339884A (zh) * 2020-02-19 2020-06-26 浙江大华技术股份有限公司 图像识别方法以及相关设备、装置
CN111382281A (zh) * 2020-03-04 2020-07-07 腾讯科技（深圳）有限公司 基于媒体对象的内容的推荐方法、装置、设备和存储介质
CN111401464A (zh) * 2020-03-25 2020-07-10 北京字节跳动网络技术有限公司 分类方法、装置、电子设备及计算机可读存储介质
CN111401428A (zh) * 2020-03-12 2020-07-10 Oppo广东移动通信有限公司 一种图像分类方法、装置、电子设备和存储介质
CN111488933A (zh) * 2020-04-13 2020-08-04 上海联影智能医疗科技有限公司 图像分类方法、网络、计算机设备和存储介质
CN111582372A (zh) * 2020-05-09 2020-08-25 西交利物浦大学 图像分类方法、模型、存储介质及电子设备
CN111639525A (zh) * 2020-04-22 2020-09-08 上海擎感智能科技有限公司 一种感知算法的训练方法、装置及计算机存储介质
CN111914879A (zh) * 2020-06-16 2020-11-10 北京迈格威科技有限公司 分类参数值生成方法、图像分类方法、装置、设备及存储介质
CN112016617A (zh) * 2020-08-27 2020-12-01 中国平安财产保险股份有限公司 细粒度分类方法、装置及计算机可读存储介质
CN112182269A (zh) * 2020-09-27 2021-01-05 北京达佳互联信息技术有限公司 图像分类模型的训练、图像分类方法、装置、设备及介质
CN112329933A (zh) * 2020-10-30 2021-02-05 腾讯科技（深圳）有限公司 数据处理方法、装置、服务器及存储介质
CN112396100A (zh) * 2020-11-16 2021-02-23 中保车服科技服务股份有限公司 一种细粒度分类模型的优化方法、系统及相关装置
CN112464015A (zh) * 2020-12-17 2021-03-09 郑州信大先进技术研究院 一种基于深度学习的图像电子证据筛选方法
CN112529078A (zh) * 2020-12-07 2021-03-19 杭州海康威视数字技术股份有限公司 一种业务处理方法、装置及设备
CN112749296A (zh) * 2019-10-31 2021-05-04 北京达佳互联信息技术有限公司 一种视频推荐方法、装置、服务器及存储介质
CN112948370A (zh) * 2019-11-26 2021-06-11 上海哔哩哔哩科技有限公司 数据分类方法、装置以及计算机设备
CN113849677A (zh) * 2021-09-28 2021-12-28 焦点科技股份有限公司 一种图片自动标注的方法及系统
CN113935407A (zh) * 2021-09-29 2022-01-14 光大科技有限公司 一种异常行为识别模型确定方法及装置
CN113963189A (zh) * 2020-07-03 2022-01-21 顺丰科技有限公司 物体分类方法、装置、电子设备及存储介质
CN114065826A (zh) * 2020-07-28 2022-02-18 紫东信息科技(苏州)有限公司 图像分类模型的构建方法、分类方法、装置及电子设备
CN114373192A (zh) * 2022-01-07 2022-04-19 杭州睿胜软件有限公司 根据动物视觉特征分类识别的方法、系统及可读存储介质
CN114780719A (zh) * 2022-03-28 2022-07-22 京东城市(北京)数字科技有限公司 文本分类模型的训练方法、文本分类方法及装置
CN115909374A (zh) * 2021-09-30 2023-04-04 腾讯科技（深圳）有限公司 一种信息识别方法、装置、设备及存储介质、程序产品
CN117009575A (zh) * 2022-11-30 2023-11-07 腾讯科技（深圳）有限公司 一种视频推荐方法、装置及电子设备
CN118395196A (zh) * 2024-06-28 2024-07-26 浪潮电子信息产业股份有限公司 模型训练方法、视频定位方法、系统、设备、产品及介质
CN119883886A (zh) * 2024-12-04 2025-04-25 南京航空航天大学 一种基于深度神经网络的自适应测试用例优化与选择方法

Families Citing this family (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108304882B (zh) * 2018-02-07 2022-03-04 腾讯科技（深圳）有限公司 一种图像分类方法、装置及服务器、用户终端、存储介质
CN109002562B (zh) * 2018-08-30 2021-04-13 北京信立方科技发展股份有限公司 一种仪器识别模型训练方法和装置及仪器识别方法和装置
CN109255377A (zh) * 2018-08-30 2019-01-22 北京信立方科技发展股份有限公司 仪器识别方法、装置、电子设备及存储介质
CN111191665B (zh) * 2018-11-14 2024-11-05 北京奇虎科技有限公司 图像分类方法、装置及电子设备
CN111222360B (zh) * 2018-11-23 2023-10-31 隆基绿能科技股份有限公司 硅料熔化状态的检测方法、设备及存储介质
CN111522941B (zh) * 2019-02-03 2024-07-12 阿里巴巴集团控股有限公司 文本聚类方法、装置、电子设备及计算机存储介质
CN111611418B (zh) * 2019-02-25 2025-04-11 阿里巴巴集团控股有限公司 数据存储方法及数据查询方法
CN110008859A (zh) * 2019-03-20 2019-07-12 北京迈格威科技有限公司 基于视觉的犬只重识别方法和装置
CN110334593A (zh) * 2019-05-28 2019-10-15 浙江泽曦科技有限公司 宠物脸部识别方法及系统
US11631234B2 (en) 2019-07-22 2023-04-18 Adobe, Inc. Automatically detecting user-requested objects in images
US11468550B2 (en) 2019-07-22 2022-10-11 Adobe Inc. Utilizing object attribute detection models to automatically select instances of detected objects in images
US11302033B2 (en) 2019-07-22 2022-04-12 Adobe Inc. Classifying colors of objects in digital images
CN110489240A (zh) * 2019-08-22 2019-11-22 Oppo广东移动通信有限公司 图像识别方法、装置、云平台及存储介质
CN110674847A (zh) * 2019-08-30 2020-01-10 深圳壹账通智能科技有限公司 卡片分类保存方法、装置、计算机设备和存储介质
CN110751171B (zh) * 2019-09-06 2025-08-05 深圳平安医疗健康科技服务有限公司 图像数据分类方法、装置、计算机设备和存储介质
CN110738258B (zh) * 2019-10-16 2022-04-29 Oppo广东移动通信有限公司 图像分类方法、装置及终端设备
CN110825904B (zh) * 2019-10-24 2022-05-06 腾讯科技（深圳）有限公司 一种图像匹配方法、装置、电子设备和存储介质
CN111010611A (zh) * 2019-12-03 2020-04-14 北京达佳互联信息技术有限公司 电子相册的获取方法、装置、计算机设备和存储介质
CN112949672B (zh) * 2019-12-11 2024-10-15 顺丰科技有限公司 商品识别方法、装置、设备以及计算机可读存储介质
CN111191706A (zh) * 2019-12-25 2020-05-22 深圳市赛维网络科技有限公司 图片识别的方法、装置、设备及存储介质
CN111291222B (zh) * 2020-01-20 2023-10-27 上海依图网络科技有限公司 图像识别方法及其装置、计算机可读介质和系统
CN111325276A (zh) * 2020-02-24 2020-06-23 Oppo广东移动通信有限公司 图像分类方法及装置、电子设备、计算机可读存储介质
CN111612741B (zh) * 2020-04-22 2023-11-03 杭州电子科技大学 一种基于失真识别的精确无参考图像质量评价方法
CN111400522B (zh) * 2020-04-29 2021-06-11 广州紫为云科技有限公司 一种交通标志识别方法、训练方法及设备
CN113627455B (zh) * 2020-05-09 2025-01-07 阿里巴巴集团控股有限公司 图像类别确定方法及装置
CN111695604B (zh) * 2020-05-20 2024-07-23 平安科技（深圳）有限公司 一种图像可信度的确定方法、装置及电子设备、存储介质
CN112016586A (zh) * 2020-07-08 2020-12-01 武汉智筑完美家居科技有限公司 图片分类方法及装置
CN112069335A (zh) * 2020-09-03 2020-12-11 Oppo广东移动通信有限公司 图像分类方法、装置、电子设备及存储介质
CN112183663B (zh) * 2020-10-26 2024-07-12 北京达佳互联信息技术有限公司 一种图像分类方法、装置、电子设备及存储介质
CN113057529B (zh) * 2021-02-22 2022-04-22 江苏柯林博特智能科技有限公司 一种基于楼梯清扫机器人的垃圾分类控制系统
CN114119142A (zh) * 2021-11-11 2022-03-01 北京沃东天骏信息技术有限公司 信息推荐方法、装置和系统
CN114417986B (zh) * 2022-01-11 2025-05-23 平安科技（深圳）有限公司 基于人工智能的药物特征信息确定方法及装置
CN114565862A (zh) * 2022-02-17 2022-05-31 中国人民解放军国防科技大学 一种基于ssd卷积网络的大尺寸航拍图像目标检测方法
CN114500680B (zh) * 2022-04-19 2022-09-13 创思(广州)电子科技有限公司 一种多通道交汇的数据分类式信息传输方法
CN116167010B (zh) * 2023-04-25 2023-12-08 南方电网数字电网研究院有限公司 具有智能迁移学习能力的电力系统异常事件快速识别方法
CN116957610B (zh) * 2023-08-08 2024-10-22 南京龟兔赛跑软件研究院有限公司 一种农产品全流程信息溯源录入管理方法、系统及介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1920855A (zh) * 2005-08-26 2007-02-28 富士通株式会社 用于退化文字行的字符识别装置和方法
CN107220677A (zh) * 2017-06-28 2017-09-29 深圳源广安智能科技有限公司 一种分类准确的高光谱图像数据分类系统
CN107491720A (zh) * 2017-04-01 2017-12-19 江苏移动信息系统集成有限公司 一种基于改进型卷积神经网络的车型识别方法
CN108304882A (zh) * 2018-02-07 2018-07-20 腾讯科技（深圳）有限公司 一种图像分类方法、装置及服务器、用户终端、存储介质

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100247969B1 (ko) * 1997-07-15 2000-03-15 윤종용 대용량패턴정합장치및방법
US6647139B1 (en) * 1999-02-18 2003-11-11 Matsushita Electric Industrial Co., Ltd. Method of object recognition, apparatus of the same and recording medium therefor
CN101211341A (zh) * 2006-12-29 2008-07-02 上海芯盛电子科技有限公司 图像智能模式识别搜索方法
CN101540000B (zh) * 2008-03-20 2011-07-27 中国科学院自动化研究所 基于纹理基元统计特性分析的虹膜分类方法
CN103207870B (zh) * 2012-01-17 2020-06-02 华为技术有限公司 一种照片分类管理方法、服务器、装置及系统
CN102663451A (zh) * 2012-03-29 2012-09-12 天津科技大学 一种基于色彩空间特征的图形图像分类方法
CN102982349B (zh) * 2012-11-09 2016-12-07 深圳市捷顺科技实业股份有限公司 一种图像识别方法及装置
CN104268552B (zh) * 2014-09-04 2017-06-13 电子科技大学 一种基于部件多边形的精细类别分类方法
CN104392250A (zh) * 2014-11-21 2015-03-04 浪潮电子信息产业股份有限公司 一种基于MapReduce的图像分类方法
US9704066B2 (en) * 2015-03-20 2017-07-11 Microsoft Technology Licensing, Llc Multi-stage image classification
CN105069133B (zh) * 2015-08-18 2018-09-14 东南大学 一种基于未标记数据的数字图像分类方法
CN105160358B (zh) * 2015-09-07 2018-08-17 苏州大学张家港工业技术研究院 一种图像分类方法及系统
CN105335710A (zh) * 2015-10-22 2016-02-17 合肥工业大学 一种基于多级分类器的精细车辆型号识别方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1920855A (zh) * 2005-08-26 2007-02-28 富士通株式会社 用于退化文字行的字符识别装置和方法
CN107491720A (zh) * 2017-04-01 2017-12-19 江苏移动信息系统集成有限公司 一种基于改进型卷积神经网络的车型识别方法
CN107220677A (zh) * 2017-06-28 2017-09-29 深圳源广安智能科技有限公司 一种分类准确的高光谱图像数据分类系统
CN108304882A (zh) * 2018-02-07 2018-07-20 腾讯科技（深圳）有限公司 一种图像分类方法、装置及服务器、用户终端、存储介质

Cited By (46)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112749296A (zh) * 2019-10-31 2021-05-04 北京达佳互联信息技术有限公司 一种视频推荐方法、装置、服务器及存储介质
CN112749296B (zh) * 2019-10-31 2024-01-26 北京达佳互联信息技术有限公司 一种视频推荐方法、装置、服务器及存储介质
CN112948370A (zh) * 2019-11-26 2021-06-11 上海哔哩哔哩科技有限公司 数据分类方法、装置以及计算机设备
CN111046933B (zh) * 2019-12-03 2024-03-05 东软集团股份有限公司 图像分类方法、装置、存储介质及电子设备
CN111046933A (zh) * 2019-12-03 2020-04-21 东软集团股份有限公司 图像分类方法、装置、存储介质及电子设备
CN111061898A (zh) * 2019-12-13 2020-04-24 Oppo（重庆）智能科技有限公司 图像处理方法、装置、计算机设备及存储介质
CN111078914A (zh) * 2019-12-18 2020-04-28 书行科技(北京)有限公司 重复图片检测方法与装置
CN111078914B (zh) * 2019-12-18 2023-04-18 书行科技(北京)有限公司 重复图片检测方法与装置
CN111160218A (zh) * 2019-12-26 2020-05-15 浙江大华技术股份有限公司 一种特征向量比对方法、装置电子设备及存储介质
CN111222557A (zh) * 2019-12-31 2020-06-02 Oppo广东移动通信有限公司 图像分类方法、装置、存储介质及电子设备
CN111209423B (zh) * 2020-01-07 2023-04-07 腾讯科技（深圳）有限公司 一种基于电子相册的图像管理方法、装置以及存储介质
CN111209423A (zh) * 2020-01-07 2020-05-29 腾讯科技（深圳）有限公司 一种基于电子相册的图像管理方法、装置以及存储介质
CN111222585A (zh) * 2020-01-15 2020-06-02 深圳前海微众银行股份有限公司 数据处理方法、装置、设备及介质
CN111339884A (zh) * 2020-02-19 2020-06-26 浙江大华技术股份有限公司 图像识别方法以及相关设备、装置
CN111339884B (zh) * 2020-02-19 2023-06-06 浙江大华技术股份有限公司 图像识别方法以及相关设备、装置
CN111382281B (zh) * 2020-03-04 2023-05-02 腾讯科技（深圳）有限公司 基于媒体对象的内容的推荐方法、装置、设备和存储介质
CN111382281A (zh) * 2020-03-04 2020-07-07 腾讯科技（深圳）有限公司 基于媒体对象的内容的推荐方法、装置、设备和存储介质
CN111401428A (zh) * 2020-03-12 2020-07-10 Oppo广东移动通信有限公司 一种图像分类方法、装置、电子设备和存储介质
CN111401464A (zh) * 2020-03-25 2020-07-10 北京字节跳动网络技术有限公司 分类方法、装置、电子设备及计算机可读存储介质
CN111488933A (zh) * 2020-04-13 2020-08-04 上海联影智能医疗科技有限公司 图像分类方法、网络、计算机设备和存储介质
CN111488933B (zh) * 2020-04-13 2024-02-27 上海联影智能医疗科技有限公司 图像分类方法、网络、计算机设备和存储介质
CN111639525A (zh) * 2020-04-22 2020-09-08 上海擎感智能科技有限公司 一种感知算法的训练方法、装置及计算机存储介质
CN111582372A (zh) * 2020-05-09 2020-08-25 西交利物浦大学 图像分类方法、模型、存储介质及电子设备
CN111914879B (zh) * 2020-06-16 2024-04-26 北京迈格威科技有限公司 分类参数值生成方法、图像分类方法、装置、设备及存储介质
CN111914879A (zh) * 2020-06-16 2020-11-10 北京迈格威科技有限公司 分类参数值生成方法、图像分类方法、装置、设备及存储介质
CN113963189A (zh) * 2020-07-03 2022-01-21 顺丰科技有限公司 物体分类方法、装置、电子设备及存储介质
CN114065826A (zh) * 2020-07-28 2022-02-18 紫东信息科技(苏州)有限公司 图像分类模型的构建方法、分类方法、装置及电子设备
CN112016617B (zh) * 2020-08-27 2023-12-01 中国平安财产保险股份有限公司 细粒度分类方法、装置及计算机可读存储介质
CN112016617A (zh) * 2020-08-27 2020-12-01 中国平安财产保险股份有限公司 细粒度分类方法、装置及计算机可读存储介质
CN112182269A (zh) * 2020-09-27 2021-01-05 北京达佳互联信息技术有限公司 图像分类模型的训练、图像分类方法、装置、设备及介质
CN112182269B (zh) * 2020-09-27 2023-11-28 北京达佳互联信息技术有限公司 图像分类模型的训练、图像分类方法、装置、设备及介质
CN112329933B (zh) * 2020-10-30 2022-09-27 腾讯科技（深圳）有限公司 数据处理方法、装置、服务器及存储介质
CN112329933A (zh) * 2020-10-30 2021-02-05 腾讯科技（深圳）有限公司 数据处理方法、装置、服务器及存储介质
CN112396100B (zh) * 2020-11-16 2024-05-24 中保车服科技服务股份有限公司 一种细粒度分类模型的优化方法、系统及相关装置
CN112396100A (zh) * 2020-11-16 2021-02-23 中保车服科技服务股份有限公司 一种细粒度分类模型的优化方法、系统及相关装置
CN112529078A (zh) * 2020-12-07 2021-03-19 杭州海康威视数字技术股份有限公司 一种业务处理方法、装置及设备
CN112464015A (zh) * 2020-12-17 2021-03-09 郑州信大先进技术研究院 一种基于深度学习的图像电子证据筛选方法
CN113849677A (zh) * 2021-09-28 2021-12-28 焦点科技股份有限公司 一种图片自动标注的方法及系统
CN113935407A (zh) * 2021-09-29 2022-01-14 光大科技有限公司 一种异常行为识别模型确定方法及装置
CN115909374A (zh) * 2021-09-30 2023-04-04 腾讯科技（深圳）有限公司 一种信息识别方法、装置、设备及存储介质、程序产品
CN114373192A (zh) * 2022-01-07 2022-04-19 杭州睿胜软件有限公司 根据动物视觉特征分类识别的方法、系统及可读存储介质
CN114780719A (zh) * 2022-03-28 2022-07-22 京东城市(北京)数字科技有限公司 文本分类模型的训练方法、文本分类方法及装置
CN117009575A (zh) * 2022-11-30 2023-11-07 腾讯科技（深圳）有限公司 一种视频推荐方法、装置及电子设备
CN118395196A (zh) * 2024-06-28 2024-07-26 浪潮电子信息产业股份有限公司 模型训练方法、视频定位方法、系统、设备、产品及介质
CN119883886A (zh) * 2024-12-04 2025-04-25 南京航空航天大学 一种基于深度神经网络的自适应测试用例优化与选择方法
CN119883886B (zh) * 2024-12-04 2025-11-04 南京航空航天大学 一种基于深度神经网络的自适应测试用例优化与选择方法

Also Published As

Publication number Publication date
CN108304882B (zh) 2022-03-04
CN108304882A (zh) 2018-07-20

Similar Documents

Publication Publication Date Title
WO2019154262A1 (zh) 一种图像分类方法及服务器、用户终端、存储介质
JP7500689B2 (ja) 制御されていない照明条件の画像中の肌色を識別する技術
US20230020965A1 (en) Method and apparatus for updating object recognition model
WO2019020049A1 (zh) 一种图像检索方法、装置及电子设备
CN108319633B (zh) 一种图像处理方法、装置及服务器、系统、存储介质
CN114863201B (zh) 三维检测模型的训练方法、装置、计算机设备和存储介质
CN106203490A (zh) 一种安卓平台下基于属性学习和交互反馈的图像在线识别、检索方法
WO2018036146A1 (zh) 基于卷积神经网络的目标匹配方法、装置及存储介质
US11531829B2 (en) Automatic image annotation
CN116597267B (zh) 图像识别方法、装置、计算机设备和存储介质
CN117173624A (zh) 在计算装置中的对象识别模型学习方法
WO2024027347A9 (zh) 内容识别方法、装置、设备、存储介质及计算机程序产品
CN106407281B (zh) 图像检索方法及装置
CN110162689B (zh) 信息推送方法、装置、计算机设备和存储介质
Li et al. An attention-based prototypical network for forest fire smoke few-shot detection
CN110674342B (zh) 查询目标图像的方法和装置
CN119229204B (zh) 一种细粒度多模态提示引导的视觉关系识别方法及装置
CN114168728B (zh) 一种图像识别模型的训练方法及图像识别方法
CN115995079A (zh) 图像语义相似度分析方法和同语义图像检索方法
CN114266921A (zh) 图像描述信息获取方法、装置、服务器及存储介质
CN115909357B (zh) 基于人工智能的目标识别方法、模型训练方法和装置
CN108596068B (zh) 一种动作识别的方法和装置
Anuwa et al. Car logo recognition using yolov8 and microsoft azure custom vision
US20220198124A1 (en) Method and apparatus for recognizing handwriting inputs in multiple-user environment
CN108345905A (zh) 多特征图片识别方法、装置及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19751832

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19751832

Country of ref document: EP

Kind code of ref document: A1