WO2019105106A1 - 图像分类方法及个性化推荐方法、计算机设备及存储介质 - Google Patents

图像分类方法及个性化推荐方法、计算机设备及存储介质 Download PDF

Info

Publication number
WO2019105106A1
WO2019105106A1 PCT/CN2018/106196 CN2018106196W WO2019105106A1 WO 2019105106 A1 WO2019105106 A1 WO 2019105106A1 CN 2018106196 W CN2018106196 W CN 2018106196W WO 2019105106 A1 WO2019105106 A1 WO 2019105106A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
classified
feature
sub
global
Prior art date
Application number
PCT/CN2018/106196
Other languages
English (en)
French (fr)
Inventor
顾佳伟
马林
刘威
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2019105106A1 publication Critical patent/WO2019105106A1/zh
Priority to US16/675,831 priority Critical patent/US11238315B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G06F18/2113—Selection of the most significant subset of features by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G06F18/2115—Selection of the most significant subset of features by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features

Definitions

  • the present application relates to the field of image processing technologies, and in particular, to an image classification method, a personalized recommendation method, a computer device, and a storage medium.
  • image features are obtained by analyzing images, and images are classified according to image features and classification algorithms.
  • image features used in the existing image classification methods are relatively simple, and when classified according to this, the classification accuracy is insufficient.
  • an image classification method a personalized recommendation method, a computer device, and a storage medium are proposed.
  • An image classification method implemented by a computer device comprising the following steps:
  • the classification result is determined by classifying the image to be classified according to the global image feature of the image to be classified, the image feature of the local key region of the image to be classified, and the associated feature of the image to be classified.
  • the application also provides a personalized recommendation method, which is implemented by a computer device, including:
  • a classification result corresponding to the image to be classified wherein the classification result is obtained by extracting a global image feature of the image to be classified; and determining a local key of the image to be classified according to the image to be classified and the global image feature of the image to be classified And acquiring an image feature of the local key region of the image to be classified; acquiring an associated feature of the image to be classified according to the global image feature of the image to be classified and the image feature of the local key region of the image to be classified; The global image feature of the classified image, the image feature of the local key area of the image to be classified, and the associated feature of the image to be classified, and the image to be classified is classified and determined;
  • a computer device comprising a memory and a processor, wherein the memory stores a computer program, the computer program being executed by the processor, causing the processor to perform the following steps:
  • the classification result is determined by classifying the image to be classified according to the global image feature of the image to be classified, the image feature of the local key region of the image to be classified, and the associated feature of the image to be classified.
  • a computer device comprising a memory and a processor, wherein the computer stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:
  • One or more non-volatile storage media storing a computer program, when executed by one or more processors, cause one or more processors to perform the following steps:
  • the classification result is determined by classifying the image to be classified according to the global image feature of the image to be classified, the image feature of the local key region of the image to be classified, and the associated feature of the image to be classified.
  • One or more non-volatile storage media storing a computer program, when executed by one or more processors, cause one or more processors to perform the following steps:
  • the above image classification method, personalized recommendation method, computer storage medium and device classify the classified image according to the global image feature, the image feature of the local key area and the associated feature. That is, in the process of image classification, not only the global image features are considered, but also the image features of the local key regions are considered, and the image feature information according to the image classification is enriched, and the image classification can be accurately performed according to the rich feature information. Classification to improve image classification accuracy. Subsequent based on the classification result, the information of the object to be recommended can be accurately obtained and pushed, so that accurate recommendation can be achieved.
  • FIG. 1 is a schematic diagram of an application environment of an image classification method according to an embodiment of the present application.
  • FIG. 2 is a schematic flow chart of an image classification method according to an embodiment
  • FIG. 3 is a schematic flowchart of a sub-step S220 in an image classification method according to another embodiment
  • step S322 is a schematic diagram of a sub-flow of step S322 in the image classification method of another embodiment
  • FIG. 5 is a schematic diagram of a sub-flow of step S240 in an image classification method according to another embodiment
  • FIG. 6 is a schematic diagram of a sub-flow of step S250 in an image classification method according to another embodiment
  • FIG. 7 is a schematic diagram of a sub-flow before step S653 in the image classification method of another embodiment
  • FIG. 8 is a schematic diagram of a sub-flow of step S760 in an image classification method according to another embodiment
  • FIG. 9 is a schematic diagram of an image classification method according to an embodiment
  • FIG. 10 is a flow chart of a personalized recommendation method of an embodiment
  • Figure 11 is a block diagram showing an image classification device of an embodiment
  • FIG. 12 is a schematic diagram of a sub-module of the determining module 120 in the image classification device of another embodiment
  • FIG. 13 is a schematic diagram of a sub-module of the area weight obtaining module 222 in the image classification apparatus of another embodiment
  • FIG. 14 is a schematic diagram of a sub-module of the association feature acquisition module 140 in the image classification apparatus of another embodiment
  • 15 is a schematic diagram of a sub-module of the classification module 150 in the image classification device of another embodiment
  • 16 is a block diagram of a personalized recommendation device of an embodiment
  • Figure 17 is a block diagram showing the structure of a computer device in an embodiment.
  • FIG. 1 is a schematic diagram of an application environment of an image classification method according to an embodiment of the present application.
  • the application environment relates to a terminal 110 and a server 120.
  • Terminal 110 and server 120 can communicate over a network.
  • the terminal 110 may specifically be a desktop terminal or a mobile terminal, and the mobile terminal may include, but is not limited to, a mobile phone, a tablet computer, a notebook computer, and the like.
  • the server 120 can be implemented by a stand-alone server or a server cluster composed of a plurality of servers.
  • the image classification method can be applied to the server 120 or the terminal 110.
  • the terminal 110 can access the corresponding server 120 through the network to request a corresponding classification result, and the server 120 can push the classification result to The terminal 110, the user of the terminal 110 can perform image search, download, and the like according to the classification result.
  • an image classification method is provided. This embodiment is mainly illustrated by the method being applied to the server 120 (or the terminal 110) in FIG. 1 described above. Referring to FIG. 2, the image classification method specifically includes steps S210-S250.
  • Image features are information that characterizes the image, including global image features and local image features.
  • Global image features can represent features on the entire image and are used to describe the overall characteristics of the image.
  • the local image feature is relative to the global image feature and refers to the local representation of the image feature, reflecting the local specificity of the image.
  • the global image feature of the image to be classified is extracted as one of the features on which the subsequent classification is based.
  • the global image feature may be a feature matrix.
  • S220 Determine a local key area of the image to be classified according to the image to be classified and the global image feature of the image to be classified.
  • the local key area is a partial area in the image to be classified, and the global image feature includes image features of each sub-area, and the local key area can be determined from the image to be classified by the global image feature.
  • Different types of images may have the same overall characteristics. Classification by global image features alone may lead to classification errors. To obtain accurate classification, the key point is to find some local regions with slight differences.
  • the local key region of the image to be classified is determined, that is, the corresponding local key region is extracted from the image.
  • the image feature extraction is needed to obtain the image features of the local key area, that is, the image features representing the partial areas in the image to be classified.
  • S240 Acquire an associated feature of the image to be classified according to the global image feature of the image to be classified and the image feature of the local key region of the image to be classified.
  • the above-mentioned global image feature and the image feature of the local key region are features obtained by processing the same image to be classified, and there must be an association. Therefore, in this embodiment, it is also required to acquire the global image feature and the image feature of the local key region.
  • the association feature between the features refers to the feature of some relevance between the features, which can further characterize the features of the image to be classified, and use it as one of the features of the subsequent classification. Considering the global image features and the local key regions The relevance of image features can improve the accuracy of image classification.
  • S250 classify the classified image according to the global image feature of the image to be classified, the image feature of the local key region of the image to be classified, and the associated feature of the image to be classified.
  • the global image features of the images to be classified, the image features of the local key regions, and the associated feature combinations are used as the features on which the classification is based. These features can be combined to more fully characterize the image, not only considering the global, but also considering the local and global The correlation between the parts, according to which classification, can obtain accurate classification results.
  • the classified image is classified according to the global image feature of the image to be classified, the image feature of the local key region, and the associated feature. That is, in the process of image classification, not only the global image features are considered, but also the image features of the local key regions are considered, and the image feature information according to the image classification is enriched, and the image classification can be accurately performed according to the rich feature information. Classification to improve image classification accuracy.
  • the image classification method described above may be a fine-grained image classification method that can be used to classify fine-grained images.
  • the step S220 of determining the local key area of the image to be classified according to the image to be classified and the global image feature of the image to be classified includes:
  • S321 Acquire, according to the global image feature of the image to be classified, a sub-image feature corresponding to each sub-region of the image to be classified.
  • Each sub-region can together form an entire image to be classified, and the global image feature includes sub-image features of each sub-region.
  • each sub-region in the image to be classified can be obtained.
  • Corresponding sub-image features It can be understood that any one of the sub-image features corresponding to the image to be classified corresponds to one sub-region.
  • the global image feature corresponding to the image to be classified Ig is 28 ⁇ 28 512-dimensional vectors, which can be understood as 512 feature 28 ⁇ 28 feature maps, with 28 ⁇ 28 feature vectors, and 512 in each feature vector.
  • the feature elements, 28 ⁇ 28 feature vectors constitute a global image feature of the image Ig to be classified, wherein each feature vector corresponds to a sub-image feature of one sub-region, so that each global image feature of the image to be classified Ig can be obtained. Sub-image features corresponding to the sub-regions.
  • the sub-image feature of the sub-region is a feature representation of the sub-region of the image to be classified.
  • the sub-image features of each sub-region need to be weighted respectively, and the weight is larger, indicating The more important this sub-area is.
  • S323 Determine a local key area of the image to be classified according to the sub-image feature of each sub-area, the weight of the sub-image feature of each sub-area, and the image to be classified.
  • the weights corresponding to the sub-image features of each sub-region are respectively acquired, the importance degree of each sub-region can be known, and the corresponding local key regions can be extracted from the image to be classified. In this way, in determining the local key regions, the weights of the sub-image features of each sub-region are considered, and the accuracy of the local key regions can be ensured, thereby improving the accuracy of subsequent classification.
  • the sub-region with the largest weight in each sub-region may be selected as the local key region, or the sub-region with the weight greater than the preset value may be selected from each sub-region, and the local region is determined according to the sub-region whose weight is greater than the preset value. Key areas. Since the sub-regions whose weight is greater than the preset value may overlap, the sub-regions whose weight is greater than the preset value may be merged to obtain a local key region.
  • the weights corresponding to the sub-image features of each sub-region are respectively obtained, and the process of determining the local key regions of the image to be classified according to the sub-image features of each sub-region, the weight of the sub-image features of each sub-region, and the image to be classified. Is the process of determining local key areas based on the attention mechanism (attention mechanism).
  • the step S322 of acquiring the weight corresponding to the sub-image features of each sub-region includes:
  • S421 Acquire a preset high-dimensional feature vector.
  • the preset high-dimensional feature may be a sub-image feature corresponding to the convolutional god to process the sample image to obtain a convolution feature according to the convolution god, and then the sample image acquired according to the convolution feature in the preset local key region.
  • the dimensions of the sub-image features of the sub-regions of the image to be classified are the same.
  • S422 Combine the sub-image features of each sub-region with the preset high-dimensional feature vectors to obtain combined features of the sub-regions.
  • the sub-image features of each sub-region are respectively spliced with a preset high-dimensional feature vector, and the number of elements in the combined feature is the sum of the number of elements of the sub-image feature of the sub-region and the number of elements in the preset high-dimensional feature vector.
  • S423 The combined features of the sub-regions are respectively input as the multi-layer perceptron, and the multi-layer perceptron respectively performs prediction, and the weights corresponding to the sub-image features of the sub-regions are respectively obtained.
  • Multilayer Perceptron is a feedforward artificial neural network model that maps multiple input data sets onto a single output data set.
  • the number of nodes of the input layer of the multi-layer perceptron is the same as the number of elements in the combined feature
  • the combined feature of the sub-region is used as the input of the multi-layer perceptron
  • the input combined features are mapped according to the multi-layer perceptron to
  • the weights of the sub-image features of one sub-region are obtained, and the mapping process is repeated for the combined features of the sub-regions respectively, and the weights corresponding to the sub-image features of each sub-region are respectively obtained.
  • the formula for obtaining the weights corresponding to the sub-image features of each sub-region is as follows:
  • the weight of the sub-image feature of the i-th sub-region of the image to be classified is combined, and the MLP is a multi-layer perceptron, which represents h i as the input of the multi-layer perceptron, and the multi-layer perceptron predicts the output according to the input, and the output result is
  • the global image feature corresponding to the image to be classified is a feature of L ⁇ M ⁇ M dimension, wherein L and M are positive integers greater than or equal to 1, respectively, which can be understood as M ⁇ M L-dimensional vectors, one for each sub-region.
  • Sub-image feature then It can also be understood as the i-th sub-image feature in the global image feature, and the number of elements in the sub-image feature is L.
  • the method before the prediction is obtained by the multi-layer perceptron, the training is required to obtain the multi-layer perceptron, that is, before the step S423, the multi-layer perceptron needs to be obtained in advance. Therefore, before the combined features of the sub-regions are respectively input as the multi-layer perceptron, the method further includes the steps of: extracting global image features of each sample image; and acquiring each sub-sample of the sample image according to the global image feature of the sample image Sub-image features corresponding to the regions respectively; weighting the sub-image features of each sub-region of the sample image to obtain initial weights corresponding to the sub-image features of each sub-region; and sub-image features of each sub-region according to the sample image and An initial weight of the sub-image features of each sub-region, obtaining an image context vector of each sample image; performing weighting processing on each of the image context vectors to obtain each weighted image context vector; and respectively respectively, each of the weighted image context vectors
  • a sample image can be understood as an image for training, which is an image of a known classification.
  • the foregoing step is a process of performing multi-layer perceptron training, and the process of acquiring the global image feature of the sample image is similar to the process of acquiring the global image feature of the image to be classified, and correspondingly, each sub-region of the acquired sample image corresponds to
  • the process of acquiring the sub-image features is similar to the process of obtaining the sub-image features corresponding to the respective sub-regions of the image to be classified. The difference is only in the image, the former is the sample image, and the latter is the image to be classified.
  • the image context vector is a feature expression of a salient region of the image to be classified, which can be understood as a feature expression of a local key region.
  • the sub-image features of the sub-regions of the sample image may be processed according to the initial weights of the sub-image features of the sub-regions of the sample image to obtain an image context. vector.
  • the formula for obtaining the image context vector S is:
  • q i is a sub-image feature of the i-th sub-region of the sample image
  • a i is an initial weight of the sub-image feature of the i-th sub-region of the sample image.
  • ⁇ is the weight of the image context vector, and the weighted image context vector R can be obtained by performing weighting processing on each of the image context vectors.
  • the training input feature is obtained. Since there are a plurality of sample images, each training input feature can be obtained, and the initial multilayer sensor is obtained according to each training input feature. Train to obtain the above multilayer sensor. Specifically, during the training process, the softmax regression function ⁇ s is used for training, and the training result label label1 is expressed by the following formula:
  • Label1 Arg max( ⁇ s (R, W1, b1)).
  • W1 is a weight matrix
  • b1 is an offset vector
  • Arg max(f(e)) is a value of a variable e corresponding to f(e) taking a maximum value.
  • the step S240 of acquiring the associated feature of the image to be classified according to the global image feature of the image to be classified and the image feature of the local key region of the image to be classified includes:
  • S542 Multiply the transposition feature by the image feature of the local key region of the image to be classified to obtain a product feature.
  • the size of the matrix a is m rows and n columns
  • the size of the matrix b is n rows and m columns
  • the size of the matrix result obtained by multiplying the matrix a by the matrix b is m rows and m columns
  • the value at the position of the jth column is the sum of the n products obtained by multiplying the n numbers on the i-th row of the matrix a and the n numbers on the j-th column of the second matrix.
  • Multiplying the transposed feature by the image features of the local key region is to multiply the elements of one row of the transposed matrix and the elements of the columns of the image features of the local key region, respectively, to obtain a row of the product features until the transfer
  • Each row of the matrix is subjected to the above-described multiplication and summation to obtain a product feature, thereby realizing a product between different feature dimensions between the global image feature and the image feature of the local key region, and obtaining the associated feature, that is, the product feature.
  • A is the global image feature of the image to be classified
  • B is the image feature of the local key region of the image to be classified
  • a T is the transposition feature.
  • a i is the i-th feature vector in the global image feature A of the image to be classified
  • B j is the j-th feature vector in the image feature B of the local key region of the image to be classified.
  • S543 averaging the pooled product feature to obtain the associated feature of the image to be classified.
  • Pooling can be understood as compression, which is the aggregation statistics of features at different locations. For example, calculating the average value of a particular feature on an area of an image as a value for the region, thus reducing the dimension while improving the results, It is easy to over-fitting, and the operation of this aggregation is called pooling. Pooling includes averaging pooling and pooling. The average value of a specific feature on the area is taken as a value of the area, which is called average pooling. The maximum value of a specific feature on the area is used as the A value for a zone, called maximum pooling.
  • the product features are averaged and pooled to obtain associated features to ensure the accuracy of the associated features.
  • the associated features of the obtained image to be classified are in the form of L ⁇ L.
  • the step S210 of extracting the global image feature of the image to be classified includes: performing feature extraction on the image to be classified according to the first convolutional neural network, and obtaining a global image feature of the image to be classified.
  • the step S230 of extracting the image features of the local key regions includes: performing feature extraction on the local key regions of the classified image according to the second convolutional neural network, and obtaining image features of the local key regions of the image to be classified.
  • the convolutional neural network is a feedforward neural network.
  • the artificial neurons can respond to surrounding units and can perform large-scale image processing.
  • the convolutional neural network includes a convolutional layer and a pooling layer.
  • the feature result obtained by feature extraction by convolutional neural network is characterized by three spatial dimensions, which can be understood as obtaining multiple feature images.
  • image A is processed by convolution image network, and the obtained global image and feature are
  • the feature of the 512 ⁇ 28 ⁇ 28 form can be understood as 512 feature maps with a size of 28 ⁇ 28, which can also be understood as 28 ⁇ 28 512-dimensional single vectors, that is, 512 elements in a single vector.
  • feature extraction may be performed by using a corresponding convolutional neural network respectively, and the feature of the global image and the local key region are satisfied when the feature extraction is performed by the first convolutional neural network and the second convolutional neural network respectively.
  • the dimensions of the image features are the same.
  • the step of classifying the classified image according to the global image feature of the image to be classified, the image feature of the local key region of the image to be classified, and the associated feature of the image to be classified is performed.
  • S250 includes:
  • S651 Converting the global image feature of the image to be classified into a one-dimensional global image feature vector, converting the image feature of the local key region of the image to be classified into a one-dimensional local image feature vector, and converting the associated feature of the image to be classified into one-dimensional Associate feature vectors.
  • S652 Combine the one-dimensional global image feature vector, the one-dimensional local image feature vector, and the one-dimensional associated feature vector to obtain a one-dimensional image merge feature vector corresponding to the image to be classified.
  • the number of elements in the one-dimensional image merge feature vector is the sum of the number of elements in the one-dimensional global image feature vector, the number of elements of the one-dimensional partial image feature vector, and the number of elements of the one-dimensional associated feature vector.
  • S653 The one-dimensional image merge feature vector is used as an input of the trained classification model, and the classified image is classified according to the trained classification model to determine the classification result.
  • a one-dimensional vector can be understood as a row of vectors (which can have multiple columns, that is, there can be multiple elements in a row vector) or a column of vectors (which can have multiple rows). Since the global image features obtained in the process of extracting features and the image features of the local key regions may be in a multi-dimensional matrix form, when the classification model is used for classification, the input of the classification model should be a one-dimensional vector form, one-dimensional Each element in the vector is used as the input of the node of the classification input layer. Therefore, when performing image classification, the obtained global image feature needs to be transformed into a one-dimensional global image feature vector, and the image features of the local key region are transformed into one-dimensional locality.
  • the image feature vector transforms the associated feature into a one-dimensional associated feature vector. Then, in order to improve the classification accuracy, the above-mentioned one-dimensional feature vector is merged into a one-dimensional image combined feature vector, which is used as the input of the trained classification model.
  • the initial classification result may be a probability that the images to be classified belong to each classification category respectively
  • the final classification result that is, the classification image to be classified, is determined according to the probability that the images to be classified belong to each classification category respectively.
  • the classification result may be a classification category corresponding to the maximum probability in the initial classification result as a final classification result.
  • the trained classification model can be a trained softmax linear regression model.
  • the elements in each dimension of the image features may be merged into one dimension.
  • the one-dimensional vector can be a single-row vector, including 100 elements, which can be understood as a row of 100 columns. Dimension vector.
  • the one-dimensional associated feature vector may be dimension-reduced to update the one-dimensional associated feature vector.
  • the one-dimensional image merge feature vector is used as the input of the trained classification model, and before the step S653 of classifying the classified image according to the trained classification model, the method further includes the following steps:
  • S710 Extract global image features of each sample image.
  • S720 Determine local key regions of each sample image according to each sample image and global image features of each sample image.
  • S730 Extract image features of local key regions of each sample image.
  • S740 Acquire an associated feature corresponding to each sample image according to global image features of each sample image and image features of local key regions of each sample image.
  • the trained classification model needs to be obtained, and the above process is the process of obtaining the trained classification model. Since in the process of predicting (ie, classifying the classified image), the global image feature according to the image to be classified, the image feature of the local key region, and the associated feature of the image to be classified are adopted, so that the training and the prediction need to be corresponding, thereby ensuring the basis The accuracy of the classification model after training for classification.
  • the feature extraction is first performed on each sample image, and the global image features of each sample image, the image features of the local key regions of each sample image, and the associated features of each sample image are obtained, and the feature extraction process and the image to be classified are performed on the sample image.
  • the feature extraction process is similar except that the image of the feature to be extracted is in the embodiment.
  • the image of the feature to be extracted is each sample image, and when the prediction is performed, the image of the feature to be extracted is the image to be classified.
  • S750 Initialize the classification model to obtain an initial classification model.
  • the classification model corresponds to the model parameters.
  • the classification model needs to be initialized, that is, the structure of the classification model and the model parameters are initialized to obtain the initial model parameters.
  • the training process is to continuously correct the initial model parameters in the initial classification model. The process, until the training result meets the requirements, obtains the optimal model parameters, and the classification model corresponding to the optimal model parameters is the model after training.
  • the initial classification model can be an initial softmax linear regression model.
  • the initial classification model is trained according to global image features of each sample image, image features of local key regions of each sample image, and associated features of each sample image, to obtain a trained classification model.
  • the initial model parameters in the initial classification model are continuously corrected by the global image feature of each sample image, the image features of the local key regions of each sample image, and the associated features of each sample image.
  • the classification model is trained to obtain the trained classification model, and the trained classification model can be used to classify the classified images.
  • the gradient descent formula used is: as well as among them, It is the cost function in the softmax linear regression model, which is used to measure the degree of fit of the training sample to the linear model.
  • the initial classification model includes a first initial classification model and a second initial classification model.
  • the step S760 of acquiring the trained classification model is performed according to the global image feature of each sample image, the image feature of the local key region of each sample image, and the associated feature of each sample image. :
  • the corresponding initial classification model is initialized, and different initial classification models are separately trained according to different features, that is, the first initial classification model is initialized according to the global image features of the sample image, which can be understood, first The number of nodes of the input layer of the initial classification model is determined according to the number of elements in the global image feature of the sample image, and the second initial classification model is initialized according to the image features of the local key regions of the sample image. It can be understood that the second initial classification model The number of nodes in the input layer is determined by the number of elements in the image features of the local key regions of the sample image, so that two independent target classification models can be obtained.
  • the two independent target classification models cannot satisfy the training requirements according to the associated features, but they are respectively related according to the association.
  • the global image features of each sample image on which the feature is based and the image features of the local key regions of each sample image are trained, and there is a certain correlation.
  • the first target classification model and the second target classification model are obtained, and the initial merge classification is obtained.
  • the model trains the initial merged classification model according to the associated features of each sample image to obtain the trained classification model.
  • the independence of the feature is considered, the relevance of the feature is considered, the accuracy of the trained classification model is improved, and the classification accuracy is improved.
  • the first initial classification model and the second initial classification model may each be an initial softmax linear regression model.
  • the first target classification model corresponds to the model structure and the model parameters after the training is completed
  • the second target classification model corresponds to the model structure and the model parameters after the training is completed
  • the initial combination obtained by combining the two classification models is obtained.
  • the classification model can be understood as the corresponding structure is the corresponding superposition of each layer node in the two classification models, and the corresponding model parameters are the superposition of the model parameters of the two classification models.
  • the first target classification model and the second target classification model are respectively three-layer structure
  • the first target classification model has 512 input nodes in the first layer, 500 nodes in the second layer, and 5 nodes in the third layer ( Can be understood as the output layer)
  • the first layer of the second target classification model has 512 input nodes
  • the second layer has 500 nodes
  • the third layer has 5 nodes.
  • the first layer of the two classification models The input nodes are superimposed into 1024 input nodes
  • the nodes of the second layer of the two classification models are superimposed with 1000 nodes
  • the nodes of the third layer of the two classification models are superimposed into 10 nodes
  • the model parameters of the two classification models have been trained.
  • the initial merged classification model is obtained by initializing other missing model parameters (model parameters between nodes in different layers from different classification models are missing at the beginning of the merge).
  • the initial classification model may be initialized according to the global image feature of the sample image, the image feature of the local key region of the sample image, and the associated feature of the sample image. It can be understood that the node of the input layer of the initial classification model The quantity is determined by the sum of the number of elements in the global image feature of the sample image, the number of elements in the image feature of the local key region of the sample image, and the number of elements in the associated feature of the sample image, according to the global image feature of each sample image, each sample The image features of the local key regions of the image and the associated features of each sample image are combined to obtain the sample merge feature, and the initial classification model is trained according to the sample qualified features to obtain the trained classification model.
  • the formula for classifying the classified images according to the trained classification model is:
  • Label2 Arg max( ⁇ s (c, W2, b2)).
  • W2 is a weight matrix
  • b2 is an offset vector
  • c is a one-dimensional image combining feature vector
  • the global image feature A of the image to be classified Ig and the image feature B of the local key region Ia respectively have a classification channel corresponding to a separate softmax linear regression model, that is, a one-dimensional global corresponding to the global image feature A of the image to be classified Ig
  • the image features and the corresponding softmax linear regression model are used for image classification, and the image classification can also be performed according to the one-dimensional partial image feature vector corresponding to the image feature B of the local key region Ia and the corresponding softmax linear regression model.
  • the feature information is classified by the rich feature information, and the accuracy is improved.
  • the global image feature A of the image to be classified Ig and the image feature B of the local key region Ia the associated feature is determined, and the global image feature of the image to be classified Ig can be obtained.
  • the one-dimensional global image feature vector corresponding to A, the one-dimensional local image feature vector corresponding to the image feature B of the local key region Ia, and the one-dimensional associated feature vector corresponding to the associated feature are combined to obtain a one-dimensional image merge feature vector corresponding to the image to be classified.
  • the one-dimensional image merge feature vector is used as the input of the trained classification model, and the classified image is classified according to the trained classification model, which can improve the classification accuracy.
  • the image classification method of the specific embodiment can improve the classification ability, ensure the accuracy of classification, provide convenience for subsequent user search, and provide a good basis for personalized recommendation of images.
  • the above image classification method can be applied to the fields of personalized recommendation, robot visual recognition, and automatic driving object recognition, and is applied to personalized recommendation as an example.
  • the present application further provides an individualized recommendation method of an embodiment. This embodiment is mainly applied to the server 120 in FIG. 1 by using the method, and includes the following steps:
  • S110 Acquire a classification result determined by classifying the image to be classified by the image classification method.
  • the object can be understood as a target transaction, and the information of each object can be stored in a preset recommendation information database, wherein the object can include commodities, images, articles, and characters, and the images used by different users in the terminal are different, and the image can be utilized as
  • the user provides personalized recommendation, first classifies the images, obtains accurate classification results, and finds the object to be recommended corresponding to the classification result, so as to improve the accuracy of the information to be recommended, and then push the information of the object to be recommended to realize the personality. Recommended for users to view.
  • the personalized recommendation method may be applied to a server or a terminal.
  • the step S130 of the information about the to-be-recommended object may be sent to the terminal to be recommended, and the user may view the received information on the terminal.
  • Object information to be recommended Object information to be recommended.
  • the terminal itself searches for the object to be recommended, that is, the object to be recommended is obtained on the terminal, and the user can obtain the information of the object to be recommended by viewing the terminal.
  • the classification result determined by the server may be directly obtained, and based on the The classification result finds the information of the object to be recommended and recommends it to the terminal.
  • the server may obtain the classification result of the image to be classified from the terminal, and the server may also determine the classification result based on the global image feature of the image to be classified, the image feature of the local key region, and the associated feature. Then, based on the classification result, the information of the object to be recommended is searched and recommended to the terminal.
  • the terminal may obtain the classification result of the image to be classified from the server, and the terminal may also be based on the global image feature of the image to be classified and the image feature of the local key region. And the associated features determine the classification results. Then, based on the classification result, the information of the object to be recommended is searched and pushed to the user. If the image classification method is applied to the terminal, the terminal determines the classification result of the image to be classified through the image classification method, and directly obtains the classification result and searches for the object to be recommended based on the classification result, and then pushes the information to the user.
  • the above personalized recommendation method can obtain the classification result determined by the above image classification method, improve the accuracy of the classification result, and then, according to the classification result, find the corresponding object to be recommended, and can accurately obtain the information of the object to be recommended and push it to implement Recommend the recommendation of the object information to improve the accuracy of the recommendation.
  • an image classification apparatus of an embodiment may be provided in the server 120 or the terminal 10 of FIG. 1, including:
  • the global feature extraction module 110 is configured to extract global image features of the image to be classified.
  • the determining module 120 is configured to determine a local key area of the image to be classified according to the image to be classified and the global image feature of the image to be classified.
  • the local feature extraction module 130 is configured to extract image features of the local key regions of the image to be classified.
  • the association feature acquisition module 140 is configured to acquire an association feature of the image to be classified according to the global image feature of the image to be classified and the image feature of the local key region of the image to be classified.
  • a classification module 150 configured to classify the image to be classified according to the global image feature of the image to be classified, the image feature of the local key region of the image to be classified, and the associated feature of the image to be classified .
  • the image classification device described above classifies the image to be classified according to the global image feature, the image feature of the local key region, and the associated feature. That is, in the process of image classification, not only the global image features are considered, but also the image features of the local key regions are considered, and the image feature information according to the image classification is enriched, and the image classification can be accurately performed according to the rich feature information. Classification to improve image classification accuracy.
  • the determining module 120 includes:
  • the area feature acquiring module 221 is configured to acquire, according to the global image feature of the image to be classified, a sub-image feature corresponding to each sub-region of the image to be classified.
  • the area weight obtaining module 222 is configured to obtain weights corresponding to the sub-image features of each of the sub-areas.
  • the local area determining module 223 is configured to determine a local key area of the image to be classified according to the sub-image feature of each of the sub-areas, the weight of the sub-image feature of each of the sub-areas, and the image to be classified.
  • the area weight obtaining module 222 includes:
  • the preset vector obtaining module 321 is configured to acquire a preset high-dimensional feature vector.
  • a vector merging module 322 configured to combine the sub-image features of each of the sub-regions with the preset high-dimensional feature vector to obtain a combined feature of each of the sub-regions;
  • the prediction module 323 is configured to respectively use the combined features of the sub-regions as input of the multi-layer perceptron, and perform prediction according to the multi-layer perceptron to obtain weights corresponding to the sub-image features of each of the sub-regions.
  • the associated feature acquiring module 140 includes:
  • the transposition module 441 is configured to perform transposition processing on the global image feature of the image to be classified to obtain a transposition feature.
  • the product module 442 is configured to multiply the transposed feature and the image feature of the local key region of the image to be classified to obtain a product feature.
  • the pooling module 443 is configured to average pool the product features to obtain the associated features.
  • the global feature extraction module 110 is configured to perform feature extraction on the image to be classified according to the first convolutional neural network to obtain a global image feature of the image to be classified.
  • the local feature extraction module 130 is configured to perform feature extraction on a local key region of the image to be classified according to the second convolutional neural network, and obtain image features of the local key region of the image to be classified.
  • the classification module 150 includes:
  • a conversion module 551 configured to convert the global image feature of the image to be classified into a one-dimensional global image feature vector, and convert the image feature of the local key region of the image to be classified into a one-dimensional partial image feature vector, The associated features of the image to be classified are converted into one-dimensional associated feature vectors.
  • the feature merging module 552 is configured to combine the one-dimensional global image feature vector, the one-dimensional partial image feature vector, and the one-dimensional associated feature vector to obtain a one-dimensional image merge feature vector corresponding to the image to be classified.
  • the number of elements in the one-dimensional image merge feature vector is the sum of the number of elements in the one-dimensional global image feature vector, the number of elements of the one-dimensional partial image feature vector, and the number of elements of the one-dimensional associated feature vector.
  • the image classification module 553 is configured to use the one-dimensional image merge feature vector as an input of the trained classification model, and classify the image to be classified according to the trained classification model.
  • the image classification device further includes an initialization module and a training module.
  • the global feature extraction module 110 is further configured to extract global image features of each sample image
  • the determining module 120 is further configured to determine a local key area of each of the sample images according to each of the sample images and global image features of each of the sample images;
  • the local feature extraction module 130 is further configured to extract image features of local key regions of each of the sample images
  • the association feature acquiring module 140 is further configured to acquire, according to the global image feature of each of the sample images and the image features of the local key regions of each of the sample images, the associated features respectively corresponding to the sample images;
  • the initialization module is configured to initialize a classification model to obtain an initial classification model
  • the training module is configured to train the initial classification model according to global image features of each of the sample images, image features of local key regions of each of the sample images, and associated features of the sample images.
  • the trained classification model is described.
  • the initial classification model includes a first initial classification model and a second initial classification model.
  • the training module includes: a first training module, a second training module, a model combining module, and a third training module.
  • a first training module configured to perform training on the first initial classification model according to global image features of each of the sample images to obtain a first target classification model
  • a second training module configured to train the second initial classification model according to image features of local key regions of each of the sample images to obtain a second target classification model
  • a model merging module configured to merge the first target classification model and the second target classification model to obtain an initial merge classification model
  • a third training module configured to train the initial merge classification model according to an association feature of each of the sample images to obtain the trained classification model.
  • the initial classification model may be initialized according to the global image feature of the sample image, the image feature of the local key region of the sample image, and the associated feature of the sample image. It can be understood that the node of the input layer of the initial classification model The quantity is determined by the sum of the number of elements in the global image feature of the sample image, the number of elements in the image feature of the local key region of the sample image, and the number of elements in the associated feature of the sample image, according to the global image feature of each sample image, each sample The image features of the local key regions of the image and the associated features of each sample image are combined to obtain the sample merge feature, and the initial classification model is trained according to the sample qualified features to obtain the trained classification model.
  • the present application further provides a personalized recommendation device, which may be disposed in the server 120 or the terminal 10 of FIG. 1, and includes:
  • the classification result obtaining module 610 is configured to acquire a classification result determined by classifying the image to be classified by the image classification device.
  • the searching module 620 is configured to search for object information to be recommended corresponding to the classification result.
  • the pushing module 630 is configured to push the to-be-recommended object information.
  • the personalized recommendation device may obtain the classification result determined by the image classification method, improve the accuracy of the classification result, and then, according to the classification result, find the information of the object to be recommended, and accurately obtain the information of the object to be recommended and push it to implement Recommend the recommendation of the object information to improve the accuracy of the push.
  • Figure 17 is a diagram showing the internal structure of a computer device in one embodiment.
  • the computer device may specifically be the terminal 110 or the server 120 in FIG. 1, and it can be understood that the above method can be implemented by the computer device.
  • the computer device includes a processor, memory, and network interface coupled by a system bus.
  • the memory comprises a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium of the computer device stores an operating system, and may also store a computer program.
  • the processor may implement the image classification method and the personalized recommendation method.
  • the processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device.
  • the processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device.
  • the internal memory may also store a computer program that, when executed by the processor, causes the processor to perform an image classification method and a personalized recommendation method.
  • a network interface is used to connect and communicate with the network 130.
  • the computer device may further include an input device and a display screen, and the display screen of the computer device may be a liquid crystal display or an electronic ink display screen, and the input device of the computer device may be a touch covered on the display screen.
  • the layer may also be a button, a trackball or a touchpad provided on the casing of the computer device, or an external keyboard, a touchpad or a mouse.
  • FIG. 17 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation of the computer device to which the solution of the present application is applied.
  • the specific computer device may It includes more or fewer components than those shown in the figures, or some components are combined, or have different component arrangements.
  • the image classification device provided herein can be implemented in the form of a computer program that can be run on a computer device as shown in FIG.
  • the program modules constituting the image classification device may be stored in a memory of the computer device, such as the global feature extraction module 110, the determination module 120, the local feature extraction module 130, the associated feature acquisition module 140, and the classification module 150 shown in FIG.
  • the computer program of each program module causes the processor to perform the steps in the image processing method of the various embodiments of the present application described in this specification.
  • the computer device shown in FIG. 17 can perform step S210 by the global feature extraction module 110 in the image processing apparatus shown in FIG.
  • the computer device can perform step S220 through the determination module 120.
  • the computer device may perform step S230 through the local feature extraction module 130.
  • the computer device can perform step S240 through the association feature acquisition module 140.
  • the computer device can perform step S240 through the classification module 150.
  • the personalized recommendation device provided by the present application can be implemented in the form of a computer program that can be run on a computer device as shown in FIG.
  • the various program modules constituting the personalized recommendation device can be stored in the memory of the computer device.
  • the computer program of each program module causes the processor to perform the steps in the personalized recommendation method of the various embodiments of the present application described in this specification.
  • the present application provides a computer device of an embodiment, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, the step of implementing the image classification method described above when the processor executes the computer program.
  • the present application provides a computer device of an embodiment, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, the step of implementing the personalized recommendation method described above when the processor executes the computer program.
  • the present application provides a computer readable storage medium of an embodiment, on which is stored a computer program that, when executed by a processor, implements the steps of the image classification method described above.
  • the present application provides a computer readable storage medium of an embodiment, on which is stored a computer program that, when executed by a processor, implements the steps of the personalized recommendation method described above.
  • Non-volatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM) or external cache memory.
  • RAM is available in a variety of formats, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization chain.
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDRSDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • Synchlink DRAM SLDRAM
  • Memory Bus Radbus
  • RDRAM Direct RAM
  • DRAM Direct Memory Bus Dynamic RAM
  • RDRAM Memory Bus Dynamic RAM

Abstract

一种图像分类方法及个性化推荐方法、计算机存储介质及设备，一个实施例的方法包括：提取待分类图像的全局图像特征；根据待分类图像以及全局图像特征，确定待分类图像的局部关键区域；提取局部关键区域的图像特征；根据全局图像特征以及局部关键区域的图像特征，获取关联特征；根据全局图像特征、局部关键区域的图像特征以及关联特征，对待分类图像进行分类。即在进行图像分类的过程中，不但考虑了全局图像特征，还考虑了局部关键区域的图像特征以及关联特征，丰富了图像分类所依据的图像特征信息，依据上述丰富的特征信息进行图像分类时，能够准确分类，提高图像分类准确性。

Description

图像分类方法及个性化推荐方法、计算机设备及存储介质
本申请要求于2017年11月30日提交中国专利局，申请号为2017112445724，申请名称为“图像分类方法及装置、个性化推荐方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，特别涉及一种图像分类方法及个性化推荐方法、计算机设备及存储介质。
背景技术
目前，各种系统、软件和网站中可提供多种多样的图像以供用户选择，为便于用户对图像的查找，提供有图像分类的功能。
在现有图像分类方法中，通过对图像进行分析，获得图像特征，根据图像特征以及分类算法对图像进行分类。然而，现有图像分类方法中采用的图像特征比较单一，据此进行分类时，导致分类准确性不足。
发明内容
根据本申请提供的各种实施例，提出一种图像分类方法及个性化推荐方法、计算机设备及存储介质。
一种图像分类方法，该方法由计算机设备实施，包括以下步骤：
提取待分类图像的全局图像特征；
根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域；
提取所述待分类图像的局部关键区域的图像特征；
根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征；
根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类确定分类结果。
本申请还提供一种个性化推荐方法，该方法由计算机设备实施，包括：
待分类图像对应的分类结果；其中，所述分类结果通过提取待分类图像的全局图像特征；根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域；提取所述待分类图像的局部关键区域的图像特征；根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征；根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类确定；
查找与所述分类结果对应的待推荐信息；
推送所述推荐信息。
一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机程序，所述计算机程序被所述处理器执行时，使得所述处理器执行如下步骤：
提取待分类图像的全局图像特征；
根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域；
提取所述待分类图像的局部关键区域的图像特征；
根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征；
根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类确定分类结果。
一种计算机设备，包括存储器和处理器，存储器中存储有计算机程序，计算机程序被处理器执行时，使得处理器执行如下步骤：
获取待分类图像对应的分类结果；其中，所述分类结果通过提取待分类 图像的全局图像特征；根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域；提取所述待分类图像的局部关键区域的图像特征；根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征；根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类确定；
查找与所述分类结果对应的待推荐信息；
推送所述推荐信息。
一个或多个存储有计算机程序的非易失性存储介质，所述计算机程序被一个或多个处理器执行时，使得一个或多个处理器执行如下步骤：
提取待分类图像的全局图像特征；
根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域；
提取所述待分类图像的局部关键区域的图像特征；
根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征；
根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类确定分类结果。
一个或多个存储有计算机程序的非易失性存储介质，所述计算机程序被一个或多个处理器执行时，使得一个或多个处理器执行如下步骤：
获取待分类图像对应的分类结果；其中，所述分类结果通过提取待分类图像的全局图像特征；根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域；提取所述待分类图像的局部关键区域的图像特征；根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征；根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所 述待分类图像的关联特征，对所述待分类图像进行分类确定；
查找与所述分类结果对应的待推荐信息；
推送所述推荐信息。
上述图像分类方法及个性化推荐方法、计算机存储介质及设备，根据全局图像特征、局部关键区域的图像特征以及关联特征，对待分类图像进行分类。即在进行图像分类的过程中，不但考虑了全局图像特征，还考虑了局部关键区域的图像特征，丰富了图像分类所依据的图像特征信息，依据上述丰富的特征信息进行图像分类时，能够准确分类，提高图像分类准确性。后续基于分类结果可准确获取待推荐对象信息并推送，从而可实现准确推荐。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请的一个实施例的图像分类方法的应用环境示意图；
图2为一个实施例的图像分类方法的流程示意图；
图3为另一个实施例的图像分类方法中步骤S220的子的流程示意图；
图4为另一个实施例的图像分类方法中步骤S322的子流程示意图；
图5为另一个实施例的图像分类方法中步骤S240的子流程示意图；
图6为另一个实施例的图像分类方法中步骤S250的子流程示意图；
图7为另一个实施例的图像分类方法中步骤S653之前的子流程示意图；
图8为另一个实施例的图像分类方法中步骤S760的子流程示意图；
图9为一具体实施例的图像分类方法的原理图；
图10为一个实施例的个性化推荐方法的流程图；
图11为一个实施例的图像分类装置的模块示意图；
图12为另一个实施例的图像分类装置中确定模块120的子模块示意图；
图13为另一个实施例的图像分类装置中区域权重获取模块222的子模块示意图；
图14为另一个实施例的图像分类装置中关联特征获取模块140的子模块示意图；
图15为另一个实施例的图像分类装置中分类模块150的子模块示意图；
图16为一个实施例的个性化推荐装置的模块示意图；
图17为一个实施例中计算机设备的结构框图。
具体实施方式
为使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步的详细说明。应当理解，此处所描述的具体实施方式仅仅用以解释本申请，并不限定本申请的保护范围。
图1为本申请一个实施例中图像分类方法的应用环境示意图。参照图1，该应用环境涉及终端110和服务器120。终端110和服务器120可通过网络进行通信。终端110具体可以是台式终端或移动终端，移动终端可以包括但不限于手机、平板电脑、笔记本电脑等。服务器120可以用独立的服务器或者是多个服务器组成的服务器集群来实现。图像分类方法可应用于服务器120或终端110，在图像分类方法应用于服务器120中时，终端110通过网络可访问对应的服务器120，以请求相应的分类结果，服务器120可将该分类结果推送至终端110，终端110的用户可以根据该分类结果进行图像查找、下载等等。
如图2所示，在一个实施例中，提供了一种图像分类方法。本实施例主要以该方法应用于上述图1中的服务器120(或终端110)来举例说明。参照图2，该图像分类方法具体包括步骤S210-S250。
S210：提取待分类图像的全局图像特征。
图像特征为表征图像特点的信息，包括全局图像特征和局部图像特征，全局图像特征指能表示整幅图像上的特征，用于描述图像的整体特征。局部图像特征是相对全局图像特征而言的，指图像特征的局部表达，反映图像上具有的局部特殊性。在本实施例中，通过提取待分类图像的全局图像特征，作为后续分类时所依据的特征之一。在本实施例中，全局图像特征可以为特征矩阵。
S220：根据待分类图像以及待分类图像的全局图像特征，确定待分类图像的局部关键区域。
局部关键区域为待分类图像中的部分区域，全局图像特征中包括了各子区域的图像特征，通过全局图像特征可从待分类图像中确定局部关键区域。不同种类的图像之间可能存在相同的整体特征，仅仅通过全局图像特征来进行分类容易导致分类错误，为得到准确的分类，关键点是寻找一些存在细微差别的局部区域。在本实施例中，在获得全局图像特征的基础上，确定待分类图像的局部关键区域，即从图像中提取对应的局部关键区域。
S230：提取待分类图像的局部关键区域的图像特征。
局部关键区域确定后，则需要对其进行图像特征提取，获得局部关键区域的图像特征，即为表征待分类图像中部分区域的图像特征。
S240：根据待分类图像的全局图像特征以及待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征。
上述全局图像特征和局部关键区域的图像特征是对同一待分类图像进行处理得到的特征，其中必然存在关联，因此，在本实施例中，还需要获取全局图像特征以及局部关键区域的图像特征之间的关联特征，关联特征指特征之间存在的某种关联性的特征，能进一步表征待分类图像的特征，将其作为后续分类依据的特征之一，考虑了全局图像特征和局部关键区域的图像特征的关联性，可提高图像分类的准确性。
S250：根据待分类图像的全局图像特征、待分类图像的局部关键区域的图像特征以及待分类图像的关联特征，对待分类图像进行分类确定分类结果。
即将待分类图像的全局图像特征、局部关键区域的图像特征以及关联特征组合作为分类时所依据的特征，这些特征组合后能更加全面地表征图像，不但考虑了全局，而且考虑了局部以及全局和局部之间的关联性，据此进行分类时，能够获得准确的分类结果。
上述图像分类方法，根据待分类图像的全局图像特征、局部关键区域的图像特征以及关联特征，对待分类图像进行分类。即在进行图像分类的过程中，不但考虑了全局图像特征，还考虑了局部关键区域的图像特征，丰富了图像分类所依据的图像特征信息，依据上述丰富的特征信息进行图像分类时，能够准确分类，提高图像分类准确性。
在一个示例中，上述图像分类方法可以为细粒度图像分类方法，可用于对细粒度图像的分类。
如图3所示，其中一个实施例中，上述根据待分类图像以及待分类图像的全局图像特征，确定待分类图像的局部关键区域的步骤S220包括：
S321：根据待分类图像的全局图像特征，获取待分类图像的各子区域分别对应的子图像特征。
各子区域可一起构成整个待分类图像，全局图像特征中包括各子区域的子图像特征，在获得待分类图像的全局图像特征之后，根据全局图像特征，可获取待分类图像中各子区域分别对应的子图像特征。可以理解，待分类图像对应的任意一个子图像特征对应一个子区域。例如，待分类图像Ig对应的全局图像特征为28×28个512维度的向量，可以理解为512张大小28×28为的特征图，有28×28个特征向量，每个特征向量中有512个特征元素，28×28个特征向量组成了待分类图像Ig的全局图像特征，其中，每个特征向量对应一个子区域的子图像特征，从而，可根据待分类图像Ig的全局图像特征获得各子区域分别对应的子图像特征。
S322：获取各子区域的子图像特征分别对应的权重。
子区域的子图像特征是对待分类图像的子区域的特征表征，为衡量各子区域的重要程度，在本实施例中，需要给各子区域的子图像特征分别赋予权 重，权重越大，表示该子区域越重要。
S323：根据各子区域的子图像特征、各子区域的子图像特征的权重以及待分类图像，确定待分类图像的局部关键区域。
各子区域的子图像特征分别对应的权重获取后，可获知各子区域的重要程度，即可从待分类图像中提取对应的局部关键区域。如此，确定局部关键区域的过程中，考虑了各子区域的子图像特征的权重，可确保局部关键区域的准确性，进而可提高后续分类准确性。在一个示例中，可选取各子区域中权重最大的子区域作为局部关键区域，也可以从各子区域中选取权重大于预设值的子区域，根据权重大于预设值的子区域，确定局部关键区域。由于权重大于预设值的子区域可能存在重合，从而，可将权重大于预设值的各子区域求并集，得到局部关键区域。
可以理解，上述获取各子区域的子图像特征分别对应的权重以及根据各子区域的子图像特征、各子区域的子图像特征的权重以及待分类图像，确定待分类图像的局部关键区域的过程，是根据attention机制(注意力机制)确定局部关键区域的过程。
如图4所示，其中一个实施例中，上述获取各子区域的子图像特征分别对应的权重的步骤S322包括：
S421：获取预设高维特征向量。
在一个示例中，预设高维特征可以是预先根据卷积神将网络对样本图像进行处理得到卷积特征，然后根据卷积特征获取的样本图像在预设局部关键区域对应的子图像特征，其维度与待分类图像的子区域的子图像特征的维度是相同的。
S422：将各子区域的子图像特征分别与预设高维特征向量进行合并，获得各子区域的组合特征。
可以理解为将各子区域的子图像特征分别与预设高维特征向量进行拼接，组合特征中的元素数量为子区域的子图像特征的元素数量与预设高维特征向量中元素数量的和。
S423：将各子区域的组合特征分别作为多层感知器的输入，根据多层感知器分别进行预测，获得各子区域的子图像特征分别对应的权重。
多层感知器(MLP，Multilayer Perceptron)是一种前馈人工神经网络模型，其将输入的多个数据集映射到单一的输出的数据集上。在本实施例中，多层感知器的输入层的节点数量与组合特征中元素数量相同，将子区域的组合特征作为多层感知器的输入，根据多层感知器将输入的组合特征映射到单一的输出数据上，得到一个子区域的子图像特征的权重，分别对各子区域的组合特征重复上述映射过程，得到各子区域的子图像特征分别对应的权重。
在一个示例中，获得各子区域的子图像特征分别对应的权重的公式如下：
Figure PCTCN2018106196-appb-000001
其中，
Figure PCTCN2018106196-appb-000002
为待分类图像的第i个子区域的子图像特征的权重，h i为待分类图像的第i个子区域的组合特征，h i为待分类图像的第i个子区域的子图像特征z i与预设高维特征向量γ合并而成，MLP为多层感知器，表示以h i作为多层感知器的输入，多层感知器根据输入进行预测得到输出结果，该输出结果即为
Figure PCTCN2018106196-appb-000003
待分类图像对应的全局图像特征为L×M×M维度的特征，其中，L和M分别为大于或等于1的正整数，可以理解为M×M个L维度的向量，每个子区域对应一个子图像特征，则
Figure PCTCN2018106196-appb-000004
也可以理解为全局图像特征中第i个子图像特征，且子图像特征中的元素数量为L。在一个示例中，L可以为512，可以M为28。
在其中一个实施例中，在通过多层感知器进行预测得到权重之前，需要进行训练获得多层感知器，即上述步骤S423之前，需要预先获得多层感知器。从而，在将各子区域的组合特征分别作为多层感知器的输入之前，还包括以下步骤：提取各样本图像的全局图像特征；根据样本图像的全局图像特征，获取所述样本图像的各子区域分别对应的子图像特征；对样本图像的各子区域的子图像特征分别进行权重初始化，获得各子区域的子图像特征分别对应的初始权重；根据样本图像的各子区域的子图像特征以及各子区域的子图像特征的初始权重，获得各样本图像的图像上下文向量；分别对各所述图像上 下文向量进行加权处理，获得各加权图像上下文向量；将各所述加权图像上下文向量分别与预设高维特征向量合并，获得各训练输入特征；获取初始多层感知器；根据各样本图像对应的各训练输入特征，对初始多层感知器进行训练，获得多层感知器。
样本图像可以理解为用于训练的图像，是已知分类的图像。本实施例中上述步骤为进行多层感知器训练的过程，获取样本图像的全局图像特征的过程与上述获取待分类图像的全局图像特征的过程类似，另外，获取样本图像的各子区域分别对应的子图像特征的获取过程与上述获取待分类图像的各子区域分别对应的子图像特征的过程类似，区别点仅在于图像不同，前者是样本图像，后者是待分类图像。
图像上下文向量为待分类图像的显著区域的特征表达，可以理解为局部关键区域的特征表达。在得到样本图像的各子区域的子图像特征的初始权重后，即可根据样本图像的各子区域的子图像特征的初始权重对样本图像的各子区域的子图像特征进行处理，获得图像上下文向量。
在一个示例中，获得图像上下文向量S的公式为：
Figure PCTCN2018106196-appb-000005
q i为样本图像的第i各子区域的子图像特征，a i为样本图像的第i各子区域的子图像特征的初始权重。
获得加权图像上下文向量R的公式为：
R＝β·S。
β为图像上下文向量的权重，根据对各所述图像上下文向量进行加权处理即可获得加权图像上下文向量R。
再根据加权图像上下文向量R与预设高维特征向量γ合并得到训练输入特征，由于样本图像的数量有多个，从而，可获得各训练输入特征，根据各训练输入特征对初始多层感知器进行训练，得到上述多层感知器。具体地，在训练过程中，采用softmax回归函数σ s进行训练，训练结果标签label1通过 以下公式表示:
label1＝Arg max(σ s(R,W1,b1))。
其中，W1为权值矩阵，b1为偏置向量，Arg max(f(e))是使得f(e)取得最大值时所对应的变量e的值。
如图5所示，其中一个实施例中，上述根据待分类图像的全局图像特征以及待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征的步骤S240包括：
S541：对待分类图像的全局图像特征进行转置处理，获得转置特征。
对于一个矩阵，把它的第一行变成第一列，第二行变成第二列，......，最末一行变为最末一列，从而得到一个新的矩阵，这一过程称为矩阵的转置。由于图像的不同特征维度之间存在着有价值的联系，为了挖掘全局图像特征与局部关键区域的图像特征之间存在的有效关联信息，需要根据全局图像特征与局部关键区域的图像特征之间的不同特征维度(例如，行与列分别代表不同特征维度)之间的点乘来获取两者之间的关联，从而，首先，将全局图像特征进行转置处理，获得转置特征。
S542：将转置特征与待分类图像的局部关键区域的图像特征相乘，获得乘积特征。
对于矩阵a和b，矩阵a的大小为m行n列，矩阵b的大小为n行m列，矩阵a与矩阵b相乘后的矩阵结果的大小为m行m列，矩阵结果中第i行第j列位置点上的值为矩阵a第i行上n个数与第二个矩阵第j列上的n个数对应相乘后所得的n个乘积之和。
将转置特征与局部关键区域的图像特征相乘即是将转置矩阵的一行的元素分别与局部关键区域的图像特征的各列的元素对应相乘求和，得到乘积特征的一行，直到转置矩阵的各行执行完上述相乘求和，得到乘积特征，从而，实现将全局图像特征与局部关键区域的图像特征之间不同特征维度之间的乘积，获得存在关联的特征即乘积特征。
具体地，获得乘积特征X的公式如下：
X＝A TB。
其中，A为待分类图像的全局图像特征，B为待分类图像的局部关键区域的图像特征，A T为转置特征。A iB j∈R M×M，A i为待分类图像的全局图像特征A中第i个特征向量，B j为待分类图像的局部关键区域的图像特征B中第j个特征向量。
S543：平均池化乘积特征，获得待分类图像的关联特征。
池化可以理解为压缩，是对不同位置的特征进行聚合统计，例如，计算图像一个区域上的某个特定特征的平均值作为该区域的一个值，如此，可降低维度，同时改善结果，不容易过拟合，这种聚合的操作称为池化。池化包括平均池化和做大池化，上述将区域上的某个特定特征的平均值，作为该区域的一个值，称为平均池化，将区域上的某个特定特征的最大值作为该区域的一个值，称为最大池化。
在得到具有关联特性的乘积特征之后，为避免维度太多以及数据量大影响分类效率，需要对乘积特征进行池化处理，以降低乘积特征的维度。在本实施例中，对乘积特征进行平均池化处理，得到关联特征，以确保关联特征的精确度。平均池化后，得到的待分类图像的关联特征为L×L形式的特征。
其中一个实施例中，提取待分类图像的全局图像特征的步骤S210包括：根据第一卷积神经网络对待分类图像进行特征提取，获得待分类图像的全局图像特征。
上述提取局部关键区域的图像特征的步骤S230包括：根据第二卷积神经网络对待分类图像的局部关键区域进行特征提取，获得待分类图像的局部关键区域的图像特征。
卷积神经网络是一种前馈神经网络，人工神经元可以响应周围单元，可以进行大型图像处理，卷积神经网络包括卷积层和池化层。通过卷积神经网络进行特征提取时得到的特征结果为三个空间维度的特征，可以理解为得到多个特征图，比如，通过卷积图像网络对图像A进行处理，得到的全局图像及特征为512×28×28形式的特征，可以理解为是512张大小为28×28的特征 图，也可以理解为28×28个512维的单个向量，即单个向量中有512个元素。在本实施例中，可分别采用对应的卷积神经网络进行特征提取，通过第一卷积神经网络与第二卷积神经网络分别进行特征提取时，满足得到的全局图像特征和局部关键区域的图像特征的维度是相同。
如图6所示，其中一个实施例中，上述根据待分类图像的全局图像特征、待分类图像的局部关键区域的图像特征以及待分类图像的关联特征，对待分类图像进行分类确定分类结果的步骤S250包括：
S651：将待分类图像的全局图像特征转化为一维全局图像特征向量，将待分类图像的局部关键区域的图像特征转化为一维局部图像特征向量，将待分类图像的关联特征转化为一维关联特征向量。
S652：合并一维全局图像特征向量、一维局部图像特征向量以及一维关联特征向量，获得待分类图像对应的一维图像合并特征向量。
其中，一维图像合并特征向量中的元素数量为一维全局图像特征向量中的元素数量、一维局部图像特征向量的元素数量和一维关联特征向量的元素数量之和。
S653：将一维图像合并特征向量作为已训练分类模型的输入，根据已训练分类模型对待分类图像进行分类确定分类结果。
一维向量可以理解为一行向量(其中可以有多列，即一个行向量中可以有多个元素)或一列向量(其中可以有多行)。由于在提取特征过程中获得的全局图像特征和局部关键区域的图像特征可能是多维的矩阵形式，然而，在利用分类模型进行分类时，分类模型的输入应当为一维的向量形式，一维的向量中各元素分别作为分类输入层的节点的输入，从而，在进行图像分类时，需要将获得的全局图像特征转化为一维全局图像特征向量，将局部关键区域的图像特征转化为一维局部图像特征向量，将关联特征转化为一维关联特征向量，然后为了提高分类准确性，将上述的一维的特征向量合并成一维图像合并特征向量，将其作为已训练分类模型的输入，利用已训练分类模型进行图像分类，获得初始分类结果。具体地，分类类别有多种，初始分类结果可 以为待分类图像分别属于各分类类别的概率，根据待分类图像分别属于各分类类别的概率，确定最终分类结果即上述对待分类图像进行分类确定的分类结果，可以是将初始分类结果中最大概率对应的分类类别作为最终分类结果。在一个示例中，已训练分类模型可以为已训练softmax线性回归模型。
在一个示例中，在将上述图像特征转化为一维的特征向量的过程中，可以通过将图像特征的每维中的元素合并成一维。例如，对一个大小为10×10矩阵形式的特征，将其转换为一维的向量后，该一维的向量可以是单行向量，其中，包括100个元素，可以理解为1行100列的一维向量。
在一个示例中，为避免一维关联特征向量中元素数量较多导致维度过高的问题，还可对一维关联特征向量进行降维，以更新一维关联特征向量。
如图7所示，在其中一个实施例中，将一维图像合并特征向量作为已训练分类模型的输入，根据已训练分类模型对待分类图像进行分类的步骤S653之前，还包括步骤：
S710：提取各样本图像的全局图像特征。
S720：根据各样本图像以及各样本图像的全局图像特征，确定各样本图像的局部关键区域。
S730：提取各样本图像的局部关键区域的图像特征。
S740：根据各样本图像的全局图像特征以及各样本图像的局部关键区域的图像特征，获取各样本图像分别对应的关联特征。
可以理解，在将一维图像合并特征向量作为已训练分类模型的输入进行分类之前，需要得到已训练分类模型，上述过程即为得到已训练分类模型的过程。由于在预测(即对待分类图像分类)过程中，采用根据待分类图像的全局图像特征、局部关键区域的图像特征和待分类图像的关联特征，从而，训练和预测时需要对应，以此保证根据训练后的分类模型进行分类时的准确性。
即首先对各样本图像分别进行特征提取，获得各样本图像的全局图像特征、各样本图像的局部关键区域的图像特征以及各样本图像的关联特征，上 述对样本图像的特征提取过程与对待分类图像的特征提取过程是类似的，不同之处仅在于待提取特征的图像，在本实施例中，待提取特征的图像是各样本图像，而进行预测时，待提取特征的图像是待分类图像。
S750：初始化分类模型，获得初始分类模型。
分类模型对应有模型参数，则在进行训练之前，需要对分类模型进行初始化，即初始化分类模型的结构以及模型参数，得到初始模型参数，训练过程即是对初始分类模型中的初始模型参数不断修正的过程，直到训练结果满足要求，得到最优模型参数，该最优模型参数对应的分类模型即为训练后的模型。在一个示例中，初始分类模型可以为初始softmax线性回归模型。
S760：根据各样本图像的全局图像特征、各样本图像的局部关键区域的图像特征以及各样本图像的关联特征，对初始分类模型进行训练，获得已训练分类模型。
在本实施例中，是通过各样本图像的全局图像特征、各样本图像的局部关键区域的图像特征以及各样本图像的关联特征，对初始分类模型中的初始模型参数进行不断修正，实现对初始分类模型进的训练，得到已训练分类模型，后续即可利用该已训练分类模型对待分类图像进行分类。
在一个示例中，其在根据softmax线性回归模型进行分类时，采用的梯度下降公式为：
Figure PCTCN2018106196-appb-000006
以及
Figure PCTCN2018106196-appb-000007
其中，
Figure PCTCN2018106196-appb-000008
为softmax线性回归模型中的代价函数，用来衡量训练样本对线性模式的拟合程度。
如图8所示，在其中一个实施例中，初始分类模型包括第一初始分类模型以及第二初始分类模型。
在本实施例中，上述根据各样本图像的全局图像特征、各样本图像的局部关键区域的图像特征以及各样本图像的关联特征，对初始分类模型进行训练，获得已训练分类模型的步骤S760包括：
S861：根据各样本图像的全局图像特征对第一初始分类模型进行训练获得第一目标分类模型；
S862：根据各样本图像的局部关键区域的图像特征对第二初始分类模型进行训练获得第二目标分类模型；
即针对不同的特征，初始化对应的初始分类模型，再根据不同特征分别对不同的初始分类模型进行分开训练，即第一初始分类模型是根据样本图像的全局图像特征初始化得到，可以理解，第一初始分类模型的输入层的节点数量是根据样本图像的全局图像特征中元素数量决定，第二初始分类模型是根据样本图像的局部关键区域的图像特征初始化得到，可以理解，第二初始分类模型的输入层的节点数量是根据样本图像的局部关键区域的图像特征中元素数量决定，如此，可得到两个独立的目标分类模型。
S863：合并第一目标分类模型与第二目标分类模型，获得初始合并分类模型；
S864：根据各样本图像的关联特征对初始合并分类模型进行训练，获得已训练分类模型。
考虑到不同特征之间的关联性，还需要根据各样本图像的关联特征进行分类模型训练，然而，之前独立的两个目标分类模型虽不能满足根据关联特征的训练要求，但其分别根据得到关联特征所依据的各样本图像的全局图像特征和各样本图像的局部关键区域的图像特征进行训练，存在一定的关联性，从而，将第一目标分类模型与第二目标分类模型，获得初始合并分类模型，再根据各样本图像的关联特征对初始合并分类模型进行训练，获得已训练分类模型。如此，即考虑了特征的独立性，有考虑了特征的关联性，提高已训练分类模型的准确性，进而提高分类准确性。在一个示例中，第一初始分类模型和第二初始分类模型可以分别为初始softmax线性回归模型。
在一个示例中，第一目标分类模型对应有模型结构和已训练完成后的模型参数，第二目标分类模型对应有模型结构和已训练完成后的模型参数，合并两种分类模型得到的初始合并分类模型可以理解为对应的结构为两种分类模型中各层节点的对应叠加，对应的模型参数是两种分类模型的模型参数的叠加。例如，第一目标分类模型和第二目标分类模型分别为三层结构，第一 目标分类模型的第一层有512个输入节点，第二层有500个节点，第三层有5个节点(可以理解为输出层)，第二目标分类模型的第一层有512个输入节点，第二层有500个节点，第三层有5个节点，合并时，两个分类模型的第一层的输入节点叠加为1024个输入节点，两个分类模型的第二层的节点叠加1000个节点，两个分类模型的第三层的节点叠加为10个节点，两个分类模型已训练后的模型参数保留，在对其他缺少的模型参数(两层中来自不同分类模型的节点之间的模型参数在合并初期是缺失的)进行初始化，得到初始合并分类模型。
在另一个实施例中，初始分类模型可以是根据样本图像的全局图像特征、样本图像的局部关键区域的图像特征以及样本图像的关联特征初始化得到的，可以理解，初始分类模型的输入层的节点数量是样本图像的全局图像特征中元素数量、样本图像的局部关键区域的图像特征中元素数量和样本图像的关联特征中元素数量的和决定，则可根据各样本图像的全局图像特征、各样本图像的局部关键区域的图像特征以及各样本图像的关联特征进行合，获得样本合并特征，根据样本合格特征对初始分类模型进行训练，获得已训练分类模型。
在本实施例中，根据已训练分类模型对待分类图像进行分类得到分类标签的公式为：
label2＝Arg max(σ s(c,W2,b2))。
其中，W2为权值矩阵，b2为偏置向量，c为一维图像合并特征向量。
下面以一具体实施例对上述图像分类方法加以具体说明。
如图9所示，为通过本具体实施例的图像分类方法进行分类的原理图，首先，根据卷积神经网络对待分类图像Ig进行特征提取，获得全局图像特征A。
然后，根据待分类图像Ig、待分类图像Ig的全局图像特征A以及attention机制获取待分类图像Ig对应的局部关键区域Ia；
其次，根据卷积神经网络对局部关键区域Ia进行特征提取，获得局部关 键区域Ia的图像特征B。
再次，待分类图像Ig的全局图像特征A和局部关键区域Ia的图像特征B分别有对应单独的softmax线性回归模型的分类通道，即可根据待分类图像Ig的全局图像特征A对应的一维全局图像特征以及对应的softmax线性回归模型进行图像分类，也可根据局部关键区域Ia的图像特征B对应的一维局部图像特征向量以及对应的softmax线性回归模型进行图像分类。
为了丰富特征类型，通过丰富的特征信息进行分类，提高准确性，根据待分类图像Ig的全局图像特征A和局部关键区域Ia的图像特征B确定关联特征，可将待分类图像Ig的全局图像特征A对应的一维全局图像特征向量、局部关键区域Ia的图像特征B对应的一维局部图像特征向量以及关联特征对应的一维关联特征向量合并，获得待分类图像对应的一维图像合并特征向量，将一维图像合并特征向量作为已训练分类模型的输入，根据已训练分类模型对待分类图像进行分类，能够提高分类准确性。
综上，本具体实施例的图像分类方法可提高分类能力，确保分类的准确性，为后续用户查找提供便利，且对图像进行个性化推荐提供良好的基础。
上述图像分类方法可应用于个性化推荐、机器人视觉识别以及自动驾驶物体识别等领域，以应用于个性化推荐为例，如图10所示，本申请还提供一种实施例的个性化推荐方法，本实施例主要以该方法应用于上述图1中的服务器120来举例说明，包括以下步骤：
S110：获取通过上述图像分类方法对待分类图像进行分类确定的分类结果。
通过上述图像分类方法对待分类图像进行分类时，可获得准确的分类结果。
S120：查找与分类结果对应的待推荐对象信息。
S130：推送待推荐对象信息。
对象可理解为目标事务，可在预设推荐信息数据库中存储各对象的信息，其中，对象可包括商品、图像、物品以及人物等，不同用户在终端使用的图 像各异，可利用这图像为用户提供个性化推荐，首先对这些图像进行分类，获得准确的分类结果，并查找与分类结果对应的待推荐对象信息，以提高待推荐对象信息的准确性，然后推送待推荐对象信息，实现个性化推荐，以便用户查看。
具体地，上述个性化推荐方法可应用于服务器或终端，在应用于服务器时，上述推送所述待推荐对象信息的步骤S130具体可以为向终端发送待推荐对象信息，用户可查看终端上接收的待推荐对象信息。另外，上述个性化推荐方法在应用于终端时，通过终端自身进行待推荐对象信息的查找，即在终端上得到待推荐对象信息，用户可通过查看终端即可获知待推荐对象信息。
在一个示例中，在个性化推荐方法应用于服务器时，若上述图像分类方法应用于服务器，服务器通过图像分类方法确定待分类图像的分类结果后，可直接获取服务器确定的分类结果，并基于该分类结果查找待推荐对象信息并将其推荐给终端。若上述图像分类方法应用于终端，服务器可从终端获取待分类图像的分类结果，服务器也可基于待分类图像的全局图像特征、局部关键区域的图像特征以及关联特征确定分类结果。后续再基于分类结果查找待推荐对象信息并将其推荐给终端。
在个性化推荐方法应用于终端时，若上述图像分类方法应用于服务器，则终端可从服务器获取待分类图像的分类结果，终端也可基于待分类图像的全局图像特征、局部关键区域的图像特征以及关联特征确定分类结果。然后再基于分类结果查找待推荐对象信息并推送给用户。若上述图像分类方法应用于终端，终端通过图像分类方法确定待分类图像的分类结果后，可直接获取该分类结果并基于该分类结果查找待推荐对象信息，再将其推送给用户。
上述个性化推荐方法，可获取通过上述图像分类方法确定的分类结果，提高分类结果的准确性，然后根据分类结果查找对应的待推荐对象信息时，可准确获取待推荐对象信息并推送，实现待推荐对象信息的推荐，提高推荐准确性。
如图11所示，提供一种实施例的图像分类装置，可设置于图1的服务器 120或终端10中，包括：
全局特征提取模块110，用于提取待分类图像的全局图像特征。
确定模块120，用于根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域。
局部特征提取模块130，用于提取所述待分类图像的局部关键区域的图像特征。
关联特征获取模块140，用于根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征。
分类模块150，用于根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类确定分类结果。
上述图像分类装置，根据全局图像特征、局部关键区域的图像特征以及关联特征，对待分类图像进行分类。即在进行图像分类的过程中，不但考虑了全局图像特征，还考虑了局部关键区域的图像特征，丰富了图像分类所依据的图像特征信息，依据上述丰富的特征信息进行图像分类时，能够准确分类，提高图像分类准确性。
如图12所示，在其中一个实施例中，上述确定模块120包括：
区域特征获取模块221，用于根据所述待分类图像的全局图像特征，获取所述待分类图像的各子区域分别对应的子图像特征。
区域权重获取模块222，用于获取各所述子区域的子图像特征分别对应的权重。
局部区域确定模块223，用于根据各所述子区域的子图像特征、各所述子区域的子图像特征的权重以及所述待分类图像，确定所述待分类图像的局部关键区域。
如图13所示，在其中一个实施例中，上述区域权重获取模块222包括：
预设向量获取模块321，用于获取预设高维特征向量。
向量合并模块322，用于将各所述子区域的子图像特征分别与所述预设 高维特征向量进行合并，获得各所述子区域的组合特征；
预测模块323，用于将各所述子区域的组合特征分别作为多层感知器的输入，根据所述多层感知器分别进行预测，获得各所述子区域的子图像特征分别对应的权重。
如图14所示，在其中一个实施例中，上述关联特征获取模块140包括：
转置模块441，用于对所述待分类图像的全局图像特征进行转置处理，获得转置特征。
乘积模块442，用于将所述转置特征与所述待分类图像的局部关键区域的图像特征相乘，获得乘积特征。
池化模块443，用于平均池化所述乘积特征，获得所述关联特征。
在其中一个实施例中，上述全局特征提取模块110，具体用于根据第一卷积神经网络对所述待分类图像进行特征提取，获得所述待分类图像的全局图像特征。
上述局部特征提取模块130，具体用于根据第二卷积神经网络对所述待分类图像的局部关键区域进行特征提取，获得所述待分类图像的局部关键区域的图像特征。
如图15所示，在其中一个实施例中，上述分类模块150包括：
转换模块551，用于将所述待分类图像的全局图像特征转化为一维全局图像特征向量，将所述待分类图像的局部关键区域的图像特征转化为一维局部图像特征向量，将所述待分类图像的关联特征转化为一维关联特征向量。
特征合并模块552，用于合并所述一维全局图像特征向量、所述一维局部图像特征向量以及所述一维关联特征向量，获得所述待分类图像对应的一维图像合并特征向量。
其中，一维图像合并特征向量中的元素数量为一维全局图像特征向量中的元素数量、一维局部图像特征向量的元素数量和一维关联特征向量的元素数量之和。
图像分类模块553，用于将所述一维图像合并特征向量作为已训练分类 模型的输入，根据所述已训练分类模型对所述待分类图像进行分类。
在其中一个实施例中，上述图像分类装置还包括初始化模块和训练模块。
所述全局特征提取模块110，还用于提取各样本图像的全局图像特征；
所述确定模块120，还用于根据各所述样本图像以及各所述样本图像的全局图像特征，确定各所述样本图像的局部关键区域；
所述局部特征提取模块130，还用于提取各所述样本图像的局部关键区域的图像特征；
所述关联特征获取模块140，还用于根据各所述样本图像的全局图像特征以及各所述样本图像的局部关键区域的图像特征，获取各所述样本图像分别对应的关联特征；
上述初始化模块，用于初始化分类模型，获得初始分类模型；
上述训练模块，用于根据各所述样本图像的全局图像特征、各所述样本图像的局部关键区域的图像特征以及各所述样本图像的关联特征，对所述初始分类模型进行训练，获得所述已训练分类模型。
在其中一个实施例中，所述初始分类模型包括第一初始分类模型以及第二初始分类模型。
在本实施例中，上述训练模块包括：第一训练模块、第二训练模块、模型合并模块以及第三训练模块。
第一训练模块，用于根据各所述样本图像的全局图像特征对所述第一初始分类模型进行训练获得第一目标分类模型；
第二训练模块，用于根据各所述样本图像的局部关键区域的图像特征对所述第二初始分类模型进行训练获得第二目标分类模型；
模型合并模块，用于合并所述第一目标分类模型与所述第二目标分类模型，获得初始合并分类模型；
第三训练模块，用于根据各所述样本图像的关联特征对所述初始合并分类模型进行训练，获得所述已训练分类模型。
在另一个实施例中，初始分类模型可以是根据样本图像的全局图像特征、 样本图像的局部关键区域的图像特征以及样本图像的关联特征初始化得到的，可以理解，初始分类模型的输入层的节点数量是样本图像的全局图像特征中元素数量、样本图像的局部关键区域的图像特征中元素数量和样本图像的关联特征中元素数量的和决定，则可根据各样本图像的全局图像特征、各样本图像的局部关键区域的图像特征以及各样本图像的关联特征进行合，获得样本合并特征，根据样本合格特征对初始分类模型进行训练，获得已训练分类模型。
上述图像分类装置中的技术特征分别与上述图像分类方法中的技术特征是对应的，在此不再赘述。
如图16所示，本申请还提供一种实施例的个性化推荐装置，可设置于图1的服务器120或终端10中，包括：
分类结果获取模块610，用于获取通过上述图像分类装置对所述待分类图像进行分类确定的分类结果。
查找模块620，用于查找与所述分类结果对应的待推荐对象信息。
推送模块630，用于推送所述待推荐对象信息。
上述个性化推荐装置，可获取通过上述图像分类方法确定的分类结果，提高分类结果的准确性，然后根据分类结果查找对应的待推荐对象信息时，可准确获取待推荐对象信息并推送，实现待推荐对象信息的推荐，提高推准确性。
图17示出了一个实施例中计算机设备的内部结构图。该计算机设备具体可以是图1中的终端110或服务器120，可以理解上述方法可以由该计算机设备实施。如图17所示，该计算机设备包括通过系统总线连接的处理器、存储器和网络接口。其中，存储器包括非易失性存储介质和内存储器。该计算机设备的非易失性存储介质存储有操作系统，还可存储有计算机程序，该计算机程序被处理器执行时，可使得处理器实现上述图像分类方法和个性化推荐方法。计算机设备的处理器用于提供计算和控制能力，支撑整个计算机设备的运行。计算机设备的处理器用于提供计算和控制能力，支撑整个计算机 设备的运行。该内存储器中也可储存有计算机程序，该计算机程序被处理器执行时，可使得处理器执行图像分类方法和个性化推荐方法。网络接口用于与网络130连接和通信。在该计算机设备为终端110时，该计算机设备还可以包括输入装置和显示屏，计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏，计算机设备的输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。
本领域技术人员可以理解，图17中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，本申请提供的图像分类装置可以实现为一种计算机程序的形式，计算机程序可在如图17所示的计算机设备上运行。计算机设备的存储器中可存储组成该图像分类装置的各个程序模块，比如，图11所示的全局特征提取模块110、确定模块120、局部特征提取模块130、关联特征获取模块140和分类模块150。各个程序模块构成的计算机程序使得处理器执行本说明书中描述的本申请各个实施例的图像处理方法中的步骤。
例如，图17所示的计算机设备可以通过如图11所示的图像处理装置中的全局特征提取模块110执行步骤S210。计算机设备可通过确定模块120执行步骤S220。计算机设备可通过局部特征提取模块130执行步骤S230。计算机设备可通过关联特征获取模块140执行步骤S240。计算机设备可通过分类模块150执行步骤S240。
在一个实施例中，本申请提供的个性化推荐装置可以实现为一种计算机程序的形式，计算机程序可在如图17所示的计算机设备上运行。计算机设备的存储器中可存储组成该个性化推荐装置的各个程序模块。各个程序模块构成的计算机程序使得处理器执行本说明书中描述的本申请各个实施例的个性化推荐方法中的步骤。
本申请提供了一种实施例的计算机设备，包括存储器、处理器以及存储在存储器上并可在处理器上运行的计算机程序，处理器执行计算机程序时实现上述图像分类方法的步骤。
本申请提供了一种实施例的计算机设备，包括存储器、处理器以及存储在存储器上并可在处理器上运行的计算机程序，处理器执行计算机程序时实现上述个性化推荐方法的步骤。
本申请提供一种实施例的计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现上述图像分类方法的步骤。
本申请提供一种实施例的计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现上述个性化推荐方法的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，该程序可存储于一非易失性计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上实施例仅表达了本申请的几种实施方式，其描述较为具体和详细， 但并不能因此而理解为对本申请专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (36)

  1. 一种图像分类方法，其特征在于，该方法由计算机设备实施，包括以下步骤：
    提取待分类图像的全局图像特征；
    根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域；
    提取所述待分类图像的局部关键区域的图像特征；
    根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征；
    根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类确定分类结果。
  2. 根据权利要求1所述的图像分类方法，其特征在于，所述根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域的步骤包括：
    根据所述待分类图像的全局图像特征，获取所述待分类图像的各子区域分别对应的子图像特征；
    获取各所述子区域的子图像特征分别对应的权重；
    根据各所述子区域的子图像特征、各所述子区域的子图像特征的权重以及所述待分类图像，确定所述待分类图像的局部关键区域。
  3. 根据权利要求2所述的图像分类方法，其特征在于，所述获取各所述子区域的子图像特征分别对应的权重的步骤包括：
    获取预设高维特征向量；
    将各所述子区域的子图像特征分别与所述预设高维特征向量进行合并，获得各所述子区域的组合特征；
    将各所述子区域的组合特征分别作为多层感知器的输入，根据所述多层感知器分别进行预测，获得各所述子区域的子图像特征分别对应的权重。
  4. 根据权利要求1-3中任意一项所述的图像分类方法，其特征在于，所述根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的子图像特征，获取待分类图像的关联特征的步骤包括：
    对所述待分类图像的全局图像特征进行转置处理，获得转置特征；
    将所述转置特征与所述待分类图像的局部关键区域的子图像特征相乘，获得乘积特征；
    平均池化所述乘积特征，获得所述待分类图像的关联特征。
  5. 根据权利要求1所述的图像分类方法，其特征在于，
    所述提取待分类图像的全局图像特征的步骤包括：根据第一卷积神经网络对所述待分类图像进行特征提取，获得所述待分类图像的全局图像特征；
    所述提取所述局部关键区域的图像特征的步骤包括：根据第二卷积神经网络对所述待分类图像的局部关键区域进行特征提取，获得所述待分类图像的局部关键区域的图像特征。
  6. 根据权利要求1所述的图像分类方法，其特征在于，所述根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类的步骤包括：
    将所述待分类图像的全局图像特征转化为一维全局图像特征向量，将所述待分类图像的局部关键区域的图像特征转化为一维局部图像特征向量，将所述待分类图像的关联特征转化为一维关联特征向量；
    合并所述一维全局图像特征向量、所述一维局部图像特征向量以及所述一维关联特征向量，获得所述待分类图像对应的一维图像合并特征向量；其中，所述一维图像合并特征向量中的元素数量为所述一维全局图像特征向量中的元素数量、所述一维局部图像特征向量的元素数量和所述一维关联特征向量的元素数量之和；
    将所述一维图像合并特征向量作为已训练分类模型的输入，根据所述已训练分类模型对所述待分类图像进行分类。
  7. 一种个性化推荐方法，其特征在于，该方法由计算机设备实施，包括：
    获取待分类图像对应的分类结果；其中，所述分类结果通过提取待分类图像的全局图像特征；根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域；提取所述待分类图像的局部关键区域的图像特征；根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征；根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类确定；
    查找与所述分类结果对应的待推荐信息；
    推送所述推荐信息。
  8. 根据权利要求7所述的方法，其特征在于，所述根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域，包括：
    根据所述待分类图像的全局图像特征，获取所述待分类图像的各子区域分别对应的子图像特征；
    获取各所述子区域的子图像特征分别对应的权重；
    根据各所述子区域的子图像特征、各所述子区域的子图像特征的权重以及所述待分类图像，确定所述待分类图像的局部关键区域。
  9. 根据权利要求8所述的方法，其特征在于，所述获取各所述子区域的子图像特征分别对应的权重，包括：
    获取预设高维特征向量；
    将各所述子区域的子图像特征分别与所述预设高维特征向量进行合并，获得各所述子区域的组合特征；
    将各所述子区域的组合特征分别作为多层感知器的输入，根据所述多层感知器分别进行预测，获得各所述子区域的子图像特征分别对应的权重。
  10. 根据权利要求7-9中任意一项所述的方法，其特征在于，所述根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的子图像特征，获取待分类图像的关联特征，包括：
    对所述待分类图像的全局图像特征进行转置处理，获得转置特征；
    将所述转置特征与所述待分类图像的局部关键区域的子图像特征相乘，获得乘积特征；
    平均池化所述乘积特征，获得所述待分类图像的关联特征。
  11. 根据权利要求7所述的方法，其特征在于，
    所述提取待分类图像的全局图像特征包括：根据第一卷积神经网络对所述待分类图像进行特征提取，获得所述待分类图像的全局图像特征；
    所述提取所述局部关键区域的图像特征包括：根据第二卷积神经网络对所述待分类图像的局部关键区域进行特征提取，获得所述待分类图像的局部关键区域的图像特征。
  12. 根据权利要求7所述的方法，其特征在于，所述根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类，包括：
    将所述待分类图像的全局图像特征转化为一维全局图像特征向量，将所述待分类图像的局部关键区域的图像特征转化为一维局部图像特征向量，将所述待分类图像的关联特征转化为一维关联特征向量；
    合并所述一维全局图像特征向量、所述一维局部图像特征向量以及所述一维关联特征向量，获得所述待分类图像对应的一维图像合并特征向量；其中，所述一维图像合并特征向量中的元素数量为所述一维全局图像特征向量中的元素数量、所述一维局部图像特征向量的元素数量和所述一维关联特征向量的元素数量之和；
    将所述一维图像合并特征向量作为已训练分类模型的输入，根据所述已训练分类模型对所述待分类图像进行分类。
  13. 一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机程序，其特征在于，所述计算机程序被所述处理器执行时，使得所述处理器执行如下步骤：
    提取待分类图像的全局图像特征；
    根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域；
    提取所述待分类图像的局部关键区域的图像特征；
    根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征；
    根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类确定分类结果。
  14. 根据权利要求13所述的计算机设备，其特征在于，所述根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域的步骤包括：
    根据所述待分类图像的全局图像特征，获取所述待分类图像的各子区域分别对应的子图像特征；
    获取各所述子区域的子图像特征分别对应的权重；
    根据各所述子区域的子图像特征、各所述子区域的子图像特征的权重以及所述待分类图像，确定所述待分类图像的局部关键区域。
  15. 根据权利要求14所述的计算机设备，其特征在于，所述获取各所述子区域的子图像特征分别对应的权重的步骤包括：
    获取预设高维特征向量；
    将各所述子区域的子图像特征分别与所述预设高维特征向量进行合并，获得各所述子区域的组合特征；
    将各所述子区域的组合特征分别作为多层感知器的输入，根据所述多层感知器分别进行预测，获得各所述子区域的子图像特征分别对应的权重。
  16. 根据权利要求13-15中任意一项所述的计算机设备，其特征在于，所述根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的子图像特征，获取待分类图像的关联特征的步骤包括：
    对所述待分类图像的全局图像特征进行转置处理，获得转置特征；
    将所述转置特征与所述待分类图像的局部关键区域的子图像特征相乘，获得乘积特征；
    平均池化所述乘积特征，获得所述待分类图像的关联特征。
  17. 根据权利要求13所述的计算机设备，其特征在于，所述提取待分类图像的全局图像特征的步骤包括：根据第一卷积神经网络对所述待分类图像进行特征提取，获得所述待分类图像的全局图像特征；
    所述提取所述局部关键区域的图像特征的步骤包括：根据第二卷积神经网络对所述待分类图像的局部关键区域进行特征提取，获得所述待分类图像的局部关键区域的图像特征。
  18. 根据权利要求13所述的计算机设备，其特征在于，所述根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类的步骤包括：
    将所述待分类图像的全局图像特征转化为一维全局图像特征向量，将所述待分类图像的局部关键区域的图像特征转化为一维局部图像特征向量，将所述待分类图像的关联特征转化为一维关联特征向量；
    合并所述一维全局图像特征向量、所述一维局部图像特征向量以及所述一维关联特征向量，获得所述待分类图像对应的一维图像合并特征向量；其中，所述一维图像合并特征向量中的元素数量为所述一维全局图像特征向量中的元素数量、所述一维局部图像特征向量的元素数量和所述一维关联特征向量的元素数量之和；
    将所述一维图像合并特征向量作为已训练分类模型的输入，根据所述已训练分类模型对所述待分类图像进行分类。
  19. 一种计算机设备，包括存储器和处理器，存储器中存储有计算机程序，其特征在于，计算机程序被处理器执行时，使得处理器执行如下步骤：
    获取待分类图像对应的分类结果；其中，所述分类结果通过提取待分类图像的全局图像特征；根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域；提取所述待分类图像的局部关 键区域的图像特征；根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征；根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类确定；
    查找与所述分类结果对应的待推荐信息；
    推送所述推荐信息。
  20. 根据权利要求19所述的计算机设备，其特征在于，所述根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域，包括：
    根据所述待分类图像的全局图像特征，获取所述待分类图像的各子区域分别对应的子图像特征；
    获取各所述子区域的子图像特征分别对应的权重；
    根据各所述子区域的子图像特征、各所述子区域的子图像特征的权重以及所述待分类图像，确定所述待分类图像的局部关键区域。
  21. 根据权利要求20所述的计算机设备，其特征在于，所述获取各所述子区域的子图像特征分别对应的权重，包括：
    获取预设高维特征向量；
    将各所述子区域的子图像特征分别与所述预设高维特征向量进行合并，获得各所述子区域的组合特征；
    将各所述子区域的组合特征分别作为多层感知器的输入，根据所述多层感知器分别进行预测，获得各所述子区域的子图像特征分别对应的权重。
  22. 根据权利要求19-21中任意一项所述的计算机设备，其特征在于，所述根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的子图像特征，获取待分类图像的关联特征，包括：
    对所述待分类图像的全局图像特征进行转置处理，获得转置特征；
    将所述转置特征与所述待分类图像的局部关键区域的子图像特征相乘，获得乘积特征；
    平均池化所述乘积特征，获得所述待分类图像的关联特征。
  23. 根据权利要求19所述的计算机设备，其特征在于，
    所述提取待分类图像的全局图像特征包括：根据第一卷积神经网络对所述待分类图像进行特征提取，获得所述待分类图像的全局图像特征；
    所述提取所述局部关键区域的图像特征包括：根据第二卷积神经网络对所述待分类图像的局部关键区域进行特征提取，获得所述待分类图像的局部关键区域的图像特征。
  24. 根据权利要求19所述的计算机设备，其特征在于，所述根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类，包括：
    将所述待分类图像的全局图像特征转化为一维全局图像特征向量，将所述待分类图像的局部关键区域的图像特征转化为一维局部图像特征向量，将所述待分类图像的关联特征转化为一维关联特征向量；
    合并所述一维全局图像特征向量、所述一维局部图像特征向量以及所述一维关联特征向量，获得所述待分类图像对应的一维图像合并特征向量；其中，所述一维图像合并特征向量中的元素数量为所述一维全局图像特征向量中的元素数量、所述一维局部图像特征向量的元素数量和所述一维关联特征向量的元素数量之和；
    将所述一维图像合并特征向量作为已训练分类模型的输入，根据所述已训练分类模型对所述待分类图像进行分类。
  25. 一个或多个存储有计算机程序的非易失性存储介质，其特征在于，所述计算机程序被一个或多个处理器执行时，使得一个或多个处理器执行如下步骤：
    提取待分类图像的全局图像特征；
    根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域；
    提取所述待分类图像的局部关键区域的图像特征；
    根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征；
    根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类确定分类结果。
  26. 根据权利要求25所述的存储介质，其特征在于，所述根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域的步骤包括：
    根据所述待分类图像的全局图像特征，获取所述待分类图像的各子区域分别对应的子图像特征；
    获取各所述子区域的子图像特征分别对应的权重；
    根据各所述子区域的子图像特征、各所述子区域的子图像特征的权重以及所述待分类图像，确定所述待分类图像的局部关键区域。
  27. 根据权利要求26所述的存储介质，其特征在于，所述获取各所述子区域的子图像特征分别对应的权重的步骤包括：
    获取预设高维特征向量；
    将各所述子区域的子图像特征分别与所述预设高维特征向量进行合并，获得各所述子区域的组合特征；
    将各所述子区域的组合特征分别作为多层感知器的输入，根据所述多层感知器分别进行预测，获得各所述子区域的子图像特征分别对应的权重。
  28. 根据权利要求25-27中任意一项所述的存储介质，其特征在于，所述根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的子图像特征，获取待分类图像的关联特征的步骤包括：
    对所述待分类图像的全局图像特征进行转置处理，获得转置特征；
    将所述转置特征与所述待分类图像的局部关键区域的子图像特征相乘，获得乘积特征；
    平均池化所述乘积特征，获得所述待分类图像的关联特征。
  29. 根据权利要求25所述的存储介质，其特征在于，所述提取待分类图像的全局图像特征的步骤包括：根据第一卷积神经网络对所述待分类图像进行特征提取，获得所述待分类图像的全局图像特征；
    所述提取所述局部关键区域的图像特征的步骤包括：根据第二卷积神经网络对所述待分类图像的局部关键区域进行特征提取，获得所述待分类图像的局部关键区域的图像特征。
  30. 根据权利要求25所述的存储介质，其特征在于，所述根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类的步骤包括：
    将所述待分类图像的全局图像特征转化为一维全局图像特征向量，将所述待分类图像的局部关键区域的图像特征转化为一维局部图像特征向量，将所述待分类图像的关联特征转化为一维关联特征向量；
    合并所述一维全局图像特征向量、所述一维局部图像特征向量以及所述一维关联特征向量，获得所述待分类图像对应的一维图像合并特征向量；其中，所述一维图像合并特征向量中的元素数量为所述一维全局图像特征向量中的元素数量、所述一维局部图像特征向量的元素数量和所述一维关联特征向量的元素数量之和；
    将所述一维图像合并特征向量作为已训练分类模型的输入，根据所述已训练分类模型对所述待分类图像进行分类。
  31. 一个或多个存储有计算机程序的非易失性存储介质，其特征在于，所述计算机程序被一个或多个处理器执行时，使得一个或多个处理器执行如下步骤：
    获取待分类图像对应的分类结果；其中，所述分类结果通过提取待分类图像的全局图像特征；根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域；提取所述待分类图像的局部关键区域的图像特征；根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的图像特征，获取待分类图像的关联特征；根据所述待分 类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类确定；
    查找与所述分类结果对应的待推荐信息；
    推送所述推荐信息。
  32. 根据权利要求31所述的存储机制，其特征在于，所述根据所述待分类图像以及所述待分类图像的全局图像特征，确定所述待分类图像的局部关键区域，包括：
    根据所述待分类图像的全局图像特征，获取所述待分类图像的各子区域分别对应的子图像特征；
    获取各所述子区域的子图像特征分别对应的权重；
    根据各所述子区域的子图像特征、各所述子区域的子图像特征的权重以及所述待分类图像，确定所述待分类图像的局部关键区域。
  33. 根据权利要求32所述的存储机制，其特征在于，所述获取各所述子区域的子图像特征分别对应的权重，包括：
    获取预设高维特征向量；
    将各所述子区域的子图像特征分别与所述预设高维特征向量进行合并，获得各所述子区域的组合特征；
    将各所述子区域的组合特征分别作为多层感知器的输入，根据所述多层感知器分别进行预测，获得各所述子区域的子图像特征分别对应的权重。
  34. 根据权利要求31-33中任意一项所述的存储机制，其特征在于，所述根据所述待分类图像的全局图像特征以及所述待分类图像的局部关键区域的子图像特征，获取待分类图像的关联特征，包括：
    对所述待分类图像的全局图像特征进行转置处理，获得转置特征；
    将所述转置特征与所述待分类图像的局部关键区域的子图像特征相乘，获得乘积特征；
    平均池化所述乘积特征，获得所述待分类图像的关联特征。
  35. 根据权利要求31所述的存储机制，其特征在于，
    所述提取待分类图像的全局图像特征包括：根据第一卷积神经网络对所述待分类图像进行特征提取，获得所述待分类图像的全局图像特征；
    所述提取所述局部关键区域的图像特征包括：根据第二卷积神经网络对所述待分类图像的局部关键区域进行特征提取，获得所述待分类图像的局部关键区域的图像特征。
  36. 根据权利要求31所述的存储机制，其特征在于，所述根据所述待分类图像的全局图像特征、所述待分类图像的局部关键区域的图像特征以及所述待分类图像的关联特征，对所述待分类图像进行分类，包括：
    将所述待分类图像的全局图像特征转化为一维全局图像特征向量，将所述待分类图像的局部关键区域的图像特征转化为一维局部图像特征向量，将所述待分类图像的关联特征转化为一维关联特征向量；
    合并所述一维全局图像特征向量、所述一维局部图像特征向量以及所述一维关联特征向量，获得所述待分类图像对应的一维图像合并特征向量；其中，所述一维图像合并特征向量中的元素数量为所述一维全局图像特征向量中的元素数量、所述一维局部图像特征向量的元素数量和所述一维关联特征向量的元素数量之和；
    将所述一维图像合并特征向量作为已训练分类模型的输入，根据所述已训练分类模型对所述待分类图像进行分类。
PCT/CN2018/106196 2017-11-30 2018-09-18 图像分类方法及个性化推荐方法、计算机设备及存储介质 WO2019105106A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/675,831 US11238315B2 (en) 2017-11-30 2019-11-06 Image classification method, personalized recommendation method, computer device and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711244572.4A CN108304847B (zh) 2017-11-30 2017-11-30 图像分类方法及装置、个性化推荐方法及装置
CN201711244572.4 2017-11-30

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/675,831 Continuation US11238315B2 (en) 2017-11-30 2019-11-06 Image classification method, personalized recommendation method, computer device and storage medium

Publications (1)

Publication Number Publication Date
WO2019105106A1 true WO2019105106A1 (zh) 2019-06-06

Family

ID=62870206

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/106196 WO2019105106A1 (zh) 2017-11-30 2018-09-18 图像分类方法及个性化推荐方法、计算机设备及存储介质

Country Status (3)

Country Link
US (1) US11238315B2 (zh)
CN (2) CN108304847B (zh)
WO (1) WO2019105106A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110751218A (zh) * 2019-10-22 2020-02-04 Oppo广东移动通信有限公司 图像分类方法、图像分类装置及终端设备
CN112257758A (zh) * 2020-09-27 2021-01-22 浙江大华技术股份有限公司 细粒度图像识别方法、卷积神经网络及其训练方法

Families Citing this family (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108304847B (zh) * 2017-11-30 2021-09-28 腾讯科技（深圳）有限公司 图像分类方法及装置、个性化推荐方法及装置
US11003892B2 (en) * 2018-11-09 2021-05-11 Sap Se Landmark-free face attribute prediction
CN111191665A (zh) * 2018-11-14 2020-05-22 北京奇虎科技有限公司 图像分类方法、装置及电子设备
CN111310518B (zh) * 2018-12-11 2023-12-08 北京嘀嘀无限科技发展有限公司 图片特征提取方法、目标重识别方法、装置及电子设备
US10963757B2 (en) * 2018-12-14 2021-03-30 Industrial Technology Research Institute Neural network model fusion method and electronic device using the same
CN109740646B (zh) * 2018-12-19 2021-01-05 创新奇智(北京)科技有限公司 一种图像差异比对方法及其系统、电子装置
US10769784B2 (en) * 2018-12-21 2020-09-08 Metal Industries Research & Development Centre Image analyzing method and electrical device
JP7261022B2 (ja) * 2019-01-30 2023-04-19 キヤノン株式会社 情報処理システム、端末装置及びその制御方法、プログラム、記憶媒体
CN111582008B (zh) * 2019-02-19 2023-09-08 富士通株式会社 训练分类模型的装置和方法及利用分类模型分类的装置
CN109886223B (zh) * 2019-02-26 2022-05-17 北京旷视科技有限公司 人脸识别方法、底库录入方法、装置及电子设备
CN109961041B (zh) * 2019-03-21 2021-03-23 腾讯科技（深圳）有限公司 一种视频识别方法、装置及存储介质
CN110222598B (zh) * 2019-05-21 2022-09-27 平安科技（深圳）有限公司 一种视频行为识别方法、装置、存储介质和服务器
CN110263280B (zh) * 2019-06-11 2021-03-23 浙江工业大学 一种基于多视图的动态链路预测深度模型及应用
CN110390056B (zh) * 2019-06-18 2022-07-15 平安科技（深圳）有限公司 大数据处理方法、装置、设备及可读存储介质
CN110298405A (zh) * 2019-07-03 2019-10-01 北京字节跳动网络技术有限公司 类别识别方法及装置、存储介质与终端
CN110598717B (zh) * 2019-09-12 2022-06-21 北京迈格威科技有限公司 图像特征的提取方法、装置及电子设备
CN111144244B (zh) * 2019-12-13 2023-07-11 唐美(北京)文化科技发展有限公司 家具木材类别识别方法及装置
CN111179031B (zh) * 2019-12-23 2023-09-26 第四范式（北京）技术有限公司 一种商品推荐模型的训练方法、装置及系统
CN111177449B (zh) * 2019-12-30 2021-11-05 深圳市商汤科技有限公司 基于图片的多维信息整合方法及相关设备
CN111160286B (zh) * 2019-12-31 2023-02-28 中国电子科技集团公司信息科学研究院 一种视频真伪鉴别方法
CN111222557A (zh) * 2019-12-31 2020-06-02 Oppo广东移动通信有限公司 图像分类方法、装置、存储介质及电子设备
CN114830186A (zh) * 2020-01-10 2022-07-29 深圳市欢太科技有限公司 图像分类方法、装置、存储介质及电子设备
CN111325271B (zh) * 2020-02-18 2023-09-12 Oppo广东移动通信有限公司 图像分类方法及装置
WO2021169126A1 (zh) * 2020-02-25 2021-09-02 平安科技（深圳）有限公司 病灶分类模型训练方法、装置、计算机设备和存储介质
CN111461228B (zh) * 2020-04-01 2024-03-15 腾讯科技（深圳）有限公司 图像推荐方法和装置及存储介质
CN111598131B (zh) * 2020-04-17 2023-08-25 北京百度网讯科技有限公司 图像处理方法、装置、电子设备及存储介质
CN111694954B (zh) * 2020-04-28 2023-12-08 北京旷视科技有限公司 图像分类方法、装置和电子设备
CN111950544A (zh) * 2020-06-30 2020-11-17 杭州依图医疗技术有限公司 一种确定病理图像中感兴趣区域的方法及装置
JP2022038390A (ja) * 2020-08-26 2022-03-10 株式会社東芝 推論装置、方法、プログラムおよび学習装置
WO2022041222A1 (en) * 2020-08-31 2022-03-03 Top Team Technology Development Limited Process and system for image classification
CN112101438B (zh) * 2020-09-08 2024-04-16 南方科技大学 一种左右眼分类方法、装置、服务器和存储介质
CN112085035A (zh) * 2020-09-14 2020-12-15 北京字节跳动网络技术有限公司 图像处理方法、装置、电子设备和计算机可读介质
CN112364933A (zh) * 2020-11-23 2021-02-12 北京达佳互联信息技术有限公司 图像分类方法、装置、电子设备和存储介质
US20220176998A1 (en) * 2020-12-08 2022-06-09 Guangzhou Automobile Group Co., Ltd. Method and Device for Loss Evaluation to Automated Driving
CN112580352B (zh) * 2021-03-01 2021-06-04 腾讯科技（深圳）有限公司 关键词提取方法、装置和设备及计算机存储介质
CN113033557A (zh) * 2021-04-16 2021-06-25 北京百度网讯科技有限公司 用于训练图像处理模型和检测图像的方法、装置
CN113191436A (zh) * 2021-05-07 2021-07-30 广州博士信息技术研究院有限公司 一种人才图像标签识别方法、系统及云平台
CN113392766A (zh) * 2021-06-16 2021-09-14 哈尔滨理工大学 一种基于注意力机制的人脸表情识别方法
CN113657523A (zh) * 2021-08-23 2021-11-16 科大讯飞股份有限公司 一种图像目标分类方法、装置、设备及存储介质
CN114140637B (zh) * 2021-10-21 2023-09-12 阿里巴巴达摩院(杭州)科技有限公司 图像分类方法、存储介质和电子设备
CN114418030B (zh) * 2022-01-27 2024-04-23 腾讯科技（深圳）有限公司 图像分类方法、图像分类模型的训练方法及装置
CN114565774B (zh) * 2022-02-21 2024-04-05 辽宁师范大学 基于局部几何与全局结构联合学习的3d图卷积分类方法
CN115409855B (zh) * 2022-09-20 2023-07-07 北京百度网讯科技有限公司 图像处理方法、装置、电子设备和存储介质
CN116503933B (zh) * 2023-05-24 2023-12-12 北京万里红科技有限公司 一种眼周特征提取方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105956631A (zh) * 2016-05-19 2016-09-21 南京大学 一种面向电子图库的在线渐进式图像分类方法
US20170018075A1 (en) * 2015-07-13 2017-01-19 Delineo Diagnostics, Inc Method and apparatus for adaptive Computer-Aided Diagnosis
CN106408037A (zh) * 2015-07-30 2017-02-15 阿里巴巴集团控股有限公司 图像识别方法及装置
CN106803090A (zh) * 2016-12-05 2017-06-06 中国银联股份有限公司 一种图像识别方法和装置
US20170294040A1 (en) * 2015-03-05 2017-10-12 Nant Holdings Ip, Llc Global signatures for large-scale image recognition
CN108304847A (zh) * 2017-11-30 2018-07-20 腾讯科技（深圳）有限公司 图像分类方法及装置、个性化推荐方法及装置

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101216889A (zh) * 2008-01-14 2008-07-09 浙江大学 一种融合全局特征与局部细节信息的人脸图像超分辨率方法
CN102521601B (zh) * 2011-11-17 2013-12-25 西安电子科技大学 基于半监督条件随机场的超光谱图像分类方法
CN102542067A (zh) * 2012-01-06 2012-07-04 上海交通大学 基于尺度学习和关联标号传播的自动图像语义标注方法
WO2014032162A1 (en) * 2012-08-28 2014-03-06 Solink Corporation Transaction verification system
EP2915101A4 (en) * 2012-11-02 2017-01-11 Itzhak Wilf Method and system for predicting personality traits, capabilities and suggested interactions from images of a person
CN103281477B (zh) * 2013-05-17 2016-05-11 天津大学 基于多级别特征数据关联的多目标视觉跟踪方法
US9400925B2 (en) * 2013-11-15 2016-07-26 Facebook, Inc. Pose-aligned networks for deep attribute modeling
CN103886329B (zh) * 2014-03-21 2017-03-08 西安电子科技大学 基于张量分解降维的极化图像分类方法
CN105184298B (zh) * 2015-08-27 2018-10-19 重庆大学 一种快速局部约束低秩编码的图像分类方法
CN105117478A (zh) * 2015-09-09 2015-12-02 北京建筑大学 Pda应用程序辅助拍摄设备影像自动分类入库方法
CN105608230B (zh) * 2016-02-03 2019-05-31 南京云创大数据科技股份有限公司 一种基于图像检索的商家信息推荐系统及方法
CN107203775B (zh) * 2016-03-18 2021-07-27 斑马智行网络(香港)有限公司 一种图像分类的方法、装置和设备
US10198671B1 (en) * 2016-11-10 2019-02-05 Snap Inc. Dense captioning with joint interference and visual context
CN107066520B (zh) * 2017-02-14 2021-05-18 苏州飞搜科技有限公司 一种基于多个语义层次的图片检索方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170294040A1 (en) * 2015-03-05 2017-10-12 Nant Holdings Ip, Llc Global signatures for large-scale image recognition
US20170018075A1 (en) * 2015-07-13 2017-01-19 Delineo Diagnostics, Inc Method and apparatus for adaptive Computer-Aided Diagnosis
CN106408037A (zh) * 2015-07-30 2017-02-15 阿里巴巴集团控股有限公司 图像识别方法及装置
CN105956631A (zh) * 2016-05-19 2016-09-21 南京大学 一种面向电子图库的在线渐进式图像分类方法
CN106803090A (zh) * 2016-12-05 2017-06-06 中国银联股份有限公司 一种图像识别方法和装置
CN108304847A (zh) * 2017-11-30 2018-07-20 腾讯科技（深圳）有限公司 图像分类方法及装置、个性化推荐方法及装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110751218A (zh) * 2019-10-22 2020-02-04 Oppo广东移动通信有限公司 图像分类方法、图像分类装置及终端设备
CN112257758A (zh) * 2020-09-27 2021-01-22 浙江大华技术股份有限公司 细粒度图像识别方法、卷积神经网络及其训练方法

Also Published As

Publication number Publication date
US11238315B2 (en) 2022-02-01
CN110580482A (zh) 2019-12-17
CN108304847B (zh) 2021-09-28
US20200074243A1 (en) 2020-03-05
CN110580482B (zh) 2022-04-08
CN108304847A (zh) 2018-07-20

Similar Documents

Publication Publication Date Title
WO2019105106A1 (zh) 图像分类方法及个性化推荐方法、计算机设备及存储介质
US11514260B2 (en) Information recommendation method, computer device, and storage medium
US10909459B2 (en) Content embedding using deep metric learning algorithms
US11030415B2 (en) Learning document embeddings with convolutional neural network architectures
Bampis et al. Fast loop-closure detection using visual-word-vectors from image sequences
Memon et al. GEO matching regions: multiple regions of interests using content based image retrieval based on relative locations
US20200073968A1 (en) Sketch-based image retrieval techniques using generative domain migration hashing
WO2020248581A1 (zh) 图数据识别方法、装置、计算机设备和存储介质
US9607014B2 (en) Image tagging
US9026518B2 (en) System and method for clustering content according to similarity
US11734337B2 (en) Identifying digital attributes from multiple attribute groups utilizing a deep cognitive attribution neural network
US20200074220A1 (en) Multi-view image clustering techniques using binary compression
WO2021031704A1 (zh) 对象追踪方法、装置、计算机设备和存储介质
GB2558050A (en) Generating a compact video feature representation in a digital medium environment
US20210103611A1 (en) Context-based organization of digital media
Huang et al. Extreme learning machine with multi-scale local receptive fields for texture classification
US10831818B2 (en) Digital image search training using aggregated digital images
Zhao et al. Multiscale object detection in high-resolution remote sensing images via rotation invariant deep features driven by channel attention
Yang et al. Adaptive graph nonnegative matrix factorization with the self-paced regularization
Xiong et al. An interpretable fusion siamese network for multi-modality remote sensing ship image retrieval
Wang et al. Glocal energy-based learning for few-shot open-set recognition
CN112733901B (zh) 基于联邦学习和区块链的结构化动作分类方法与装置
Mi et al. Principal component analysis based on block-norm minimization
Zhang et al. Classify social image by integrating multi-modal content
Chang et al. Learning local semantic distances with limited supervision

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18883115

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18883115

Country of ref document: EP

Kind code of ref document: A1