CN111325242A - Image classification method, terminal and computer storage medium - Google Patents

Image classification method, terminal and computer storage medium Download PDF

Info

Publication number
CN111325242A
CN111325242A CN202010078654.1A CN202010078654A CN111325242A CN 111325242 A CN111325242 A CN 111325242A CN 202010078654 A CN202010078654 A CN 202010078654A CN 111325242 A CN111325242 A CN 111325242A
Authority
CN
China
Prior art keywords
image
trained
feature vector
fine
group
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010078654.1A
Other languages
Chinese (zh)
Inventor
戴秋菊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN202010078654.1A priority Critical patent/CN111325242A/en
Publication of CN111325242A publication Critical patent/CN111325242A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

本申请实施例公开了一种图像的分类方法,该方法应用于一终端中,包括:获取待分类图像,采用预先训练好的细粒度分类模型,对待分类图像进行分类,得到分类后的图像。本申请实施例还同时提供了一种终端及计算机存储介质。

Figure 202010078654

The embodiment of the present application discloses an image classification method, which is applied to a terminal and includes: acquiring an image to be classified, using a pre-trained fine-grained classification model, classifying the to-be-classified image, and obtaining a classified image. The embodiments of the present application also provide a terminal and a computer storage medium at the same time.

Figure 202010078654

Description

一种图像的分类方法、终端及计算机存储介质Image classification method, terminal and computer storage medium

技术领域technical field

本申请涉及细粒度图像分类技术,尤其涉及一种图像的分类方法、终端及计算机存储介质。The present application relates to a fine-grained image classification technology, and in particular, to an image classification method, a terminal, and a computer storage medium.

背景技术Background technique

细粒度图像分类技术是图像分类的一个分支,由于其类别都属于同一个大的类别,比如不同品种的狗都属于狗这一个大类,所以其类别直接的差异性比较小,但是存在背景和外形的多样性等导致类别之间仍然有很多的差异性。Fine-grained image classification technology is a branch of image classification. Since its categories belong to the same large category, for example, dogs of different breeds belong to the same category of dogs, so the direct differences between their categories are relatively small, but there are background and There are still many differences between categories due to the diversity of shapes.

目前,图像细粒度分类方法大致可以分为以下几个分支:基于现有分类网络的微调、基于细粒度特征学习的方法、基于目标块的检测与分类结合的方法以及基于视觉注意力机制的方法,然而,上述方法主要集中通用分类算法的基础上进行改进的,目前分类算法最常用的损失函数为softmax,而softmax对于细粒度分类的类间区分度不高;由此可以看出,现有的采用细粒度图像分类算法导致图像分类的准确性较差。At present, image fine-grained classification methods can be roughly divided into the following branches: fine-tuning based on existing classification networks, methods based on fine-grained feature learning, methods based on the combination of target block detection and classification, and methods based on visual attention mechanism , however, the above methods are mainly improved on the basis of general classification algorithms. At present, the most commonly used loss function for classification algorithms is softmax, and softmax is not highly discriminative between classes for fine-grained classification; it can be seen that the existing The use of fine-grained image classification algorithms results in poor image classification accuracy.

发明内容SUMMARY OF THE INVENTION

本申请实施例提供一种图像的分类方法、终端及计算机存储介质,能够提高图像分类的准确性。Embodiments of the present application provide an image classification method, a terminal, and a computer storage medium, which can improve the accuracy of image classification.

本申请的技术方案是这样实现的:The technical solution of the present application is realized as follows:

本申请实施例提供了一种图像的分类方法,该方法应用于一终端中,包括:An embodiment of the present application provides a method for classifying images, and the method is applied to a terminal, including:

获取待分类图像;Get images to be classified;

采用预先训练好的细粒度分类模型,对所述待分类图像进行分类,得到分类后的图像;Use a pre-trained fine-grained classification model to classify the to-be-classified image to obtain a classified image;

其中,所述训练好的细粒度分类模型采用以下方式得到:Wherein, the trained fine-grained classification model is obtained in the following manner:

根据获取到的待训练图像集的图像标签,对所述待训练图像集的图像进行分组,得到分组后的待训练图像集;其中,所述图像标签用于表征图像的类别;According to the obtained image labels of the to-be-trained image set, the images of the to-be-trained image set are grouped to obtain a grouped to-be-trained image set; wherein, the image labels are used to characterize the category of the image;

对所述分组后的待训练图像集中图像提取特征向量,得到特征向量组;Extracting feature vectors from the grouped images in the to-be-trained image set to obtain a feature vector group;

采用所述特征向量组对细粒度分类模型进行训练,以确定出所述细粒度分类模型中损失函数与目标函数的取值最小时的模型参数,得到所述训练好的细粒度分类模型。The fine-grained classification model is trained by using the feature vector group to determine the model parameters in the fine-grained classification model when the value of the loss function and the objective function is the smallest, and the trained fine-grained classification model is obtained.

本申请实施例提供了一种终端,所述终端包括:An embodiment of the present application provides a terminal, where the terminal includes:

获取模块,用于获取待分类图像;The acquisition module is used to acquire the images to be classified;

分类模块,用于采用预先训练好的细粒度分类模型,对所述待分类图像进行分类,得到分类后的图像;A classification module, configured to use a pre-trained fine-grained classification model to classify the to-be-classified image to obtain a classified image;

其中,所述训练好的细粒度分类模型采用以下方式得到:Wherein, the trained fine-grained classification model is obtained in the following manner:

根据获取到的待训练图像集的图像标签,对所述待训练图像集的图像进行分组,得到分组后的待训练图像集;其中,所述图像标签用于表征图像的类别;According to the obtained image labels of the to-be-trained image set, the images of the to-be-trained image set are grouped to obtain a grouped to-be-trained image set; wherein, the image labels are used to characterize the category of the image;

对所述分组后的待训练图像集中图像提取特征向量,得到特征向量组;Extracting feature vectors from the grouped images in the to-be-trained image set to obtain a feature vector group;

采用所述特征向量组对细粒度分类模型进行训练,以确定出所述细粒度分类模型中损失函数与目标函数的取值最小时的模型参数,得到所述训练好的细粒度分类模型。The fine-grained classification model is trained by using the feature vector group to determine the model parameters in the fine-grained classification model when the value of the loss function and the objective function is the smallest, and the trained fine-grained classification model is obtained.

本申请实施例还提供了一种终端,所述终端包括:处理器以及存储有所述处理器可执行指令的存储介质,所述存储介质通过通信总线依赖所述处理器执行操作,当所述指令被所述处理器执行时,执行上述一个或多个实施例所述图像的分类方法。An embodiment of the present application further provides a terminal, the terminal includes: a processor and a storage medium storing executable instructions of the processor, the storage medium depends on the processor to perform operations through a communication bus, and when the When the instructions are executed by the processor, the image classification method according to one or more of the above embodiments is executed.

本申请实施例提供了一种计算机存储介质,存储有可执行指令,当所述可执行指令被一个或多个处理器执行的时候,所述处理器执行上述一个或多个实施例所述图像的分类方法。Embodiments of the present application provide a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the images described in the above one or more embodiments. classification method.

本申请实施例提供了一种图像的分类方法、终端及计算机存储介质,该方法应用于一终端中,包括:获取待分类图像,采用预先训练好的细粒度分类模型,对待分类图像进行分类,得到分类后的图像,其中,训练好的细粒度分类模型采用以下方式得到:根据获取到的待训练图像集的图像标签,对待训练图像集的图像进行分组,得到分组后的待训练图像集;其中,图像标签用于表征图像的类别,对分组后的待训练图像集中图像提取特征向量,得到特征向量组,采用特征向量组对细粒度分类模型进行训练,以确定出细粒度分类模型中损失函数与目标函数的取值最小时的模型参数,得到训练好的细粒度分类模型;也就是说,在本申请实施例中,通过采用预先训练好的细粒度分类模型对待分类图像进行分类,其中,训练好的细粒度分类模型是根据待训练图像集的图像标签对待训练图像集的图像进行分组并提取特征向量,采用特征向量组对细粒度分类模型进行训练,通过在模型中设置损失函数与目标函数,并在其取值最小时得到模型参数,从而得出训练好的细粒度分类模型,这样,通过在模型中设置损失函数与目标函数,使得得到的训练好的细粒度分类模型更加优化,在此基础上在进行图像分类时,提高了图像分类的准确性,进而提高了用户的体验。The embodiments of the present application provide an image classification method, a terminal, and a computer storage medium. The method is applied to a terminal and includes: acquiring an image to be classified, using a pre-trained fine-grained classification model, and classifying the image to be classified, Obtaining classified images, wherein the trained fine-grained classification model is obtained in the following manner: according to the obtained image labels of the image set to be trained, the images of the image set to be trained are grouped, and the grouped image set to be trained is obtained; Among them, the image label is used to represent the category of the image, and the feature vector is extracted from the grouped images to be trained to obtain a feature vector group, and the feature vector group is used to train the fine-grained classification model to determine the loss in the fine-grained classification model. The model parameter when the value of the function and the objective function is the smallest, a trained fine-grained classification model is obtained; that is, in the embodiment of the present application, the images to be classified are classified by adopting the pre-trained fine-grained classification model, wherein , the trained fine-grained classification model is to group the images of the to-be-trained image set according to the image labels of the to-be-trained image set and extract the feature vector, and use the feature vector group to train the fine-grained classification model. The objective function is obtained, and the model parameters are obtained when its value is the smallest, so as to obtain the trained fine-grained classification model. In this way, by setting the loss function and the objective function in the model, the obtained trained fine-grained classification model is more optimized , and on this basis, when performing image classification, the accuracy of image classification is improved, thereby improving user experience.

附图说明Description of drawings

图1为本申请实施例提供的一种可选的图像的分类方法的流程示意图;1 is a schematic flowchart of an optional image classification method provided by an embodiment of the present application;

图2为本申请实施例提供的一种可选的图像的分类方法的实例的流程示意图;2 is a schematic flowchart of an example of an optional image classification method provided by an embodiment of the present application;

图3为本申请实施例提供的一种终端的结构示意图一;FIG. 3 is a schematic structural diagram 1 of a terminal according to an embodiment of the present application;

图4为本申请实施例提供的一种终端的结构示意图二。FIG. 4 is a second schematic structural diagram of a terminal according to an embodiment of the present application.

具体实施方式Detailed ways

下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述。The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

实施例一Example 1

本申请实施例提供了一种图像的分类方法,该方法应用于一终端中,图1为本申请实施例提供的一种可选的图像的分类方法的流程示意图,参考图1所示,上述图像的分类方法可以包括:An embodiment of the present application provides an image classification method, which is applied to a terminal. FIG. 1 is a schematic flowchart of an optional image classification method provided by an embodiment of the present application. Referring to FIG. 1 , the above Image classification methods can include:

S101:获取待分类图像;S101: Obtain an image to be classified;

目前,图像的细粒度分类模型大致可以分为以下几个分支:基于现有分类网络的微调、基于细粒度特征学习的方法,基于目标块的检测与分类结合的方法以及基于视觉注意力机制的方法,其中,基于现有分类网络微调的方法通常使用现有的分类网络(例如:MobileNet,Xception等)在ImageNet上面进行初步训练得到一个训练好的分类模型,然后继续在细粒度的数据集上进行微调,使得模型能够更适用于区分子类别;基于细粒度特征学习的方法需要两个网络获取的信息结合,一个网络用来获取目标的位置信息,一个网络用于提取目标的抽象特征表达;基于目标块的检测与分类结合的细粒度分类方法借鉴了目标检测的思想,先通过目标检测模块将图像的目标区域框出来,然后基于目标区域进行细粒度分类,分类算法可以是传统的支持向量机(SVM,Support Vector Machines)分类器或者通用的分类网络;基于注意力机制的细粒度分类算法相比于通用的分类算法添加了注意力机制,使得模型更加关注目标位置的信息表达。At present, the fine-grained classification models of images can be roughly divided into the following branches: fine-tuning based on existing classification networks, methods based on fine-grained feature learning, methods based on the combination of target block detection and classification, and methods based on visual attention mechanism. method, in which the method based on the fine-tuning of the existing classification network usually uses the existing classification network (for example: MobileNet, Xception, etc.) to perform preliminary training on ImageNet to obtain a trained classification model, and then continue on the fine-grained dataset. Fine-tune the model to make the model more suitable for distinguishing sub-categories; the method based on fine-grained feature learning requires the combination of information obtained by two networks, one network is used to obtain the location information of the target, and the other is used to extract the abstract feature expression of the target; The fine-grained classification method based on the combination of detection and classification of target blocks draws on the idea of target detection. First, the target area of the image is framed by the target detection module, and then the fine-grained classification is performed based on the target area. The classification algorithm can be a traditional support vector. Compared with the general classification algorithm, the fine-grained classification algorithm based on the attention mechanism adds an attention mechanism, which makes the model pay more attention to the information expression of the target position.

其中,上述方法主要集中通用分类算法的基础上进行改进的,目前分类算法最常用的损失函数为softmax,而softmax对于细粒度分类的类间区分度不高,存在以下缺点:第一,类间特征中心之间的距离较近,容易造成类间误分问题;第二,类内特征不够聚拢,导致多个类别之间的特征分布存在交叠,同样会造成类间误分;第三,添加检测模块的算法会引入复杂的运算,增加计算成本造成更多的时延;从而导致图像分类的准确度较低。Among them, the above methods are mainly improved on the basis of general classification algorithms. At present, the most commonly used loss function of classification algorithms is softmax, and softmax is not highly discriminative between classes for fine-grained classification, and has the following shortcomings: First, between classes The distance between the feature centers is short, which is easy to cause the problem of misclassification between classes; second, the intra-class features are not clustered enough, resulting in overlapping feature distributions between multiple classes, which will also cause misclassification between classes; third, The algorithm of adding the detection module will introduce complex operations, increase the computational cost and cause more time delay, resulting in lower accuracy of image classification.

为了提高图像分类的准确度,这里,终端先获取待分类图像,其中,待分类图像中可以包含待分类的对象,例如:狗,车,树之类的对象,针对狗,车,树的类别再进行细分,以分类出待分类图像中的狗,车,树。In order to improve the accuracy of image classification, here, the terminal first obtains the image to be classified, wherein, the image to be classified may contain objects to be classified, such as: dog, car, tree and other objects, for the category of dog, car, tree Subdivide again to classify the dog, car, tree in the image to be classified.

S102:采用预先训练好的细粒度分类模型,对待分类图像进行分类,得到分类后的图像;S102: Using a pre-trained fine-grained classification model, classify the images to be classified, and obtain the classified images;

为了实现对待分类图像进行分类,这里,采用预先训练好的细粒度分类模型对待分类图像进行分类,其中,训练好的细粒度分类模型采用以下方式得到:In order to classify the images to be classified, here, a pre-trained fine-grained classification model is used to classify the images to be classified, wherein the trained fine-grained classification model is obtained in the following manner:

根据获取到的待训练图像集的图像标签,对待训练图像集的图像进行分组,得到分组后的待训练图像集;According to the obtained image labels of the image set to be trained, the images of the image set to be trained are grouped to obtain the grouped image set to be trained;

对分组后的待训练图像集中图像提取特征向量,得到特征向量组;Extract feature vectors from the grouped images to be trained to obtain feature vector groups;

采用特征向量组对细粒度分类模型进行训练,以确定出细粒度分类模型中损失函数与目标函数的取值最小时的模型参数,得到训练好的细粒度分类模型。The feature vector group is used to train the fine-grained classification model to determine the model parameters when the value of the loss function and the objective function in the fine-grained classification model is the smallest, and the trained fine-grained classification model is obtained.

具体来说,终端先获取待训练图像集以及待训练图像集中图像的图像标签,其中,图像标签用于表征图像的类别;例如,针对车的类别进行分类的图像而言,图像标签可以按照品牌对车进行分类,待训练图像集中图像标签可以包括比亚迪,奥迪,宝马等等。Specifically, the terminal first obtains the image set to be trained and the image labels of the images in the to-be-trained image set, where the image labels are used to represent the category of the images; for example, for images classified according to the category of cars, the image labels can be classified according to the brand To classify the car, the image labels in the image set to be trained can include BYD, Audi, BMW, etc.

在获取到待训练图像集的图像标签之后,根据待训练图像集的图像标签对待训练图像集进行分组,可以得到分组后的待训练图像集,针对每组待训练图像集,对每组待训练图像集中的图像,进行特征向量的提取,然后得到每组待训练图像集中图像的特征向量,即用每组特征向量构成特征向量组。After the image labels of the to-be-trained image sets are obtained, the to-be-trained image sets are grouped according to the image labels of the to-be-trained image sets, and the grouped to-be-trained image sets can be obtained. Extract the feature vectors for the images in the image set, and then obtain the feature vectors of each group of images in the image set to be trained, that is, use each set of feature vectors to form a feature vector group.

在得到特征向量组之后,采用特征向量组对细粒度分类模型进行训练,这里,需要说明的是,该细粒度分类模型中不仅包括损失函数,还包括目标函数,在训练模型的过程中,加入优化的目标函数,可以使得类内特征更加聚拢,类间特征更加疏远,这样,使得训练好的细粒度分类模型在对类别的分类上更加准确。After the feature vector group is obtained, the feature vector group is used to train the fine-grained classification model. Here, it should be noted that the fine-grained classification model includes not only the loss function, but also the objective function. In the process of training the model, adding The optimized objective function can make the intra-class features more concentrated and the inter-class features more distant, so that the trained fine-grained classification model can be more accurate in classifying categories.

具体地,在对细粒度分类模型进行训练,主要是通过迭代的方式确定出损失函数与目标函数取值最小时的模型参数,在确定出模型参数之后,就可以得到训练好的细粒度分类模型。Specifically, when the fine-grained classification model is trained, the model parameters with the smallest loss function and the objective function are determined through iteration. After the model parameters are determined, the trained fine-grained classification model can be obtained. .

为了实现对待训练模型的分组并确定出特征向量组,在一种可选的实施例中,根据获取到的待训练图像集的图像标签,对待训练图像集的图像进行分组,得到分组后的待训练图像集,包括:In order to realize the grouping of the models to be trained and determine the feature vector group, in an optional embodiment, the images of the image set to be trained are grouped according to the obtained image labels of the image set to be trained, and the grouped images to be trained are obtained. Training image set, including:

将待训练图像集中的图像依次确定为第一图像;Determining the images in the image set to be trained as the first image in sequence;

针对第一图像,从除了第一图像以外的待训练图像集中,选取出第二图像和第三图像;其中,第二图像的图像标签与第一图像的图像标签相同,第三图像的图像标签与第一图像的图像标签不同;For the first image, select the second image and the third image from the set of images to be trained except the first image; wherein, the image label of the second image is the same as the image label of the first image, and the image label of the third image is the same as the image label of the first image. different from the image tag of the first image;

利用第一图像,第二图像和第三图像构成一组,以得到分组后的待训练图像集;Using the first image, the second image and the third image to form a group to obtain a grouped image set to be trained;

相应地,对分组后的待训练图像集中图像提取特征向量,得到特征向量组,包括:Correspondingly, a feature vector is extracted from the images in the grouped images to be trained to obtain a feature vector group, including:

对第一图像,第二图像和第三图像,分别采用细粒度分类模型提取出特征向量,得到第一图像的特征向量,第二图像的特征向量和第三图像的特征向量;For the first image, the second image and the third image, a fine-grained classification model is used to extract the feature vector, respectively, to obtain the feature vector of the first image, the feature vector of the second image and the feature vector of the third image;

利用第一图像的特征向量,第二图像的特征向量和第三图像的特征向量,形成特征向量组。Using the feature vector of the first image, the feature vector of the second image and the feature vector of the third image, a feature vector group is formed.

从待训练图像集中先确定出第一图像,可以依次将待训练图像集中的每个图像依次确定为第一图像,然后为第一图像,从除了第一图像以外的待训练图像集中选取出第二图像和第三图像,选取的规则为:The first image is first determined from the set of images to be trained, each image in the set of images to be trained can be sequentially determined as the first image, and then the first image, and the first image is selected from the set of images to be trained except the first image. For the second image and the third image, the selection rules are:

第一图像的图像标签与第二图像的图像标签相同,第一图像的图像标签与第三图像的图像标签不同,也就是说,首先在待训练图像集中遍历图像,将每个图像确定为第一图像,然后按照上述规则选取第二图像和第三图像,从而利用第一图像,第二图像和第三图像形成一组,可以得到若干组待训练图像集,即为分组后的待训练图像集。The image label of the first image is the same as the image label of the second image, and the image label of the first image is different from that of the third image. That is to say, first traverse the images in the image set to be trained, and determine each image as the first image. One image, then select the second image and the third image according to the above rules, so that the first image, the second image and the third image are used to form a group, and several groups of images to be trained can be obtained, that is, the grouped images to be trained set.

在得到分组后的待训练图像集之后,每组待训练图像集中均包括三个图像,分别为第一图像,第二图像和第三图像,这里,对每组待训练图像集中的三个图像分别提取出特征向量,得到第一图像的特征向量,第二图像的特征向量和第三图像的特征向量,并利用第一图像的特征向量,第二图像的特征向量和第三图像的特征向量组成一组特征向量,按照上述方式可以获得若干组特征向量,从而得到特征向量组。After the grouped image sets to be trained are obtained, each set of images to be trained includes three images, namely the first image, the second image and the third image. Here, the three images in each set of images to be trained are Extract the feature vectors respectively to obtain the feature vector of the first image, the feature vector of the second image and the feature vector of the third image, and use the feature vector of the first image, the feature vector of the second image and the feature vector of the third image A set of eigenvectors is formed, and several sets of eigenvectors can be obtained according to the above method, thereby obtaining a set of eigenvectors.

为了使得训练得到的细粒度分类模型的分类准确性更高,需要对特征向量组进行筛选,在一种可选的实施例中,在对分组后的待训练图像集中图像提取特征向量,得到特征向量组之后,在采用特征向量组对细粒度分类模型进行训练,以确定出细粒度分类模型中损失函数与目标函数的取值最小时的模型参数,得到训练好的细粒度分类模型之前,该方法还包括:In order to make the classification accuracy of the fine-grained classification model obtained by training higher, the feature vector group needs to be screened. In an optional embodiment, feature vectors are extracted from the grouped images to be trained to obtain the features After the vector group, the feature vector group is used to train the fine-grained classification model to determine the model parameters when the value of the loss function and the objective function in the fine-grained classification model is the smallest, and before the trained fine-grained classification model is obtained. Methods also include:

选取出特征向量组中不满足预设条件的组别;Select the groups that do not meet the preset conditions in the eigenvector group;

从特征向量组中删除掉不满足预设条件的组别,以更新特征向量组。The groups that do not meet the preset conditions are deleted from the feature vector group to update the feature vector group.

具体来说,从特征向量组中选取出不满足预设条件的组别,将将不满足预设条件的组别删除掉,从而更新特征向量组,其中,不满足预设条件的组别可以有多种形式,总的来说,需要将第一图像与第二图像的类内特征较大的组别删除掉,和/或,将第一图像与第三图像的类间特征较小的组别删除掉,这里,本申请实施例对此不作具体限定。Specifically, the groups that do not meet the preset conditions are selected from the feature vector group, and the groups that do not meet the preset conditions are deleted, so as to update the feature vector group, wherein the groups that do not meet the preset conditions can be There are various forms. Generally speaking, it is necessary to delete the group with larger intra-class features of the first image and the second image, and/or delete the first image and the third image with smaller inter-class features. The group is deleted, and this is not specifically limited in this embodiment of the present application.

为了选取出特征向量组中不满足预设条件的组别,在一种可选的实施例中,选取出特征向量组中不满足预设条件的组别,包括:In order to select the groups in the feature vector group that do not meet the preset conditions, in an optional embodiment, select the groups in the feature vector group that do not meet the preset conditions, including:

计算第一图像的特征向量与第二图像的特征向量之间的第一距离值;calculating a first distance value between the feature vector of the first image and the feature vector of the second image;

当第一距离值大于等于第一预设阈值时,将包含第一图像和第二图像的组别确定为不满足预设条件的组别,并选取出不满足预设条件的组别;When the first distance value is greater than or equal to the first preset threshold, determine the group including the first image and the second image as a group that does not meet the preset condition, and select a group that does not meet the preset condition;

和/或,and / or,

计算第一图像的特征向量与第三图像的特征向量之间的第二距离值;calculating a second distance value between the feature vector of the first image and the feature vector of the third image;

当第二距离值小于等于第二预设阈值时,将包含第一图像和第三图像的组别确定为不满足预设条件的组别,并选取出不满足预设条件的组别。When the second distance value is less than or equal to the second preset threshold, the group including the first image and the third image is determined as a group that does not meet the preset condition, and the group that does not meet the preset condition is selected.

具体来说,可以先计算出第一图像的特征向量与第二图像的特征向量之间的距离,记为第一距离值,在终端中预先设置第一预设阈值,比较第一距离值与第一预设阈值之间的大小,当第一距离值大于等于第一预设阈值时,说明,第一图像与第二图像的类内特征相差较大,而这里,需要的是第一图像与第二图像的类内特征相差较小的组别,所以,将第一距离值对应的第一图像和第二图像的特征向量组确定为不满足预设条件的组别,并将该组别从特征向量组中删除掉。Specifically, the distance between the feature vector of the first image and the feature vector of the second image can be calculated first, which is recorded as the first distance value, the first preset threshold is preset in the terminal, and the first distance value is compared with The size between the first preset thresholds, when the first distance value is greater than or equal to the first preset threshold, it means that the intra-class features of the first image and the second image are quite different, and here, the first image is needed The group that is less different from the intra-class feature of the second image, therefore, the feature vector group of the first image and the second image corresponding to the first distance value is determined as the group that does not meet the preset conditions, and the group Don't delete from the eigenvector group.

还可以先计算出第一图像的特征向量与第三图像的特征向量之间的距离,记为第二距离值,在终端中预先设置第二预设阈值,比较第二距离值与第二预设阈值之间的大小,当第二距离值小于等于第二预设阈值时,说明,第一图像与第三图像的类间特征相差较小,而这里,需要的是第一图像与第三图像的类间特征相差较大的组别,所以,将第二距离值对应的第一图像和第三图像的特征向量组确定为不满足预设条件的组别,并将该组别从特征向量组中删除掉。It is also possible to first calculate the distance between the feature vector of the first image and the feature vector of the third image, denoting it as the second distance value, preset the second preset threshold value in the terminal, and compare the second distance value with the second preset value. Set the size between the thresholds. When the second distance value is less than or equal to the second preset threshold, it means that the difference between the features of the first image and the third image is small, and here, what is needed is the first image and the third image. A group with a large difference in the inter-class features of the images, therefore, the feature vector group of the first image and the third image corresponding to the second distance value is determined as the group that does not meet the preset conditions, and the group is determined from the feature vector group. removed from the vector group.

另外,还可以计算出第一图像的特征向量与第二图像的特征向量之间的距离,记为第一距离值,计算出第一图像的特征向量与第三图像的特征向量之间的距离,记为第二距离值,在终端中设置第一预设阈值和第二预设阈值,比较第一距离值与第一预设阈值之间的关系,比较第二距离值与第二预设阈值之间的关系,当第一距离值大于等于第一预设阈值,且第二距离值小于等于第二预设阈值,说明,第一图像与第二图像的类内特征相差较大,第一图像与第三图像的类间特征相差较小,所以,将第一距离值和第二距离值所对应的组别确定为不满足预设条件的组别,并将该组别从特征向量组中删除掉。In addition, the distance between the eigenvector of the first image and the eigenvector of the second image can also be calculated, denoted as the first distance value, and the distance between the eigenvector of the first image and the eigenvector of the third image can be calculated , denoted as the second distance value, set the first preset threshold and the second preset threshold in the terminal, compare the relationship between the first distance value and the first preset threshold, and compare the second distance value with the second preset threshold The relationship between the thresholds, when the first distance value is greater than or equal to the first preset threshold, and the second distance value is less than or equal to the second preset threshold, it means that the intra-class features of the first image and the second image differ greatly, and the The difference between the features of the first image and the third image is small, so the group corresponding to the first distance value and the second distance value is determined as the group that does not meet the preset condition, and the group is converted from the feature vector removed from the group.

进一步地,为了删除掉不满足预设条件的组别,在一种可选的实施例中,选取出特征向量组中不满足预设条件的组别,包括:Further, in order to delete the groups that do not meet the preset conditions, in an optional embodiment, select the groups that do not meet the preset conditions in the feature vector group, including:

计算第一图像的特征向量与第二图像的特征向量之间的第一距离值,第一图像的特征向量与第三图像的特征向量之间的第二距离值;Calculate the first distance value between the feature vector of the first image and the feature vector of the second image, and the second distance value between the feature vector of the first image and the feature vector of the third image;

当第二距离值与第一距离值的差值大于第三预设阈值时,将包含第一图像,第二图像和第三图像的组别确定为不满足预设条件的组别,并选取出不满足预设条件的组别。When the difference between the second distance value and the first distance value is greater than the third preset threshold, the group including the first image, the second image and the third image is determined as a group that does not meet the preset conditions, and the selected group that does not meet the preset conditions.

首先,计算出第一图像的向量特征与第二图像的特征向量的距离值,即为第一距离值,并计算出第一图像的特征向量与第三图像的特征向量之间的距离值,即为第二距离值,在计算出第一距离值与第二距离值之间的差值,在终端中预先设置有第三预设阈值,比较差值与第三预设阈值之间的关系,当差值大于第三预设阈值时,说明该组别中第一图像与第三图像的类间特征间距远远大于第一图像与第二图像的类内特征间距,这样,保证了类间特征的间距为更大的值,使得训练好的细粒度分类模型的分类准确性更高。First, the distance value between the vector feature of the first image and the feature vector of the second image is calculated, which is the first distance value, and the distance value between the feature vector of the first image and the feature vector of the third image is calculated, That is, the second distance value. After calculating the difference between the first distance value and the second distance value, a third preset threshold is preset in the terminal, and the relationship between the difference and the third preset threshold is compared. , when the difference is greater than the third preset threshold, it means that the inter-class feature distance between the first image and the third image in the group is much larger than the intra-class feature distance between the first image and the second image, thus ensuring the class The distance between features is a larger value, which makes the classification accuracy of the trained fine-grained classification model higher.

所以,将该组别确定为不满足预设条件的组别,并选取出删除掉,以更新特征向量组,这样,更新后的特征向量组的类内特征更加聚拢,类间特征更加疏远,进而提高了训练好的细粒度分类模型的准确性。Therefore, the group is determined as a group that does not meet the preset conditions, and is selected and deleted to update the feature vector group. In this way, the intra-class features of the updated feature vector group are more concentrated, and the inter-class features are more distant. In turn, the accuracy of the trained fine-grained classification model is improved.

为了训练出更加优化的细粒度分类模型,在一种可选的实施例中,采用特征向量组对细粒度分类模型进行训练,以确定出细粒度分类模型中损失函数与目标函数的取值最小时的模型参数,得到训练好的细粒度分类模型,包括:In order to train a more optimized fine-grained classification model, in an optional embodiment, a feature vector group is used to train the fine-grained classification model, so as to determine the best value of the loss function and the objective function in the fine-grained classification model. hours of model parameters to obtain a trained fine-grained classification model, including:

采用特征向量组对细粒度分类模型进行训练,以确定出细粒度分类模型中损失函数取值最小且目标函数的取值最小时的模型参数,得到训练好的细粒度分类模型;Use the feature vector group to train the fine-grained classification model to determine the model parameters with the smallest value of the loss function and the smallest value of the objective function in the fine-grained classification model, and obtain the trained fine-grained classification model;

或者,or,

采用特征向量组对细粒度分类模型进行训练,以确定出细粒度分类模型中损失函数的取值与目标函数的取值之和最小时的模型参数,得到训练好的细粒度分类模型。The feature vector group is used to train the fine-grained classification model to determine the model parameters when the sum of the value of the loss function and the value of the objective function in the fine-grained classification model is the smallest, and the trained fine-grained classification model is obtained.

这里,可以采用特征向量组对细粒度分类模型进行训练,确定出模型中的损失函数取值最小时且新增加的目标函数取值最小时的模型参数,这样得到模型参数之后就完成了训练,得到了训练好的细粒度分类模型。Here, the feature vector group can be used to train the fine-grained classification model, and the model parameters when the value of the loss function in the model is the smallest and the value of the newly added objective function is the smallest can be determined. After obtaining the model parameters, the training is completed. A trained fine-grained classification model is obtained.

还可以在采用特征向量组对细粒度分类模型进行训练,为模型的损失函数设置权重值,为模型中的目标函数设置权重值,在训练中确定出算是函数的取值与目标函数的取值进行加权求和后,加权求和值最小时的模型参数,从而得到训练好的细粒度分类模型;通常,可以为损失函数和目标函数选择相同的权重值,所以只需要确定出模型中损失函数的取值与目标函数的取值之和最小时的模型参数,从而得到训练好的细粒度分类模型。It is also possible to train the fine-grained classification model by using the feature vector group, set the weight value for the loss function of the model, set the weight value for the objective function in the model, and determine the value of the function and the value of the objective function during training. After the weighted summation, the model parameters with the smallest weighted summation value are obtained to obtain a trained fine-grained classification model; usually, the same weight value can be selected for the loss function and the objective function, so only the loss function in the model needs to be determined. The model parameters when the sum of the value of , and the value of the objective function is the smallest, so as to obtain a trained fine-grained classification model.

为了优化细粒度分类模型,在模型中加入了目标函数。在一种可选的实施例中,目标函数为与第一图像的特征向量,第二图像的特征向量,第三图像的特征向量和第三预设阈值相关的函数。In order to optimize the fine-grained classification model, an objective function is added to the model. In an optional embodiment, the objective function is a function related to the feature vector of the first image, the feature vector of the second image, the feature vector of the third image and the third preset threshold.

这里,通过目标函数,可以使得在采用训练好的细粒度分类模型进行分类时,类内特征更加聚拢,类间特征更加疏远,从而在分类的过程中能够更贱准确地分类出待识别图像中待分类对象的类别。Here, through the objective function, when the fine-grained classification model is used for classification, the intra-class features are more gathered, and the inter-class features are more distant, so that the images to be recognized can be more accurately classified in the process of classification. The category of the object to be classified.

下面举实例来对上述一个或多个实施例中所述的图像的分类方法进行说明。Examples are given below to illustrate the image classification method described in one or more of the above embodiments.

在实际应用中,通常细粒度分类模型中的损失函数都是基于softmax的方法,softmax的计算公式如下所示:In practical applications, the loss function in the fine-grained classification model is usually based on the softmax method. The calculation formula of softmax is as follows:

Figure BDA0002379438970000101
Figure BDA0002379438970000101

xi表示的是第i个样本基于分类网络提取的抽象特征向量,yi表示的是第i个样本的标签,W表示的是用于全卷积网络提取特征的分类层权重,Wj表示第j列(即j类)权重,同理bj表示的是该列的偏执项,T是矩阵转置操作,N是类别数量。x i represents the abstract feature vector extracted by the ith sample based on the classification network, yi represents the label of the ith sample, W represents the weight of the classification layer used to extract features from the fully convolutional network, and W j represents The weight of the jth column (ie, class j), similarly b j represents the paranoid term of the column, T is the matrix transpose operation, and N is the number of classes.

上述公式(1)中没有包含对属于同一类别的特征的距离度量优化,在softmax的基础上,根据构建三元组,对相同类别的特征进行约束,使其距离更近,对不同类别的特征也进行限制,使其距离更远。最复杂的模块为构建三元组,图2为本申请实施例提供的一种可选的图像的分类方法的实例的流程示意图,如图2所示,构建三元组的过程如下所示:The above formula (1) does not include the optimization of the distance metric for the features belonging to the same category. On the basis of softmax, according to the construction of triples, the features of the same category are constrained to make the distance closer, and the features of different categories are constrained. Limits are also made to make them farther apart. The most complex module is to construct triples, and FIG. 2 is a schematic flowchart of an example of an optional image classification method provided by the embodiment of the present application. As shown in FIG. 2 , the process of constructing triples is as follows:

Anchor为当前输入图像Va,Positive为与Anchor输入图像标签一致的正样本Vp,Negative为与当前输入图像标签不同的负样本Vn,构建三元组的过程可以是离线的也可以是在线的,本实例采用在线的方式便于更新,根据每个分支(N)中的输入数据,采用卷积神经网络(CNN,Convolutional Neural Networks)进行处理,得到每个图像的特征向量,随机组合为三元组,可以组合N*N*N组三元组{Va,Vp,Vn}i。但是并不是所有的三元组都是合理的,过多无用的三元组会造成算法无法收敛或者收敛很慢,因此,仍然需要对三元组进行筛选,N*N*N组三元组中,只Va与Vp的图像标签一致,且Va与Vn的图像标签相反时才是满足图2描述的有效三元组。Anchor is the current input image Va, Positive is the positive sample Vp that is consistent with the Anchor input image label, Negative is the negative sample Vn that is different from the current input image label, and the process of constructing triples can be offline or online. The example adopts an online method to facilitate updating. According to the input data in each branch (N), it is processed by convolutional neural network (CNN, Convolutional Neural Networks) to obtain the feature vector of each image, which is randomly combined into triples, N*N*N sets of triples {Va, Vp, Vn} i can be combined. However, not all triples are reasonable. Too many useless triples will cause the algorithm to fail to converge or to converge very slowly. Therefore, it is still necessary to filter the triples. N*N*N triples , only when the image labels of Va and Vp are consistent, and the image labels of Va and Vn are opposite, are valid triples that satisfy the description in FIG. 2 .

由于算法的目的是使得类内特征更加聚拢,类间特征更加疏远,算法优化的目的是使得Va与Vp的距离小于Va与Vn的距离,优化目标函数描述如下:Since the purpose of the algorithm is to make the intra-class features more concentrated and the inter-class features more distant, the purpose of the algorithm optimization is to make the distance between Va and Vp smaller than the distance between Va and Vn, and the optimization objective function is described as follows:

Figure BDA0002379438970000102
Figure BDA0002379438970000102

上述公式(2)中a为margin,目的是使得类间距更大的常量,根据实验经验设置初步可以尝试0.5。该目标函数与损失结合可以提升细粒度分类的精度。In the above formula (2), a is margin, the purpose is to make the class spacing larger. According to the experimental experience, you can try 0.5 initially. The objective function combined with the loss can improve the accuracy of fine-grained classification.

通过上述实例可以提升分类算法的类间区分性与类内聚拢性,进而可以提升细粒度分类效果,上述图像的分类方法可以应用于细粒度分类识别。The above examples can improve the inter-class discrimination and intra-class aggregation of the classification algorithm, and further improve the fine-grained classification effect. The above-mentioned image classification method can be applied to fine-grained classification and recognition.

本申请实施例提供了一种图像的分类方法,该方法应用于一终端中,包括:获取待分类图像,采用预先训练好的细粒度分类模型,对待分类图像进行分类,得到分类后的图像,其中,训练好的细粒度分类模型采用以下方式得到:根据获取到的待训练图像集的图像标签,对待训练图像集的图像进行分组,得到分组后的待训练图像集;其中,图像标签用于表征图像的类别,对分组后的待训练图像集中图像提取特征向量,得到特征向量组,采用特征向量组对细粒度分类模型进行训练,以确定出细粒度分类模型中损失函数与目标函数的取值最小时的模型参数,得到训练好的细粒度分类模型;也就是说,在本申请实施例中,通过采用预先训练好的细粒度分类模型对待分类图像进行分类,其中,训练好的细粒度分类模型是根据待训练图像集的图像标签对待训练图像集的图像进行分组并提取特征向量,采用特征向量组对细粒度分类模型进行训练,通过在模型中设置损失函数与目标函数,并在其取值最小时得到模型参数,从而得出训练好的细粒度分类模型,这样,通过在模型中设置损失函数与目标函数,使得得到的训练好的细粒度分类模型更加优化,在此基础上在进行图像分类时,提高了图像分类的准确性,进而提高了用户的体验。An embodiment of the present application provides an image classification method, which is applied to a terminal, and includes: acquiring an image to be classified, using a pre-trained fine-grained classification model, classifying the image to be classified, and obtaining a classified image, The trained fine-grained classification model is obtained in the following manner: according to the obtained image labels of the image set to be trained, the images of the image set to be trained are grouped to obtain the grouped image set to be trained; wherein, the image labels are used for Characterize the category of the image, extract the feature vector from the grouped images to be trained, obtain the feature vector group, use the feature vector group to train the fine-grained classification model, and determine the value of the loss function and the objective function in the fine-grained classification model. The model parameter with the smallest value is used to obtain a trained fine-grained classification model; that is, in the embodiment of the present application, the images to be classified are classified by using a pre-trained fine-grained classification model, wherein the trained fine-grained classification model is The classification model is to group the images of the image set to be trained according to the image labels of the image set to be trained and extract the feature vector, and use the feature vector group to train the fine-grained classification model. The model parameters are obtained when the value is the smallest, so as to obtain a trained fine-grained classification model. In this way, by setting the loss function and the objective function in the model, the obtained trained fine-grained classification model is more optimized. When performing image classification, the accuracy of image classification is improved, thereby improving user experience.

实施例二Embodiment 2

图3为本申请实施例提供的一种终端的结构示意图一,如图3所示,本申请实施例提供了一种终端,包括:FIG. 3 is a schematic structural diagram 1 of a terminal provided by an embodiment of the present application. As shown in FIG. 3 , an embodiment of the present application provides a terminal, including:

获取模块31,用于获取待分类图像;an acquisition module 31, configured to acquire images to be classified;

分类模块32,用于采用预先训练好的细粒度分类模型,对待分类图像进行分类,得到分类后的图像;The classification module 32 is configured to use a pre-trained fine-grained classification model to classify the images to be classified, and obtain the classified images;

其中,训练好的细粒度分类模型采用以下方式得到:Among them, the trained fine-grained classification model is obtained in the following way:

根据获取到的待训练图像集的图像标签,对待训练图像集的图像进行分组,得到分组后的待训练图像集;其中,图像标签用于表征图像的类别;According to the obtained image labels of the to-be-trained image set, the images of the to-be-trained image set are grouped to obtain a grouped to-be-trained image set; wherein, the image labels are used to characterize the category of the image;

对分组后的待训练图像集中图像提取特征向量,得到特征向量组;Extract feature vectors from the grouped images to be trained to obtain feature vector groups;

采用特征向量组对细粒度分类模型进行训练,以确定出细粒度分类模型中损失函数与目标函数的取值最小时的模型参数,得到训练好的细粒度分类模型。The feature vector group is used to train the fine-grained classification model to determine the model parameters when the value of the loss function and the objective function in the fine-grained classification model is the smallest, and the trained fine-grained classification model is obtained.

可选的,终端根据获取到的待训练图像集的图像标签,对待训练图像集的图像进行分组,得到分组后的待训练图像集中,包括:Optionally, the terminal groups the images in the to-be-trained image set according to the acquired image labels of the to-be-trained image set, and obtains the grouped to-be-trained image set, including:

将待训练图像集中的图像依次确定为第一图像;Determining the images in the image set to be trained as the first image in sequence;

针对第一图像,从除了第一图像以外的待训练图像集中,选取出第二图像和第三图像;其中,第二图像的图像标签与第一图像的图像标签相同,第三图像的图像标签与第一图像的图像标签不同;For the first image, select the second image and the third image from the set of images to be trained except the first image; wherein, the image label of the second image is the same as the image label of the first image, and the image label of the third image is the same as the image label of the first image. different from the image tag of the first image;

利用第一图像,第二图像和第三图像构成一组,以得到分组后的待训练图像集;Using the first image, the second image and the third image to form a group to obtain a grouped image set to be trained;

相应地,终端对分组后的待训练图像集中图像提取特征向量,得到特征向量组中,包括:Correspondingly, the terminal extracts feature vectors from the images in the grouped images to be trained, and obtains a feature vector group, including:

对第一图像,第二图像和第三图像,分别采用细粒度分类模型提取出特征向量,得到第一图像的特征向量,第二图像的特征向量和第三图像的特征向量;For the first image, the second image and the third image, a fine-grained classification model is used to extract the feature vector, respectively, to obtain the feature vector of the first image, the feature vector of the second image and the feature vector of the third image;

利用第一图像的特征向量,第二图像的特征向量和第三图像的特征向量,形成特征向量组。Using the feature vector of the first image, the feature vector of the second image and the feature vector of the third image, a feature vector group is formed.

可选的,终端还用于:Optionally, the terminal is also used to:

在对分组后的待训练图像集中图像提取特征向量,得到特征向量组之后,在采用特征向量组对细粒度分类模型进行训练,以确定出细粒度分类模型中损失函数与目标函数的取值最小时的模型参数,得到训练好的细粒度分类模型之前,选取出特征向量组中不满足预设条件的组别;After the feature vector is extracted from the grouped images to be trained, and the feature vector group is obtained, the fine-grained classification model is trained by using the feature vector group to determine the best value of the loss function and the objective function in the fine-grained classification model. Hourly model parameters, before obtaining the trained fine-grained classification model, select the groups in the feature vector group that do not meet the preset conditions;

从特征向量组中删除掉不满足预设条件的组别,以更新特征向量组。The groups that do not meet the preset conditions are deleted from the feature vector group to update the feature vector group.

可选的,终端选取出特征向量组中不满足预设条件的组别中,包括:Optionally, the terminal selects the groups in the feature vector group that do not meet the preset conditions, including:

计算第一图像的特征向量与第二图像的特征向量之间的第一距离值;calculating a first distance value between the feature vector of the first image and the feature vector of the second image;

当第一距离值大于等于第一预设阈值时,将包含第一图像和第二图像的组别确定为不满足预设条件的组别,并选取出不满足预设条件的组别;When the first distance value is greater than or equal to the first preset threshold, determine the group including the first image and the second image as a group that does not meet the preset condition, and select a group that does not meet the preset condition;

和/或,and / or,

计算第一图像的特征向量与第三图像的特征向量之间的第二距离值;calculating a second distance value between the feature vector of the first image and the feature vector of the third image;

当第二距离值小于等于第二预设阈值时,将包含第一图像和第三图像的组别确定为不满足预设条件的组别,并选取出不满足预设条件的组别。When the second distance value is less than or equal to the second preset threshold, the group including the first image and the third image is determined as a group that does not meet the preset condition, and the group that does not meet the preset condition is selected.

可选的,终端选取出特征向量组中不满足预设条件的组别中,包括:Optionally, the terminal selects the groups in the feature vector group that do not meet the preset conditions, including:

计算第一图像的特征向量与第二图像的特征向量之间的第一距离值,第一图像的特征向量与第三图像的特征向量之间的第二距离值;Calculate the first distance value between the feature vector of the first image and the feature vector of the second image, and the second distance value between the feature vector of the first image and the feature vector of the third image;

当第二距离值与第一距离值的差值大于第三预设阈值时,将包含第一图像,第二图像和第三图像的组别确定为不满足预设条件的组别,并选取出不满足预设条件的组别。When the difference between the second distance value and the first distance value is greater than the third preset threshold, the group including the first image, the second image and the third image is determined as a group that does not meet the preset conditions, and the selected group that does not meet the preset conditions.

可选的,终端采用特征向量组对细粒度分类模型进行训练,以确定出细粒度分类模型中损失函数与目标函数的取值最小时的模型参数,得到训练好的细粒度分类模型中,包括:Optionally, the terminal uses the feature vector group to train the fine-grained classification model, so as to determine the model parameters when the value of the loss function and the objective function in the fine-grained classification model is the smallest, and obtain the trained fine-grained classification model, including: :

采用特征向量组对细粒度分类模型进行训练,以确定出细粒度分类模型中损失函数取值最小且目标函数的取值最小时的模型参数,得到训练好的细粒度分类模型;Use the feature vector group to train the fine-grained classification model to determine the model parameters with the smallest value of the loss function and the smallest value of the objective function in the fine-grained classification model, and obtain the trained fine-grained classification model;

或者,or,

采用特征向量组对细粒度分类模型进行训练,以确定出细粒度分类模型中损失函数的取值与目标函数的取值之和最小时的模型参数,得到训练好的细粒度分类模型。The feature vector group is used to train the fine-grained classification model to determine the model parameters when the sum of the value of the loss function and the value of the objective function in the fine-grained classification model is the smallest, and the trained fine-grained classification model is obtained.

可选的,目标函数为与第一图像的特征向量,第二图像的特征向量,第三图像的特征向量和第三预设阈值相关的函数。Optionally, the objective function is a function related to the feature vector of the first image, the feature vector of the second image, the feature vector of the third image and the third preset threshold.

在实际应用中,上述获取模块31和分类模块32可由位于终端上的处理器实现,具体为CPU、微处理器(MPU,Microprocessor Unit)、数字信号处理器(DSP,Digital SignalProcessing)或现场可编程门阵列(FPGA,Field Programmable Gate Array)等实现。In practical applications, the acquisition module 31 and the classification module 32 can be implemented by a processor located on the terminal, specifically a CPU, a microprocessor (MPU, Microprocessor Unit), a digital signal processor (DSP, Digital Signal Processing) or a field programmable Gate Array (FPGA, Field Programmable Gate Array) etc.

图4为本申请实施例提供的一种终端的结构示意图二,如图4所示,本申请实施例提供了一种终端400,包括:FIG. 4 is a second schematic structural diagram of a terminal provided by an embodiment of the present application. As shown in FIG. 4 , an embodiment of the present application provides a terminal 400, including:

处理器41以及存储有所述处理器41可执行指令的存储介质42,所述存储介质42通过通信总线43依赖所述处理器41执行操作,当所述指令被所述处理器41执行时,执行上述实施例一所述的图像的分类方法。The processor 41 and the storage medium 42 storing the executable instructions of the processor 41. The storage medium 42 relies on the processor 41 to perform operations through the communication bus 43. When the instructions are executed by the processor 41, Execute the image classification method described in the first embodiment.

需要说明的是,实际应用时,终端中的各个组件通过通信总线43耦合在一起。可理解,通信总线43用于实现这些组件之间的连接通信。通信总线43除包括数据总线之外,还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见,在图4中将各种总线都标为通信总线43。It should be noted that, in practical application, various components in the terminal are coupled together through the communication bus 43 . It can be understood that the communication bus 43 is used to realize the connection communication between these components. In addition to the data bus, the communication bus 43 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the various buses are labeled as communication bus 43 in FIG. 4 .

本申请实施例提供了一种计算机存储介质,存储有可执行指令,当所述可执行指令被一个或多个处理器执行的时候,所述处理器执行实施例一所述的图像的分类方法。An embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the image classification method according to the first embodiment. .

其中,计算机可读存储介质可以是磁性随机存取存储器(ferromagnetic randomaccess memory,FRAM)、只读存储器(Read Only Memory,ROM)、可编程只读存储器(Programmable Read-Only Memory,PROM)、可擦除可编程只读存储器(ErasableProgrammable Read-Only Memory,EPROM)、电可擦除可编程只读存储器(ElectricallyErasable Programmable Read-Only Memory,EEPROM)、快闪存储器(Flash Memory)、磁表面存储器、光盘、或只读光盘(Compact Disc Read-Only Memory,CD-ROM)等存储器。Wherein, the computer-readable storage medium may be a magnetic random access memory (ferromagnetic random access memory, FRAM), a read only memory (Read Only Memory, ROM), a programmable read only memory (Programmable Read-Only Memory, PROM), an erasable memory In addition to programmable read-only memory (ErasableProgrammable Read-Only Memory, EPROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), flash memory (Flash Memory), magnetic surface memory, optical disks, Or memory such as Compact Disc Read-Only Memory (CD-ROM).

本领域内的技术人员应明白,本申请的实施例可提供为方法、系统、或计算机程序产品。因此,本申请可采用硬件实施例、软件实施例、或结合软件和硬件方面的实施例的形式。而且,本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器和光学存储器等)上实施的计算机程序产品的形式。As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.

本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

以上所述,仅为本申请的较佳实施例而已,并非用于限定本申请的保护范围。The above descriptions are only preferred embodiments of the present application, and are not intended to limit the protection scope of the present application.

Claims (10)

1. A method for classifying images is applied to a terminal and comprises the following steps:
acquiring an image to be classified;
classifying the images to be classified by adopting a pre-trained fine-grained classification model to obtain classified images;
the trained fine-grained classification model is obtained by adopting the following method:
grouping images of the image set to be trained according to the acquired image labels of the image set to be trained to obtain a grouped image set to be trained; wherein the image tag is used for characterizing the category of the image;
extracting a characteristic vector from the grouped images to be trained in the set to obtain a characteristic vector group;
and training the fine-grained classification model by adopting the feature vector group to determine a model parameter when the values of the loss function and the target function in the fine-grained classification model are minimum, so as to obtain the trained fine-grained classification model.
2. The method according to claim 1, wherein the grouping images of the image set to be trained according to the obtained image labels of the image set to be trained to obtain a grouped image set to be trained, comprises:
sequentially determining the images in the image set to be trained as first images;
selecting a second image and a third image from the image set to be trained except the first image aiming at the first image; wherein the image label of the second image is the same as the image label of the first image, and the image label of the third image is different from the image label of the first image;
forming a group by using the first image, the second image and the third image to obtain the grouped image set to be trained;
correspondingly, the extracting the feature vectors from the grouped images to be trained in the set to obtain a feature vector group includes:
extracting feature vectors of the first image, the second image and the third image by adopting the fine-grained classification model respectively to obtain the feature vector of the first image, the feature vector of the second image and the feature vector of the third image;
and forming the characteristic vector group by using the characteristic vector of the first image, the characteristic vector of the second image and the characteristic vector of the third image.
3. The method according to claim 2, wherein after extracting feature vectors from the grouped images in the image set to be trained to obtain a feature vector group, before training a fine-grained classification model by using the feature vector group to determine a model parameter when values of a loss function and an objective function in the fine-grained classification model are minimum to obtain the trained fine-grained classification model, the method further comprises:
selecting a group which does not meet a preset condition from the feature vector group;
and deleting the groups which do not meet the preset condition from the feature vector group so as to update the feature vector group.
4. The method of claim 3, wherein the selecting the group of the feature vector group that does not satisfy the predetermined condition comprises:
calculating a first distance value between the feature vector of the first image and the feature vector of the second image;
when the first distance value is larger than or equal to a first preset threshold value, determining a group containing the first image and the second image as the group which does not meet the preset condition, and selecting the group which does not meet the preset condition;
and/or the presence of a gas in the gas,
calculating a second distance value between the feature vector of the first image and the feature vector of the third image;
and when the second distance value is smaller than or equal to a second preset threshold value, determining the group comprising the first image and the third image as the group which does not meet the preset condition, and selecting the group which does not meet the preset condition.
5. The method of claim 3, wherein the selecting the group of the feature vector group that does not satisfy the predetermined condition comprises:
calculating a first distance value between the feature vector of the first image and the feature vector of the second image, and a second distance value between the feature vector of the first image and the feature vector of the third image;
and when the difference value between the second distance value and the first distance value is larger than a third preset threshold value, determining the group containing the first image, the second image and the third image as a group which does not meet a preset condition, and selecting the group which does not meet the preset condition.
6. The method of claim 1, wherein the training of the fine-grained classification model by using the feature vector group to determine a model parameter when values of a loss function and an objective function in the fine-grained classification model are minimum to obtain the trained fine-grained classification model comprises:
training a fine-grained classification model by adopting the feature vector group to determine a model parameter when a loss function value is minimum and a target function value is minimum in the fine-grained classification model, so as to obtain the trained fine-grained classification model;
or,
and training the fine-grained classification model by adopting the feature vector group to determine a model parameter when the sum of the value of the loss function and the value of the target function in the fine-grained classification model is minimum, so as to obtain the trained fine-grained classification model.
7. The method according to claim 5 or 6, wherein the objective function is a function related to the feature vector of the first image, the feature vector of the second image, the feature vector of the third image and the third preset threshold.
8. A terminal, comprising:
the acquisition module is used for acquiring an image to be classified;
the classification module is used for classifying the images to be classified by adopting a pre-trained fine-grained classification model to obtain classified images;
the trained fine-grained classification model is obtained by adopting the following method:
grouping images of the image set to be trained according to the acquired image labels of the image set to be trained to obtain a grouped image set to be trained; wherein the image tag is used for characterizing the category of the image;
extracting a characteristic vector from the grouped images to be trained in the set to obtain a characteristic vector group;
and training the fine-grained classification model by adopting the feature vector group to determine a model parameter when the values of the loss function and the target function in the fine-grained classification model are minimum, so as to obtain the trained fine-grained classification model.
9. A terminal, characterized in that the terminal comprises: a processor and a storage medium storing instructions executable by the processor to perform operations in dependence on the processor via a communication bus, the instructions when executed by the processor performing the method of classifying an image according to any one of claims 1 to 7.
10. A computer storage medium having stored thereon executable instructions which, when executed by one or more processors, perform the method of classifying an image of any one of claims 1 to 7.
CN202010078654.1A 2020-02-03 2020-02-03 Image classification method, terminal and computer storage medium Pending CN111325242A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010078654.1A CN111325242A (en) 2020-02-03 2020-02-03 Image classification method, terminal and computer storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010078654.1A CN111325242A (en) 2020-02-03 2020-02-03 Image classification method, terminal and computer storage medium

Publications (1)

Publication Number Publication Date
CN111325242A true CN111325242A (en) 2020-06-23

Family

ID=71168785

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010078654.1A Pending CN111325242A (en) 2020-02-03 2020-02-03 Image classification method, terminal and computer storage medium

Country Status (1)

Country Link
CN (1) CN111325242A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111914761A (en) * 2020-08-04 2020-11-10 南京华图信息技术有限公司 Thermal infrared face recognition method and system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101436302A (en) * 2008-12-10 2009-05-20 南京大学 Method for sorting colors of colorful three-dimensional model based on neural network
CN109359684A (en) * 2018-10-17 2019-02-19 苏州大学 A fine-grained vehicle identification method based on weakly supervised localization and subcategory similarity measure
CN109784366A (en) * 2018-12-07 2019-05-21 北京飞搜科技有限公司 The fine grit classification method, apparatus and electronic equipment of target object
CN110263659A (en) * 2019-05-27 2019-09-20 南京航空航天大学 A kind of finger vein identification method and system based on triple loss and lightweight network
WO2019227614A1 (en) * 2018-06-01 2019-12-05 平安科技(深圳)有限公司 Method and device for obtaining triple of samples, computer device and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101436302A (en) * 2008-12-10 2009-05-20 南京大学 Method for sorting colors of colorful three-dimensional model based on neural network
WO2019227614A1 (en) * 2018-06-01 2019-12-05 平安科技(深圳)有限公司 Method and device for obtaining triple of samples, computer device and storage medium
CN109359684A (en) * 2018-10-17 2019-02-19 苏州大学 A fine-grained vehicle identification method based on weakly supervised localization and subcategory similarity measure
CN109784366A (en) * 2018-12-07 2019-05-21 北京飞搜科技有限公司 The fine grit classification method, apparatus and electronic equipment of target object
CN110263659A (en) * 2019-05-27 2019-09-20 南京航空航天大学 A kind of finger vein identification method and system based on triple loss and lightweight network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
王耀玮: "基于卷积神经网络的细粒度车辆识别系统研究", pages 034 - 1452 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111914761A (en) * 2020-08-04 2020-11-10 南京华图信息技术有限公司 Thermal infrared face recognition method and system

Similar Documents

Publication Publication Date Title
CN112949780B (en) Feature model training method, device, equipment and storage medium
Chowdhury et al. One-to-many face recognition with bilinear cnns
CN108304936B (en) Machine learning model training method and device, and expression image classification method and device
US11023806B2 (en) Learning apparatus, identifying apparatus, learning and identifying system, and recording medium
CN108681746B (en) Image identification method and device, electronic equipment and computer readable medium
CN106897738B (en) A pedestrian detection method based on semi-supervised learning
CN109117879B (en) Image classification method, device and system
JP2022141931A (en) Method and device for training living body detection model, method and apparatus for living body detection, electronic apparatus, storage medium, and computer program
US20120300980A1 (en) Learning device, learning method, and program
CN109919296B (en) A deep neural network training method, device and computer equipment
CN108595585A (en) Sample data sorting technique, model training method, electronic equipment and storage medium
WO2020114108A1 (en) Clustering result interpretation method and device
CN106709528A (en) Method and device of vehicle reidentification based on multiple objective function deep learning
CN111325276A (en) Image classification method and apparatus, electronic device, and computer-readable storage medium
Ryu et al. Generalized convolutional forest networks for domain generalization and visual recognition
WO2022199214A1 (en) Sample expansion method, training method and system, and sample learning system
CN111414875A (en) 3D point cloud head pose estimation system based on deep regression forest
Yang et al. Shape-based object retrieval by contour segment matching
CN111340057A (en) Classification model training method and device
CN113468936A (en) Food material identification method, device and equipment
CN117011577A (en) Image classification method, apparatus, computer device and storage medium
CN111325242A (en) Image classification method, terminal and computer storage medium
US20210406688A1 (en) Method and device with classification verification
CN107633421A (en) A kind of processing method and processing device of market prediction data
Yu et al. A joint multi-task cnn for cross-age face recognition

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200623