WO2020186887A1 - Target detection method, device and apparatus for continuous small sample images - Google Patents

Target detection method, device and apparatus for continuous small sample images Download PDF

Info

Publication number
WO2020186887A1
WO2020186887A1 PCT/CN2019/130818 CN2019130818W WO2020186887A1 WO 2020186887 A1 WO2020186887 A1 WO 2020186887A1 CN 2019130818 W CN2019130818 W CN 2019130818W WO 2020186887 A1 WO2020186887 A1 WO 2020186887A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
classifier
background
sample images
continuous small
Prior art date
Application number
PCT/CN2019/130818
Other languages
French (fr)
Chinese (zh)
Inventor
乔宇
陈贤煜
王亚立
Original Assignee
深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳先进技术研究院 filed Critical 深圳先进技术研究院
Publication of WO2020186887A1 publication Critical patent/WO2020186887A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2155Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

Disclosed is a target detection method for continuous small sample images, comprising: decoupling a target detector by means of decoupling a target-background classifier into a background classifier and a target classifier; obtaining a single-stage target detector after parameters are initialized and decoupled according to the weight rubbing strategy by using the previous detection task framework; obtaining supervision parameters prior to the current detection task by a knowledge distillation method, and training or detecting the target by the supervision parameters via the initialized and decoupled target detector. The learning complexity is favorably reduced by decoupling; the training requirements for continuous small sample images can be effectively adapted by the initialization of rubbing strategy; the knowledge of previous detection tasks can be effectively utilized by means of knowledge distillation so as to effectively prevent old knowledge from being forgotten and to achieve a balance between the forgetting of old knowledge and the acquisition of new knowledge.

Description

一种连续小样本图像的目标检测方法、装置及设备Target detection method, device and equipment for continuous small sample images 技术领域Technical field
本申请属于图像处理领域,尤其涉及一种连续小样本图像的目标检测方法、装置及设备。This application belongs to the field of image processing, and in particular relates to a target detection method, device and equipment for continuous small sample images.
背景技术Background technique
通过深度学习检测样本的方式,可以有效的完成对图像进行目标检测和识别。并且,随着深度检测技术的发展,对目标检测主要包括两种检测框架,即两步检测框架和单步检测框架,这些框架在主要的国际公开数据集中取得了明显的效果。Through deep learning to detect samples, it can effectively complete the target detection and recognition of the image. Moreover, with the development of in-depth detection technology, target detection mainly includes two detection frameworks, namely, a two-step detection framework and a single-step detection framework. These frameworks have achieved significant results in major international public data sets.
但是,深度检测一般是由具有完整的边界框标定的大规模数据集所驱动的,在实际的图像目标检测过程中,收集这样的数据需要耗费大量的人力物力。当对连续小样本进行训练或检测时,新的任务通常会持续到来,这些任务中会面临新的检测任务或者新的物体种类,每个目标检测任务仅仅有小样本的边界框标记,使用随机化的检测架构将会容易导致严重的过拟合问题,以及,在关注当前检测任务时,可能会遭受严重的遗忘现象,不能有效的维持在检测之前任务时的性能。However, depth detection is generally driven by a large-scale data set with complete bounding box calibration. In the actual image target detection process, collecting such data requires a lot of manpower and material resources. When training or detecting continuous small samples, new tasks usually continue to come. These tasks will face new detection tasks or new object types. Each target detection task only has a small sample of bounding box labels, using random The modernized detection architecture will easily lead to serious over-fitting problems, and may suffer serious forgetting when focusing on the current detection task, and cannot effectively maintain the performance of the previous task.
技术问题technical problem
有鉴于此,本申请实施例提供了一种连续小样本图像的目标检测方法、装置及设备,以解决现有技术中对于连续小样本图像检测时,容易导致过拟合问题,以及可能会遭受严重的遗忘现象的问题。In view of this, the embodiments of the present application provide a target detection method, device and equipment for continuous small-sample images to solve the problem of over-fitting and may suffer from the detection of continuous small-sample images in the prior art. The problem of serious forgetting.
技术解决方案Technical solutions
本申请实施例的第一方面提供了一种连续小样本图像的目标检测方法,所述小样本图像的目标检测方法包括:The first aspect of the embodiments of the present application provides a target detection method for continuous small sample images, and the target detection method for small sample images includes:
对目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器;Decouple the target detector, and decouple the target-background classifier into a background classifier and a target classifier;
根据权重拓印策略以及利用先前检测任务框架从而获得参数初始化解耦后的单阶目标检测器;According to the weighted rubbing strategy and using the previous detection task framework to obtain the single-order target detector after parameter initialization and decoupling;
通过知识蒸馏方法获取当前检测任务之前的监督参数,通过初始化后的解耦后的目标检测器结合所述监督参数对目标进行训练或检测。The supervised parameters before the current detection task are obtained by the knowledge distillation method, and the target is trained or detected by combining the supervised parameters with the decoupled target detector after initialization.
结合第一方面,在第一方面的第一种可能实现方式中,所述对目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器的步骤包括:With reference to the first aspect, in the first possible implementation of the first aspect, the step of decoupling the target detector and decoupling the target-background classifier into a background classifier and a target classifier includes:
对单阶目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器,得到背景分类器的分数向量为:The single-stage target detector is decoupled, and the target-background classifier is decoupled into a background classifier and a target classifier. The score vector of the background classifier is obtained as:
Figure PCTCN2019130818-appb-000001
Figure PCTCN2019130818-appb-000001
得到目标分类器的分数向量为:The score vector of the target classifier is:
Figure PCTCN2019130818-appb-000002
Figure PCTCN2019130818-appb-000002
其中,
Figure PCTCN2019130818-appb-000003
是背景分类器经过归一化指数函数处理之前的预测向量,p obj/p bg是目标/背景的概率,
Figure PCTCN2019130818-appb-000004
是目标分类器经过归一化指数函数处理之前的预测向量,q i是属于第i个目标的概率i=1,2,...,C。
among them,
Figure PCTCN2019130818-appb-000003
Is the prediction vector before the background classifier is processed by the normalized exponential function, p obj /p bg is the probability of the target/background,
Figure PCTCN2019130818-appb-000004
Is the prediction vector before the target classifier is processed by the normalized exponential function, and q i is the probability of belonging to the i-th target i=1, 2, ..., C.
结合第一方面,在第一方面的第二种可能实现方式中,所述根据权重拓印策略以及利用先前检测任务框架从而获得参数初始化解耦后的单阶目标检测器的步骤包括:With reference to the first aspect, in a second possible implementation manner of the first aspect, the steps of the weighted rubbing strategy and the use of a previous detection task framework to obtain a single-order target detector after parameter initialization and decoupling include:
获取与检测任务中的目标类别;Obtain and detect the target category in the task;
根据所述目标类别,在源域中获取多个包含所述目标类别的先验框;According to the target category, obtain a plurality of a priori boxes containing the target category in the source domain;
根据所获取的多个源域中的先验框分别计算所述检测任务中的目标类别的预测向量;Respectively calculating the prediction vector of the target category in the detection task according to the acquired a priori boxes in the multiple source domains;
对多个所述预测向量求和取平均后进行归一化处理,得到目标类别对应的 参数向量。The multiple prediction vectors are summed and averaged, and then normalized to obtain the parameter vector corresponding to the target category.
结合第一方面的第二种可能实现方式,在第一方面的第三种可能实现方式中,当所述检测任务中包括多个目标类别时,根据多个目标类别所对应的参数向量,得到目标分类器的初始化参数矩阵。In combination with the second possible implementation manner of the first aspect, in the third possible implementation manner of the first aspect, when the detection task includes multiple target categories, according to the parameter vectors corresponding to the multiple target categories, obtain The initialization parameter matrix of the target classifier.
结合第一方面,在第一方面的第四种可能实现方式中,所述根据权重拓印策略以及利用先前检测任务框架从而获得参数初始化解耦后的单阶目标检测器的步骤包括:With reference to the first aspect, in a fourth possible implementation of the first aspect, the steps of the weighted rubbing strategy and the use of the previous detection task framework to obtain a single-order target detector after parameter initialization and decoupling include:
根据当前检测任务T k之前的检测任务T k-1所使用的主干参数、边界框回归参数、背景分类器参数初始化当前检测任务的主干、边界框回归和背景分类器。 Initialize the backbone, bounding box regression and background classifiers of the current detection task according to the backbone parameters, bounding box regression parameters, and background classifier parameters used in the detection task T k-1 before the current detection task T k .
结合第一方面,在第一方面的第五种可能实现方式中,所述通过知识蒸馏方法获取当前检测任务之前的监督参数,通过初始化后的解耦单阶目标检测器结合所述监督参数对目标进行训练的步骤包括:In combination with the first aspect, in a fifth possible implementation manner of the first aspect, the supervised parameter pair before the current detection task is obtained by the knowledge distillation method, and the supervised parameter pair is combined with the decoupled single-stage target detector after initialization. The steps to train the target include:
根据公式计算当前目标检测任务T k的总损失函数: Calculate the total loss function of the current target detection task T k according to the formula:
Figure PCTCN2019130818-appb-000005
Figure PCTCN2019130818-appb-000005
根据所述总损失函数规范当前检测任务的目标检测器,其中,主要损失
Figure PCTCN2019130818-appb-000006
包括边界框回归损失,背景分类器损失,目标加背景分类器损失,将当前检测任务的图片送入当前检测任务之前的目标检测器获得监督参数
Figure PCTCN2019130818-appb-000007
λ为权重系数。
Standardize the target detector of the current detection task according to the total loss function, where the main loss
Figure PCTCN2019130818-appb-000006
Including bounding box regression loss, background classifier loss, target plus background classifier loss, send the image of the current detection task to the target detector before the current detection task to obtain the supervision parameters
Figure PCTCN2019130818-appb-000007
λ is the weight coefficient.
本申请实施例的第二方面提供了一种连续小样本图像的目标检测装置,所述连续小样本图像的目标检测装置包括:A second aspect of the embodiments of the present application provides a target detection device for continuous small sample images, and the target detection device for continuous small sample images includes:
解耦单元,用于对目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器;The decoupling unit is used to decouple the target detector and decouple the target-background classifier into a background classifier and a target classifier;
拓印单元,用于根据权重拓印策略以及利用先前检测任务框架从而获得参数初始化解耦后的单阶目标检测器;The rubbing unit is used for rubbing the strategy according to the weight and using the previous detection task framework to obtain the single-stage target detector after parameter initialization and decoupling;
蒸馏单元,用于通过知识蒸馏方法获取当前检测任务之前的监督参数,通过初始化后的解耦后的目标检测器结合所述监督参数对目标进行训练或检测。The distillation unit is used to obtain the supervision parameters before the current detection task through the knowledge distillation method, and train or detect the target through the decoupled target detector after initialization and the supervision parameters.
结合第二方面,在第二方面的第一种可能实现方式中,所述解耦单元用于:With reference to the second aspect, in the first possible implementation manner of the second aspect, the decoupling unit is configured to:
对单阶目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器,得到背景分类器的分数向量为:The single-stage target detector is decoupled, and the target-background classifier is decoupled into a background classifier and a target classifier. The score vector of the background classifier is obtained as:
Figure PCTCN2019130818-appb-000008
Figure PCTCN2019130818-appb-000008
得到目标分类器的分数向量为:The score vector of the target classifier is:
Figure PCTCN2019130818-appb-000009
Figure PCTCN2019130818-appb-000009
其中,
Figure PCTCN2019130818-appb-000010
是背景分类器经过归一化指数函数处理之前的预测向量,p obj/p bg是目标/背景的概率,
Figure PCTCN2019130818-appb-000011
是目标分类器经过归一化指数函数处理之前的预测向量,q i是属于第i个目标的概率i=1,2,...,C。
among them,
Figure PCTCN2019130818-appb-000010
Is the prediction vector before the background classifier is processed by the normalized exponential function, p obj /p bg is the probability of the target/background,
Figure PCTCN2019130818-appb-000011
Is the prediction vector before the target classifier is processed by the normalized exponential function, and q i is the probability of belonging to the i-th target i=1, 2, ..., C.
本申请实施例的第三方面提供了一种连续小样本图像的目标检测设备,包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序,所述处理器执行所述计算机程序时实现如权第一方面任一项所述连续小样本图像的目标检测方法的步骤。The third aspect of the embodiments of the present application provides a target detection device for continuous small-sample images, including a memory, a processor, and a computer program stored in the memory and running on the processor. The processor When the computer program is executed, the steps of the target detection method for continuous small-sample images as described in any one of the first aspect are implemented.
本申请实施例的第四方面提供了一种计算机可读存储介质,所述计算机可 读存储介质存储有计算机程序,所述计算机程序被处理器执行时实现如第一方面任一项所述连续小样本图像的目标检测方法的步骤。The fourth aspect of the embodiments of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the continuous Steps of the target detection method for small sample images.
有益效果Beneficial effect
本申请实施例与现有技术相比存在的有益效果是:通过将目标-背景分类器解耦成背景分类器和目标分类器,解耦后的分类器可以通过权重拓印策略或在先检测任务参数初始化,有利于减少学习的复杂度,并且通过拓印策略初始化,可以有效的适应连续小样本图像的训练要求,通过知识蒸馏方法可以有效的利用之前检测任务的知识,可以有效的对抗旧知识的遗忘,实现旧知识的遗忘与新知识的获取之间的平衡。Compared with the prior art, the embodiment of the present application has the beneficial effect that by decoupling the target-background classifier into a background classifier and a target classifier, the decoupled classifier can use a weighted rubbing strategy or prior detection Task parameter initialization is beneficial to reduce the complexity of learning, and through the initialization of the rubbing strategy, it can effectively adapt to the training requirements of continuous small sample images. The knowledge distillation method can effectively use the knowledge of the previous detection task, which can effectively combat the old The forgetting of knowledge realizes the balance between the forgetting of old knowledge and the acquisition of new knowledge.
附图说明Description of the drawings
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only of the present application. For some embodiments, for those of ordinary skill in the art, other drawings can be obtained from these drawings without creative labor.
图1是本申请实施例提供的一种连续小样本图像的目标检测方法的实现流程示意图;FIG. 1 is a schematic diagram of the implementation process of a target detection method for continuous small sample images provided by an embodiment of the present application;
图2是本申请实施例提供的单阶段检测器的原始框架示意图;2 is a schematic diagram of the original framework of a single-stage detector provided by an embodiment of the present application;
图3是本申请实施例提供的解耦后的单阶段检测器的框架示意图;3 is a schematic diagram of the framework of a decoupled single-stage detector provided by an embodiment of the present application;
图4是本申请实施例提供的一种连续小样本图像的目标检测装置示意图;4 is a schematic diagram of a target detection device for continuous small sample images provided by an embodiment of the present application;
图5是本申请实施例提供一种图像检测设备的示意图。FIG. 5 is a schematic diagram of an image detection device provided by an embodiment of the present application.
本发明的实施方式Embodiments of the invention
以下描述中,为了说明而不是为了限定,提出了诸如特定系统结构、技术之类的具体细节,以便透彻理解本申请实施例。然而,本领域的技术人员应当清楚,在没有这些具体细节的其它实施例中也可以实现本申请。在其它情况中,省略对众所周知的系统、装置、电路以及方法的详细说明,以免不必要的细节妨碍本申请的描述。In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are proposed for a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obstructing the description of this application.
为了说明本申请所述的技术方案,下面通过具体实施例来进行说明。In order to illustrate the technical solutions described in the present application, specific embodiments are used for description below.
图1为本申请实施例提供的一种连续小样本图像的目标检测方法的实现流程示意图,详述如下:Figure 1 is a schematic diagram of the implementation process of a method for target detection of continuous small sample images provided by an embodiment of the application, and the details are as follows:
在步骤S101中,对目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器;In step S101, the target detector is decoupled, and the target-background classifier is decoupled into a background classifier and a target classifier;
具体的,所述解耦,是指对目标检测器中的模块解耦成一系列子模块,从而减少连续小样本检测任务结构的复杂性。所述目标检测器可以选择单阶段的目标检测器,从而可以高效、精确的对目标进行检测。如图2所示,一个曲型的单阶段目标检测器包括主干、边界框回归以及目标-背景分类器。Specifically, the decoupling refers to decoupling the modules in the target detector into a series of sub-modules, thereby reducing the complexity of the task structure for continuous small sample detection. The target detector can select a single-stage target detector, so that the target can be detected efficiently and accurately. As shown in Figure 2, a curved single-stage target detector includes a backbone, bounding box regression, and target-background classifier.
对于新的检测任务,由于主干和边界框回归被不同的目标类别所共享,它们可以从上一次检测任务初始化而得到。相反地,目标-背景分类器必须被随机初始化,这是由于新的目标种类可能在先前的任务中未曾见过。然而这个事实加剧了学习目标-背景分类器的任务的难度,尤其是新任务是小样本的情况下。为了减少训练负担,本申请提出了把目标-背景分类器解耦成背景分类器和目标分类器,如图3所示。假设背景分类器的分数向量为For a new detection task, since the backbone and bounding box regression are shared by different target categories, they can be obtained from the initialization of the previous detection task. Conversely, the target-background classifier must be initialized randomly, because the new target category may not have been seen in previous tasks. However, this fact exacerbates the difficulty of the task of learning the target-background classifier, especially when the new task is a small sample. In order to reduce the training burden, this application proposes to decouple the target-background classifier into a background classifier and a target classifier, as shown in Figure 3. Suppose the score vector of the background classifier is
Figure PCTCN2019130818-appb-000012
Figure PCTCN2019130818-appb-000012
其中
Figure PCTCN2019130818-appb-000013
是背景分类器经过归一化指数函数softmax处理之前的预测向量,并且p obj/p bg是目标/背景的概率。目标分类器的分数为
among them
Figure PCTCN2019130818-appb-000013
Is the prediction vector before the background classifier is processed by the normalized exponential function softmax, and p obj /p bg is the probability of the target/background. The score of the target classifier is
Figure PCTCN2019130818-appb-000014
Figure PCTCN2019130818-appb-000014
其中
Figure PCTCN2019130818-appb-000015
是目标分类器经过归一化指数函数softmax处理之 前的预测向量,q i是属于第i个目标的概率i=1,2,...,C。之后,我们可以直接计算目标-背景的概率分数向量,即
among them
Figure PCTCN2019130818-appb-000015
Is the prediction vector before the target classifier is processed by the normalized exponential function softmax, and q i is the probability of belonging to the i-th target i=1, 2, ..., C. After that, we can directly calculate the target-background probability score vector, namely
v=[p objq 1,p objq 2,...,p objq c,p bg]   (3) v=[p obj q 1 , p obj q 2 ,..., p obj q c , p bg ] (3)
通过解耦,我们仅仅需要单独初始化目标和背景分类器而不是初始化一个目标-背景分类器。对于一个新的检测任务,背景分类器可以从上一个任务初始化得到,这是因为二元分类器在不同的目标中可以共享参数。目标分类器是一个多分类器,可以通过步骤S102提出拓印的方法解决因新任务的到来而导致的随机初始化问题。Through decoupling, we only need to initialize the target and background classifiers separately instead of initializing a target-background classifier. For a new detection task, the background classifier can be initialized from the previous task. This is because the binary classifier can share parameters among different targets. The target classifier is a multi-classifier, and the rubbing method can be proposed in step S102 to solve the random initialization problem caused by the arrival of a new task.
通过解耦操作,可以将检测架构分解为主干,边界框回归,目标分类器和背景分类器。当新任务到来的时候,可以减少学习的困难。Through decoupling operation, the detection architecture can be decomposed into the backbone, bounding box regression, target classifier and background classifier. When new tasks come, you can reduce learning difficulties.
在步骤S102中,根据权重拓印策略以及利用先前检测任务框架从而获得参数初始化解耦后的单阶目标检测器;In step S102, a single-order target detector after parameter initialization and decoupling is obtained according to the weighted rubbing strategy and the previous detection task framework;
所述权重拓印策略,是指选用公开数据集作为源域,查找检测任务中的目标类别在源域中所对应的先验框,根据先验框计算检测任务中的目标的预测向量,对预测向量进行求和取平均后,进行归一化处理得到目标类别对应的参数向量。在获取所述参数向量后,即可根据权重拓印策略所得到的参数向量初始化目标检测器中的目标分类器。The weighted rubbing strategy refers to selecting the public data set as the source domain, searching for the a priori box corresponding to the target category in the detection task in the source domain, and calculating the prediction vector of the target in the detection task according to the a priori box. After the prediction vector is summed and averaged, it is normalized to obtain the parameter vector corresponding to the target category. After obtaining the parameter vector, the target classifier in the target detector can be initialized according to the parameter vector obtained by the weight rubbing strategy.
由于每一个检测任务都是小样本,一个好的子模型初始化方案就会显得相当重要。本申请提出了一种简单但有效的解决方案。首先,本申请可以使用现已存在的大规模国际公开数据集(例如COCO)作为源域,并且预训练图3所示的解耦架构。由于主干,边界框回归和背景分类器在不同的目标种类中可 以被共享。因此,第T k-1检测任务可以被重复用来初始化第T k个检测任务,其中T 0=S是源任务。 Since each detection task is a small sample, a good sub-model initialization scheme will be very important. This application proposes a simple but effective solution. First, this application can use an existing large-scale international public data set (such as COCO) as the source domain, and pre-train the decoupling architecture shown in FIG. 3. Due to the backbone, bounding box regression and background classifiers can be shared among different target categories. Accordingly, the detection task T k-1 may be repeated to initialize the task of detection of T k, where T 0 = S is the source task.
对于目标分类器的初始化。由于新检测任务和之前检测任务的目标种类不同,当新任务到来时,这个分类器需要能够适应新任务。为了避免随机初始化,本申请可以采用权重拓印策略,重新把分类器调整成适用于目标检测设置的要求。For the initialization of the target classifier. Since the target types of the new detection task and the previous detection task are different, when the new task arrives, the classifier needs to be able to adapt to the new task. In order to avoid random initialization, this application can adopt a weight rubbing strategy to re-adjust the classifier to meet the requirements of target detection settings.
假设源任务有C S个种类,第T k个检测任务有
Figure PCTCN2019130818-appb-000016
个种类。第k个连续小样本检测任务的目标分类器集中于寻找一个参数矩阵
Figure PCTCN2019130818-appb-000017
用于搭建源域预测向量(归一化指数函数处理softmax之前)到目标域的预测向量的映射。下一步,我们可以在目标域的预测向量上使用softmax,从而得到最后的分类结果。
Suppose there are C S types of source tasks, and T k detection tasks have
Figure PCTCN2019130818-appb-000016
Categories. The target classifier of the kth continuous small sample detection task focuses on finding a parameter matrix
Figure PCTCN2019130818-appb-000017
It is used to build the mapping of the source domain prediction vector (before the normalized exponential function is processed by softmax) to the target domain prediction vector. In the next step, we can use softmax on the prediction vector of the target domain to get the final classification result.
假设在第T k个检测任务中有目标A这一个类别。首先,可以把含有目标A的照片送入单阶段的检测器主干,然后到达源域的目标分类器。对于第n个含有目标A的先验框,可以取得预测向量(softmax之前)
Figure PCTCN2019130818-appb-000018
的结果,而C S是源域目标种类的数目。然后,所选择的所有关于目标A的先验框的置信度比较大的先验框(大于某一阈值),所述置信度由先验框和真正的边界框的交并比(intersectionofunion,IoU)决定。我们用L 2归一化源域的预测向量,把这些归一化之后的预测向量求和取平均,假设总共有N个向量。我们进一步 地归一化这个平均向量,把它作为关于目标A的参数向量:
Assume that there is a category of target A in the T kth detection task. First, the photos containing target A can be sent to the single-stage detector backbone, and then reach the target classifier in the source domain. For the n-th prior box containing target A, the prediction vector (before softmax) can be obtained
Figure PCTCN2019130818-appb-000018
And C S is the number of target types in the source domain. Then, all selected a priori boxes with a higher confidence (greater than a certain threshold) of the a priori box about the target A, the confidence is determined by the intersection of the a priori box and the true bounding box (intersection of union, IoU ) Decide. We use L 2 to normalize the prediction vectors of the source domain, and average these normalized prediction vectors, assuming there are N vectors in total. We further normalize this average vector and use it as a parameter vector about target A:
Figure PCTCN2019130818-appb-000019
Figure PCTCN2019130818-appb-000019
可以发现,对于第T k个检测任务上的目标A,W A总结了丰富且相关的
Figure PCTCN2019130818-appb-000020
在源域的表示信息。它可以看成是目标A的原型并且提供一个可靠的参数初始化方案。最后,我们采用类似方案对第T k个检测任务上的其它类别进行相似的操作,从而得到初始化参数矩阵
Figure PCTCN2019130818-appb-000021
作为目标分类器。给一张查询的图片,
Figure PCTCN2019130818-appb-000022
可以把来自源域的先验框的预测向量转换成第T k个检测任务域上的向量。
It can be found, for the first goal on A T k detectors task, W A summary of the rich and relevant
Figure PCTCN2019130818-appb-000020
Representation information in the source domain. It can be regarded as a prototype of objective A and provides a reliable parameter initialization scheme. Finally, we use a similar scheme to perform similar operations on other categories on the T k- th detection task to obtain the initialization parameter matrix
Figure PCTCN2019130818-appb-000021
As the target classifier. Give a picture of the query,
Figure PCTCN2019130818-appb-000022
The prediction vector of the a priori box from the source domain can be converted into a vector on the T k- th detection task domain.
Figure PCTCN2019130818-appb-000023
Figure PCTCN2019130818-appb-000023
之后把它送入归一化指数函数softmax从而实现分类。Then it is sent to the normalized exponential function softmax to achieve classification.
综上,拓印允许所有的已经被解耦的子模型快速部署,从而可以在小样本连续检测任务下有效减缓过拟合现象。In summary, rubbing allows all the decoupled sub-models to be deployed quickly, which can effectively slow down the overfitting phenomenon under the task of continuous detection with small samples.
在步骤S103中,通过知识蒸馏方法获取当前检测任务之前的监督参数,通过初始化后的解耦后的目标检测器结合所述监督参数对目标进行训练或检测。In step S103, the supervised parameter before the current detection task is obtained by the knowledge distillation method, and the target is trained or detected by combining the supervised parameter with the decoupled target detector after initialization.
在拓印我们的解耦架构之后,我们开始训练第T k个目标检测任务。但是,仅仅聚焦于当前的任务将会导致灾难性的遗忘,也就是检测框架会逐渐忘记之前任务学到的知识,因此这些任务的检测性能将会大幅度退化。为了避免这种现象,在训练当前任务的时候,我们将蒸馏方法作为一个正则化项。具体地,我们记第T k个目标检测任务的总损失函数为 After rubbing our decoupling architecture, we start to train the T kth target detection task. However, focusing only on the current task will lead to catastrophic forgetting, that is, the detection framework will gradually forget the knowledge learned in the previous tasks, so the detection performance of these tasks will be greatly degraded. In order to avoid this phenomenon, when training the current task, we use the distillation method as a regularization term. Specifically, we record the total loss function of the T kth target detection task as
Figure PCTCN2019130818-appb-000024
Figure PCTCN2019130818-appb-000024
主要损失
Figure PCTCN2019130818-appb-000025
包括边界框回归(平滑L 1)损失,背景分类器(互熵)损失,目标加背景分类器(互熵)损失。它可以用来训练第T k个目标检测任务的检测架构。
Main loss
Figure PCTCN2019130818-appb-000025
Including bounding box regression (smooth L 1 ) loss, background classifier (mutual entropy) loss, target plus background classifier (mutual entropy) loss. It can be used to train the detection architecture of the T kth target detection task.
为了减缓遗忘,可以采用知识蒸馏的方法,即将当前检测任务送入之前的目标检测器,获取包括预测框和物品分数向量等知识参数。特别地,我们把第T k个目标检测任务的图片送入第T k-1个目标检测框架。然后,我们可以获得前面任务直至T k-1的物品的知识(例如预测框和物品分数向量)。我们使用这些知识作为额外的监督(例如
Figure PCTCN2019130818-appb-000026
),它们可以有效地规范化第T k个检测架构相应的子模型从而保留先前学得的知识。
In order to alleviate forgetting, the method of knowledge distillation can be used, that is, the current detection task is sent to the previous target detector to obtain knowledge parameters including prediction boxes and item score vectors. In particular, we send the image of the T kth target detection task into the T k- 1th target detection frame. Then, we can obtain knowledge of the items up to T k-1 from the previous task (for example, the prediction box and the item score vector). We use this knowledge as additional supervision (e.g.
Figure PCTCN2019130818-appb-000026
), they can effectively normalize the corresponding sub-models of the T k- th detection framework to retain the previously learned knowledge.
因此,通过蒸馏方法自适应平衡了知识的遗忘以及新任务的更新关系。因此,我们的检测架构可以在获取新知识的同时维持之前学到的知识,从而实现终身学习。Therefore, the distillation method adaptively balances the forgetting of knowledge and the update of new tasks. Therefore, our detection architecture can maintain the previously learned knowledge while acquiring new knowledge, thereby achieving lifelong learning.
通过本申请所述的连续小样本图像的目标检测方法,可以快速的为小样本新任务进行部署,并且可以有效的维持之前学习的任务的性能。Through the target detection method of continuous small sample images described in this application, new tasks with small samples can be quickly deployed, and the performance of previously learned tasks can be effectively maintained.
应理解,上述实施例中各步骤的序号的大小并不意味着执行顺序的先后,各过程的执行顺序应以其功能和内在逻辑确定,而不应对本申请实施例的实施过程构成任何限定。It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the order of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiment of the present application.
图4为本申请实施例提供的一种连续小样本图像的目标检测装置的结构示意图,所述连续小样本图像的目标检测装置包括:FIG. 4 is a schematic structural diagram of a target detection device for continuous small sample images provided by an embodiment of the application. The target detection device for continuous small sample images includes:
解耦单元401,用于对目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器;The decoupling unit 401 is used to decouple the target detector, and decouple the target-background classifier into a background classifier and a target classifier;
拓印单元402,用于根据权重拓印策略以及利用先前检测任务框架从而获得参数初始化解耦后的单阶目标检测器;The rubbing unit 402 is used for rubbing the strategy according to the weight and using the previous detection task framework to obtain the single-stage target detector after parameter initialization and decoupling;
蒸馏单元403,用于通过知识蒸馏方法获取当前检测任务之前的监督参数,通过初始化后的解耦后的目标检测器结合所述监督参数对目标进行训练或检测。The distillation unit 403 is configured to obtain the supervision parameters before the current detection task through the knowledge distillation method, and train or detect the target through the decoupled target detector after initialization and the supervision parameters.
优选的,所述解耦单元用于:Preferably, the decoupling unit is used for:
对单阶目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器,得到背景分类器的分数向量为:The single-stage target detector is decoupled, and the target-background classifier is decoupled into a background classifier and a target classifier. The score vector of the background classifier is obtained as:
Figure PCTCN2019130818-appb-000027
Figure PCTCN2019130818-appb-000027
得到目标分类器的分数向量为:The score vector of the target classifier is:
Figure PCTCN2019130818-appb-000028
Figure PCTCN2019130818-appb-000028
其中,
Figure PCTCN2019130818-appb-000029
是背景分类器经过归一化指数函数处理之前的预测向量,p obj/p bg是目标/背景的概率,
Figure PCTCN2019130818-appb-000030
是目标分类器经过归一化指数函数处理之前的预测向量,q i是属于第i个目标的概率i=1,2,...,C。
among them,
Figure PCTCN2019130818-appb-000029
Is the prediction vector before the background classifier is processed by the normalized exponential function, p obj /p bg is the probability of the target/background,
Figure PCTCN2019130818-appb-000030
Is the prediction vector before the target classifier is processed by the normalized exponential function, and q i is the probability of belonging to the i-th target i=1, 2, ..., C.
图4所述连续小样本图像的目标检测装置,与图1所述的连续小样本图像的目标检测方法对应。The target detection device for continuous small sample images described in FIG. 4 corresponds to the target detection method for continuous small sample images described in FIG. 1.
图5是本申请一实施例提供的连续小样本图像的目标检测设备的示意图。如图5所示,该实施例的连续小样本图像的目标检测设备5包括:处理器50、 存储器51以及存储在所述存储器51中并可在所述处理器50上运行的计算机程序52,例如连续小样本图像的目标检测程序。所述处理器50执行所述计算机程序52时实现上述各个连续小样本图像的目标检测方法实施例中的步骤。或者,所述处理器50执行所述计算机程序52时实现上述各装置实施例中各模块/单元的功能。FIG. 5 is a schematic diagram of a target detection device for continuous small sample images provided by an embodiment of the present application. As shown in FIG. 5, the target detection device 5 for continuous small sample images of this embodiment includes a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and running on the processor 50, For example, the target detection program of continuous small sample images. When the processor 50 executes the computer program 52, the steps in the foregoing embodiments of the target detection method for each continuous small sample image are implemented. Alternatively, when the processor 50 executes the computer program 52, the functions of the modules/units in the foregoing device embodiments are realized.
示例性的,所述计算机程序52可以被分割成一个或多个模块/单元,所述一个或者多个模块/单元被存储在所述存储器51中,并由所述处理器50执行,以完成本申请。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段,该指令段用于描述所述计算机程序52在所述连续小样本图像的目标检测设备5中的执行过程。例如,所述计算机程序52可以被分割成:Exemplarily, the computer program 52 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 51 and executed by the processor 50 to complete This application. The one or more modules/units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 52 in the target detection device 5 of the continuous small sample image . For example, the computer program 52 can be divided into:
解耦单元,用于对目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器;The decoupling unit is used to decouple the target detector and decouple the target-background classifier into a background classifier and a target classifier;
拓印单元,用于根据权重拓印策略以及利用先前检测任务框架从而获得参数初始化解耦后的单阶目标检测器;The rubbing unit is used for rubbing the strategy according to the weight and using the previous detection task framework to obtain the single-stage target detector after parameter initialization and decoupling;
蒸馏单元,用于通过知识蒸馏方法获取当前检测任务之前的监督参数,通过初始化后的解耦后的目标检测器结合所述监督参数对目标进行训练或检测。The distillation unit is used to obtain the supervision parameters before the current detection task through the knowledge distillation method, and train or detect the target through the decoupled target detector after initialization and the supervision parameters.
所述连续小样本图像的目标检测设备5可以是桌上型计算机、笔记本、掌上电脑及云端服务器等计算设备。所述连续小样本图像的目标检测设备可包括,但不仅限于,处理器50、存储器51。本领域技术人员可以理解,图5仅仅是连续小样本图像的目标检测设备5的示例,并不构成对连续小样本图像的目标检测设备5的限定,可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件,例如所述连续小样本图像的目标检测设备还可以包括输 入输出设备、网络接入设备、总线等。The target detection device 5 of the continuous small sample image may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The target detection device of the continuous small sample image may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art can understand that FIG. 5 is only an example of the target detection device 5 for continuous small sample images, and does not constitute a limitation on the target detection device 5 for continuous small sample images, and may include more or less than that shown in the figure. Components, or a combination of some components, or different components, for example, the target detection device of the continuous small sample image may also include input and output devices, network access devices, buses, and the like.
所称处理器50可以是中央处理单元(Central Processing Unit,CPU),还可以是其他通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现成可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。The so-called processor 50 may be a central processing unit (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
所述存储器51可以是所述连续小样本图像的目标检测设备5的内部存储单元,例如连续小样本图像的目标检测设备5的硬盘或内存。所述存储器51也可以是所述连续小样本图像的目标检测设备5的外部存储设备,例如所述连续小样本图像的目标检测设备5上配备的插接式硬盘,智能存储卡(Smart Media Card,SMC),安全数字(Secure Digital,SD)卡,闪存卡(Flash Card)等。进一步地,所述存储器51还可以既包括所述连续小样本图像的目标检测设备5的内部存储单元也包括外部存储设备。所述存储器51用于存储所述计算机程序以及所述连续小样本图像的目标检测设备所需的其他程序和数据。所述存储器51还可以用于暂时地存储已经输出或者将要输出的数据。The memory 51 may be an internal storage unit of the target detection device 5 for continuous small sample images, such as a hard disk or a memory of the target detection device 5 for continuous small sample images. The memory 51 may also be an external storage device of the target detection device 5 of the continuous small sample image, for example, a plug-in hard disk equipped on the target detection device 5 of the continuous small sample image, or a smart memory card (SmartMedia Card). ,SMC), Secure Digital (SD) card, Flash Card, etc. Further, the memory 51 may also include both the internal storage unit of the target detection device 5 of the continuous small sample image and an external storage device. The memory 51 is used to store the computer program and other programs and data required by the target detection device of the continuous small sample image. The memory 51 can also be used to temporarily store data that has been output or will be output.
所属领域的技术人员可以清楚地了解到,为了描述的方便和简洁,仅以上述各功能单元、模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能单元、模块完成,即将所述装置的内部结构划分成不同的功能单元或模块,以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中,上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。另外,各功能单元、模 块的具体名称也只是为了便于相互区分,并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。Those skilled in the art can clearly understand that for the convenience and conciseness of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned functions can be allocated to different functional units and modules as required. Module completion means dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist alone physically, or two or more units can be integrated into one unit. The above-mentioned integrated units can be hardware-based Formal realization can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of this application. For the specific working process of the units and modules in the foregoing system, reference may be made to the corresponding process in the foregoing method embodiment, which is not repeated here.
在上述实施例中,对各个实施例的描述都各有侧重,某个实施例中没有详述或记载的部分,可以参见其它实施例的相关描述。In the foregoing embodiments, the description of each embodiment has its own focus. For parts that are not detailed or recorded in a certain embodiment, reference may be made to related descriptions of other embodiments.
本领域普通技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。A person of ordinary skill in the art may be aware that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.
在本申请所提供的实施例中,应该理解到,所揭露的装置/终端设备和方法,可以通过其它的方式实现。例如,以上所描述的装置/终端设备实施例仅仅是示意性的,例如,所述模块或单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口,装置或单元的间接耦合或通讯连接,可以是电性,机械或其它的形式。In the embodiments provided in this application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the device/terminal device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units. Or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元 中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
所述集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请实现上述实施例方法中的全部或部分流程,也可以通过计算机程序来指令相关的硬件来完成,所述的计算机程序可存储于一计算机可读存储介质中,该计算机程序在被处理器执行时,可实现上述各个方法实施例的步骤。其中,所述计算机程序包括计算机程序代码,所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括:能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM,Read-Only Memory)、随机存取存储器(RAM,Random Access Memory)、电载波信号、电信信号以及软件分发介质等。需要说明的是,所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减,例如在某些司法管辖区,根据立法和专利实践,计算机可读介质不包括是电载波信号和电信信号。If the integrated module/unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, this application implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media. It should be noted that the content contained in the computer-readable medium can be appropriately added or deleted in accordance with the requirements of the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to the legislation and patent practice, the computer-readable medium Does not include electrical carrier signals and telecommunication signals.
以上所述实施例仅用以说明本申请的技术方案,而非对其限制;尽管参照前述实施例对本申请进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围,均应包含在本申请的保护范围之内。The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of protection of this application.

Claims (10)

  1. 一种连续小样本图像的目标检测方法,其特征在于,所述小样本图像的目标检测方法包括:A target detection method for continuous small sample images is characterized in that the target detection method for small sample images includes:
    对目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器;Decouple the target detector, and decouple the target-background classifier into a background classifier and a target classifier;
    根据权重拓印策略以及利用先前检测任务框架从而获得参数初始化解耦后的单阶目标检测器;According to the weighted rubbing strategy and using the previous detection task framework to obtain the single-order target detector after parameter initialization and decoupling;
    通过知识蒸馏方法获取当前检测任务之前的监督参数,通过初始化后的解耦后的目标检测器结合所述监督参数对目标进行训练或检测。The supervised parameters before the current detection task are obtained by the knowledge distillation method, and the target is trained or detected by combining the supervised parameters with the decoupled target detector after initialization.
  2. 根据权利要求1所述的连续小样本图像的目标检测方法,其特征在于,所述对目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器的步骤包括:The target detection method for continuous small sample images according to claim 1, wherein the step of decoupling the target detector and decoupling the target-background classifier into a background classifier and a target classifier comprises:
    对单阶目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器,得到背景分类器的分数向量为:The single-stage target detector is decoupled, and the target-background classifier is decoupled into a background classifier and a target classifier. The score vector of the background classifier is obtained as:
    Figure PCTCN2019130818-appb-100001
    Figure PCTCN2019130818-appb-100001
    得到目标分类器的分数向量为:The score vector of the target classifier is:
    Figure PCTCN2019130818-appb-100002
    Figure PCTCN2019130818-appb-100002
    其中,
    Figure PCTCN2019130818-appb-100003
    是背景分类器经过归一化指数函数处理之前的预测向量,p obj/p bg是目标/背景的概率,
    Figure PCTCN2019130818-appb-100004
    是目标分类器经过归一化指数函数处理之前的预测向量,q i是属于第i个目标的概率i=1,2,...,C。
    among them,
    Figure PCTCN2019130818-appb-100003
    Is the prediction vector before the background classifier is processed by the normalized exponential function, p obj /p bg is the probability of the target/background,
    Figure PCTCN2019130818-appb-100004
    Is the prediction vector before the target classifier is processed by the normalized exponential function, and q i is the probability of belonging to the i-th target i=1, 2, ..., C.
  3. 根据权利要求1所述的连续小样本图像的目标检测方法,其特征在于, 所述根据权重拓印策略以及利用先前检测任务框架从而获得参数初始化解耦后的单阶目标检测器的步骤包括:The target detection method for continuous small-sample images according to claim 1, wherein the step of using a weighted rubbing strategy and using a previous detection task framework to obtain a single-stage target detector after parameter initialization and decoupling comprises:
    获取与检测任务中的目标类别;Obtain and detect the target category in the task;
    根据所述目标类别,在源域中获取多个包含所述目标类别的先验框;According to the target category, obtain a plurality of a priori boxes containing the target category in the source domain;
    根据所获取的多个源域中的先验框分别计算所述检测任务中的目标类别的预测向量;Respectively calculating the prediction vector of the target category in the detection task according to the acquired a priori boxes in the multiple source domains;
    对多个所述预测向量求和取平均后进行归一化处理,得到目标类别对应的参数向量。The multiple prediction vectors are summed and averaged and then normalized to obtain a parameter vector corresponding to the target category.
  4. 根据权利要求3所述的连续小样本图像的目标检测方法,其特征在于,当所述检测任务中包括多个目标类别时,根据多个目标类别所对应的参数向量,得到目标分类器的初始化参数矩阵。The target detection method for continuous small-sample images according to claim 3, wherein when the detection task includes multiple target categories, the initialization of the target classifier is obtained according to the parameter vectors corresponding to the multiple target categories Parameter matrix.
  5. 根据权利要求1所述的连续小样本图像的目标检测方法,其特征在于,所述根据权重拓印策略以及利用先前检测任务框架从而获得参数初始化解耦后的单阶目标检测器的步骤包括:The target detection method for continuous small-sample images according to claim 1, wherein the step of using a weighted rubbing strategy and using a previous detection task framework to obtain a single-stage target detector after parameter initialization and decoupling comprises:
    根据当前检测任务T k之前的检测任务T k-1所使用的主干参数、边界框回归参数、背景分类器参数初始化当前检测任务的主干、边界框回归和背景分类器。 Initialize the backbone, bounding box regression and background classifiers of the current detection task according to the backbone parameters, bounding box regression parameters, and background classifier parameters used in the detection task T k-1 before the current detection task T k .
  6. 根据权利要求1所述的连续小样本图像的目标检测方法,其特征在于,所述通过知识蒸馏方法获取当前检测任务之前的监督参数,通过初始化后的解耦单阶目标检测器结合所述监督参数对目标进行训练的步骤包括:The target detection method for continuous small-sample images according to claim 1, wherein the supervised parameter before the current detection task is obtained by the knowledge distillation method, and the decoupled single-stage target detector after initialization is combined with the supervised The steps of parameter training on the target include:
    根据公式计算当前目标检测任务T k的总损失函数: Calculate the total loss function of the current target detection task T k according to the formula:
    Figure PCTCN2019130818-appb-100005
    Figure PCTCN2019130818-appb-100005
    根据所述总损失函数规范当前检测任务的目标检测器,其中,主要损失
    Figure PCTCN2019130818-appb-100006
    包括边界框回归损失,背景分类器损失,目标加背景分类器损失,将当前检测任务的图片送入当前检测任务之前的目标检测器获得监督参数
    Figure PCTCN2019130818-appb-100007
    λ为权重系数。
    Standardize the target detector of the current detection task according to the total loss function, where the main loss
    Figure PCTCN2019130818-appb-100006
    Including bounding box regression loss, background classifier loss, target plus background classifier loss, send the image of the current detection task to the target detector before the current detection task to obtain the supervision parameters
    Figure PCTCN2019130818-appb-100007
    λ is the weight coefficient.
  7. 一种连续小样本图像的目标检测装置,其特征在于,所述连续小样本图像的目标检测装置包括:A target detection device for continuous small-sample images is characterized in that the target detection device for continuous small-sample images includes:
    解耦单元,用于对目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器;The decoupling unit is used to decouple the target detector and decouple the target-background classifier into a background classifier and a target classifier;
    拓印单元,用于根据权重拓印策略以及利用先前检测任务框架从而获得参数初始化解耦后的单阶目标检测器;The rubbing unit is used for rubbing the strategy according to the weight and using the previous detection task framework to obtain the single-stage target detector after parameter initialization and decoupling;
    蒸馏单元,用于通过知识蒸馏方法获取当前检测任务之前的监督参数,通过初始化后的解耦后的目标检测器结合所述监督参数对目标进行训练或检测。The distillation unit is used to obtain the supervision parameters before the current detection task through the knowledge distillation method, and train or detect the target through the decoupled target detector after initialization and the supervision parameters.
  8. 根据权利要求7所述的连续小样本图像的目标检测装置,其特征在于,所述解耦单元用于:The target detection device for continuous small-sample images according to claim 7, wherein the decoupling unit is used for:
    对单阶目标检测器进行解耦,将目标-背景分类器解耦成背景分类器和目标分类器,得到背景分类器的分数向量为:The single-stage target detector is decoupled, and the target-background classifier is decoupled into a background classifier and a target classifier. The score vector of the background classifier is obtained as:
    Figure PCTCN2019130818-appb-100008
    Figure PCTCN2019130818-appb-100008
    得到目标分类器的分数向量为:The score vector of the target classifier is:
    Figure PCTCN2019130818-appb-100009
    Figure PCTCN2019130818-appb-100009
    其中,
    Figure PCTCN2019130818-appb-100010
    是背景分类器经过归一化指数函数处理之前的预测 向量,p obj/p bg是目标/背景的概率,
    Figure PCTCN2019130818-appb-100011
    是目标分类器经过归一化指数函数处理之前的预测向量,q i是属于第i个目标的概率i=1,2,...,C。
    among them,
    Figure PCTCN2019130818-appb-100010
    Is the prediction vector before the background classifier is processed by the normalized exponential function, p obj /p bg is the probability of the target/background,
    Figure PCTCN2019130818-appb-100011
    Is the prediction vector before the target classifier is processed by the normalized exponential function, and q i is the probability of belonging to the i-th target i=1, 2, ..., C.
  9. 一种连续小样本图像的目标检测设备,包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序,其特征在于,所述处理器执行所述计算机程序时实现如权利要求1至6任一项所述连续小样本图像的目标检测方法的步骤。A target detection device for continuous small sample images, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program when the computer program is executed. The steps of realizing the target detection method of continuous small sample images according to any one of claims 1 to 6.
  10. 一种计算机可读存储介质,所述计算机可读存储介质存储有计算机程序,其特征在于,所述计算机程序被处理器执行时实现如权利要求1至6任一项所述连续小样本图像的目标检测方法的步骤。A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the continuous small sample image according to any one of claims 1 to 6 is realized The steps of the target detection method.
PCT/CN2019/130818 2019-03-15 2019-12-31 Target detection method, device and apparatus for continuous small sample images WO2020186887A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910199257.7 2019-03-15
CN201910199257.7A CN110033026B (en) 2019-03-15 2019-03-15 Target detection method, device and equipment for continuous small sample images

Publications (1)

Publication Number Publication Date
WO2020186887A1 true WO2020186887A1 (en) 2020-09-24

Family

ID=67236129

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/130818 WO2020186887A1 (en) 2019-03-15 2019-12-31 Target detection method, device and apparatus for continuous small sample images

Country Status (2)

Country Link
CN (1) CN110033026B (en)
WO (1) WO2020186887A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112465111A (en) * 2020-11-17 2021-03-09 大连理工大学 Three-dimensional voxel image segmentation method based on knowledge distillation and countertraining
CN112661013A (en) * 2020-12-17 2021-04-16 北京航天自动控制研究所 Automatic wharf bridge crane left lock pad detection method and system
CN113222020A (en) * 2021-05-13 2021-08-06 中国科学院计算技术研究所厦门数据智能研究院 Domain increment learning method based on data conversion and knowledge distillation
CN113378936A (en) * 2021-06-11 2021-09-10 长沙军民先进技术研究有限公司 Small sample target detection method based on fast RCNN

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110033026B (en) * 2019-03-15 2021-04-02 深圳先进技术研究院 Target detection method, device and equipment for continuous small sample images
CN111461191B (en) * 2020-03-25 2024-01-23 杭州跨视科技有限公司 Method and device for determining image sample set for model training and electronic equipment
CN111932561A (en) * 2020-09-21 2020-11-13 深圳大学 Real-time enteroscopy image segmentation method and device based on integrated knowledge distillation
CN112633406A (en) * 2020-12-31 2021-04-09 天津大学 Knowledge distillation-based few-sample target detection method
CN113743180B (en) * 2021-05-06 2024-05-07 西安电子科技大学 CNNKD-based radar HRRP small sample target identification method
CN113822373B (en) * 2021-10-27 2023-09-15 南京大学 Image classification model training method based on integration and knowledge distillation
CN116152258B (en) * 2023-04-23 2023-07-04 广东电网有限责任公司佛山供电局 Transmission line insulator defect detection method, device, equipment and medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090030864A1 (en) * 2002-03-11 2009-01-29 International Business Machines Corporation Method for constructing segmentation-based predictive models
CN103473549A (en) * 2013-09-23 2013-12-25 北京智诺英特科技有限公司 Image target detecting method and device
CN108960266A (en) * 2017-05-22 2018-12-07 阿里巴巴集团控股有限公司 Image object detection method and device
CN110033026A (en) * 2019-03-15 2019-07-19 深圳先进技术研究院 A kind of object detection method, device and the equipment of continuous small sample image

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7729531B2 (en) * 2006-09-19 2010-06-01 Microsoft Corporation Identifying repeated-structure elements in images
CN104732248B (en) * 2015-03-24 2017-11-21 电子科技大学 Human body target detection method based on Omega shape facilities
DE102015205524B4 (en) * 2015-03-26 2020-11-05 Mando Corporation Method and device for classifying an object, for example a traffic sign, in an image
CN108229658B (en) * 2017-12-27 2020-06-12 深圳先进技术研究院 Method and device for realizing object detector based on limited samples
CN109447943B (en) * 2018-09-21 2020-08-14 中国科学院深圳先进技术研究院 Target detection method, system and terminal equipment

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090030864A1 (en) * 2002-03-11 2009-01-29 International Business Machines Corporation Method for constructing segmentation-based predictive models
CN103473549A (en) * 2013-09-23 2013-12-25 北京智诺英特科技有限公司 Image target detecting method and device
CN108960266A (en) * 2017-05-22 2018-12-07 阿里巴巴集团控股有限公司 Image object detection method and device
CN110033026A (en) * 2019-03-15 2019-07-19 深圳先进技术研究院 A kind of object detection method, device and the equipment of continuous small sample image

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112465111A (en) * 2020-11-17 2021-03-09 大连理工大学 Three-dimensional voxel image segmentation method based on knowledge distillation and countertraining
CN112661013A (en) * 2020-12-17 2021-04-16 北京航天自动控制研究所 Automatic wharf bridge crane left lock pad detection method and system
CN113222020A (en) * 2021-05-13 2021-08-06 中国科学院计算技术研究所厦门数据智能研究院 Domain increment learning method based on data conversion and knowledge distillation
CN113378936A (en) * 2021-06-11 2021-09-10 长沙军民先进技术研究有限公司 Small sample target detection method based on fast RCNN
CN113378936B (en) * 2021-06-11 2024-03-08 长沙军民先进技术研究有限公司 Faster RCNN-based few-sample target detection method

Also Published As

Publication number Publication date
CN110033026B (en) 2021-04-02
CN110033026A (en) 2019-07-19

Similar Documents

Publication Publication Date Title
WO2020186887A1 (en) Target detection method, device and apparatus for continuous small sample images
WO2021238281A1 (en) Neural network training method, image classification system, and related device
WO2018108129A1 (en) Method and apparatus for use in identifying object type, and electronic device
WO2022213465A1 (en) Neural network-based image recognition method and apparatus, electronic device, and medium
WO2022037541A1 (en) Image processing model training method and apparatus, device, and storage medium
WO2021098618A1 (en) Data classification method and apparatus, terminal device and readable storage medium
CN111814620A (en) Face image quality evaluation model establishing method, optimization method, medium and device
WO2022141858A1 (en) Pedestrian detection method and apparatus, electronic device, and storage medium
WO2022028147A1 (en) Image classification model training method and apparatus, computer device, and storage medium
CN114494784A (en) Deep learning model training method, image processing method and object recognition method
JP2017527013A (en) Adaptive characterization as a service
WO2021238586A1 (en) Training method and apparatus, device, and computer readable storage medium
EP4343616A1 (en) Image classification method, model training method, device, storage medium, and computer program
CN113920382B (en) Cross-domain image classification method based on class consistency structured learning and related device
CN110619349A (en) Plant image classification method and device
WO2023241385A1 (en) Model transferring method and apparatus, and electronic device
US11847599B1 (en) Computing system for automated evaluation of process workflows
US20230021551A1 (en) Using training images and scaled training images to train an image segmentation model
CN112989869B (en) Optimization method, device, equipment and storage medium of face quality detection model
CN114360053A (en) Action recognition method, terminal and storage medium
CN114821140A (en) Image clustering method based on Manhattan distance, terminal device and storage medium
CN113609018A (en) Test method, training method, device, apparatus, medium, and program product
CN113807407A (en) Target detection model training method, model performance detection method and device
Anggoro et al. Classification of Solo Batik patterns using deep learning convolutional neural networks algorithm
WO2023231355A1 (en) Image recognition method and apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19919854

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS (EPO FORM 1205A DATED 09.11.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19919854

Country of ref document: EP

Kind code of ref document: A1