CN115082775B - Super-resolution Enhanced Small Target Detection Method Based on Image Blocking - Google Patents

Super-resolution Enhanced Small Target Detection Method Based on Image Blocking Download PDF

Info

Publication number
CN115082775B
CN115082775B CN202210888803.XA CN202210888803A CN115082775B CN 115082775 B CN115082775 B CN 115082775B CN 202210888803 A CN202210888803 A CN 202210888803A CN 115082775 B CN115082775 B CN 115082775B
Authority
CN
China
Prior art keywords
image
input image
small target
block
height
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210888803.XA
Other languages
Chinese (zh)
Other versions
CN115082775A (en
Inventor
杨明浩
黄雷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Automation of Chinese Academy of Science
Original Assignee
Institute of Automation of Chinese Academy of Science
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Automation of Chinese Academy of Science filed Critical Institute of Automation of Chinese Academy of Science
Priority to CN202210888803.XA priority Critical patent/CN115082775B/en
Publication of CN115082775A publication Critical patent/CN115082775A/en
Application granted granted Critical
Publication of CN115082775B publication Critical patent/CN115082775B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/40—Filling a planar surface by adding surface attributes, e.g. colour or texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention belongs to the field of target detection, and particularly relates to a super-resolution enhanced small target detection method, a system and equipment based on image blocking, aiming at solving the problem of low detection accuracy of the existing small target detection method. The method comprises the following steps: acquiring a scene image to be subjected to small target detection as an input image; calculating the width and height of a standard block when an input image is blocked; obtaining the step length of the input image in the horizontal direction and the vertical direction; filling the input image, and blocking the filled input image to obtain each image block of the input image after blocking; and performing image enhancement on each obtained image block by adopting a pre-trained super-resolution model, inputting the trained target detection model after the image block is enhanced, obtaining a rectangular area corresponding to a small target object in each image block in the input image, and performing regression and non-maximum suppression processing to obtain a detection result. The invention improves the accuracy of small target detection.

Description

基于图像分块的超分辨率增强小目标检测方法Super-resolution Enhanced Small Target Detection Method Based on Image Blocking

技术领域technical field

本发明属于目标检测领域，具体涉及一种基于图像分块的超分辨率增强小目标检测方法、系统、设备。The invention belongs to the field of target detection, and in particular relates to a super-resolution enhanced small target detection method, system and equipment based on image segmentation.

背景技术Background technique

在计算机视觉任务中，小目标物体检测和语义分割一直是一个公认的难题，相比常规目标检测，小目标的检测准确率往往只有普通目标的50%左右。MS COCO数据集将面积小于32*32的物体或目标相对原图小于10%认为是小物体，且在包罗COCO等公共数据集合，小目标的数目更多，以COCO为例41%比例的小对象、34%比例的中等规格对象 、24% 比例的大型对象，这些数据集也来源于生活中，在实际应用中，摄像头捕捉大量的小目标，通常由于小目标的检测难度较大，这些小目标处理方式往往是不做处理的，这不可避免地损失了关键的信息。在目标检测的许多方面，如机场驱鸟、卫星图像目标检测、汽车零部件检测等方面都涉及小目标检测，由于目前在小目标检测场景没有特别有效的方法，在这些应用场景中，主要以人工干预为主，最终小目标检测的结果不仅准确率低而且耗时耗力，因此为了应对这样高难度的应用场景。基于此，本发明提出一种基于图像分块的超分辨率增强小目标检测方法，简单来说在训练阶段采用适当的策略增强训练小目标检测算法模型，预测阶段采用合适比例将图像分块、增强、检测，核心思想是通过图像分块将小目标转化为正常目标大小，并且保证图像块比例接近于目标检测算法标注输入大小，避免因输入图像归一化带来的图像畸变和小目标信息损失，从而达到提升小目标检测的准确率和效率的效果。In computer vision tasks, small target object detection and semantic segmentation have always been a recognized problem. Compared with conventional target detection, the detection accuracy of small targets is often only about 50% of that of ordinary targets. The MS COCO data set regards objects with an area smaller than 32*32 or objects that are less than 10% of the original image as small objects, and in public data sets including COCO, the number of small objects is more, taking COCO as an example 41% of the small objects Objects, 34% of medium-sized objects, and 24% of large-scale objects. These data sets also come from real life. In practical applications, cameras capture a large number of small objects. Usually, it is difficult to detect small objects. The target processing method is often not processed, which inevitably loses key information. Many aspects of target detection, such as airport bird repelling, satellite image target detection, auto parts detection, etc., involve small target detection. Since there is currently no particularly effective method for small target detection scenarios, in these application scenarios, mainly based on Manual intervention is the main method, and the final result of small target detection is not only low in accuracy but also time-consuming and labor-intensive. Therefore, in order to deal with such difficult application scenarios. Based on this, the present invention proposes a super-resolution enhanced small target detection method based on image segmentation. Simply speaking, in the training phase, an appropriate strategy is used to enhance the training small target detection algorithm model, and in the prediction phase, the image is divided into blocks, Enhancement and detection, the core idea is to convert small targets into normal target sizes through image blocks, and ensure that the image block ratio is close to the target detection algorithm to mark the input size, avoiding image distortion and small target information caused by normalization of input images Loss, so as to achieve the effect of improving the accuracy and efficiency of small target detection.

发明内容Contents of the invention

为了解决现有技术中的上述问题，即为了解决现有的小目标检测方法检测准确率较低的问题，本发明第一方面，提出了一种基于图像分块的超分辨率增强小目标检测方法，该方法包括：In order to solve the above problems in the prior art, that is, in order to solve the problem of low detection accuracy of the existing small target detection method, the first aspect of the present invention proposes a super-resolution enhanced small target detection based on image segmentation method, which includes:

S100，获取待进行小目标检测的场景图像，作为输入图像；S100, acquiring a scene image to be subjected to small target detection as an input image;

S200，获取预构建的目标检测模型其在训练时训练样本中小目标物体的平均宽高，并结合所述目标检测模型设定输入的宽高，计算所述输入图像分块时标准块的宽高；S200, obtain the average width and height of the small target objects in the training samples of the pre-built target detection model, and set the input width and height in combination with the target detection model, and calculate the width and height of the standard block when the input image is divided into blocks ;

Figure 643157DEST_PATH_IMAGE001
Figure 643157DEST_PATH_IMAGE001

Figure 919418DEST_PATH_IMAGE002
Figure 919418DEST_PATH_IMAGE002

其中，

Figure 356215DEST_PATH_IMAGE003
、
Figure 48752DEST_PATH_IMAGE004
表示训练样本中小目标物体的平均宽、高，
Figure 698039DEST_PATH_IMAGE005
、
Figure 422282DEST_PATH_IMAGE006
表示所述目标 检测模型设定输入的宽、高，
Figure 826718DEST_PATH_IMAGE007
、
Figure 878988DEST_PATH_IMAGE008
表示所述输入图像分块时标准块的宽、高，
Figure 456600DEST_PATH_IMAGE009
表示设定 的第一百分比数值； in,
Figure 356215DEST_PATH_IMAGE003
,
Figure 48752DEST_PATH_IMAGE004
Indicates the average width and height of small target objects in the training samples,
Figure 698039DEST_PATH_IMAGE005
,
Figure 422282DEST_PATH_IMAGE006
Indicates the width and height of the target detection model setting input,
Figure 826718DEST_PATH_IMAGE007
,
Figure 878988DEST_PATH_IMAGE008
Indicates the width and height of the standard block when the input image is divided into blocks,
Figure 456600DEST_PATH_IMAGE009
Indicates the set first percentage value;

S300，分别根据输入图像分块时标准块的宽减去设定水平方向的重叠值、输入图像分块时标准块的高减去设定垂直方向的重叠值，得到所述输入图像在水平方向、垂直方向上分块的步长；S300. According to the width of the standard block when the input image is divided into blocks minus the overlap value set in the horizontal direction, and the height of the standard block when the input image is divided into blocks minus the overlap value set in the vertical direction, obtain the input image in the horizontal direction , the block step size in the vertical direction;

S400，结合S300得到的输入图像在水平方向、垂直方向上分块的步长，对所述输入图像进行填充，并按照卷积的方式对填充后的输入图像进行分块，得到所述输入图像分块后各图像块及各图像块的起始坐标在所述输入图像中的坐标；S400, combining the horizontal and vertical block step sizes of the input image obtained in S300, filling the input image, and performing convolution on the filled input image to obtain the input image Coordinates of each image block and the starting coordinates of each image block in the input image after being divided into blocks;

S500，采用预训练的超分别率模型对S400得到的各图像块进行图像增强，增强后，输入训练好的目标检测模型，得到所述输入图像中各图像块中小目标物体对应的矩形区域，并进行回归、非极大值抑制处理，进而得到检测结果。S500, using the pre-trained super-resolution model to perform image enhancement on each image block obtained in S400, after the enhancement, input the trained target detection model to obtain a rectangular area corresponding to the small target object in each image block in the input image, and Perform regression and non-maximum suppression processing to obtain the detection results.

在一些优选的实施方式中，所述目标检测模型，其训练方法为：In some preferred embodiments, the training method of the target detection model is:

A100，获取训练样本，构建训练集；所述训练样本包括场景样本图像及其对应的小目标物体检测结果的真值标签；A100, obtaining a training sample and constructing a training set; the training sample includes a scene sample image and a true value label of a corresponding small target object detection result;

A200，通过预训练的超分辨率网络对所述场景样本图像进行图像增强，得到第一增强图像；A200, performing image enhancement on the scene sample image through a pre-trained super-resolution network to obtain a first enhanced image;

A300，获取所述第一增强图像中各小目标物体所在区域对应的矩形框，并对矩形框在上下左右方向对应的分辨率进行设定百分比的增强，将增强后的第一增强图像作为第二增强图像；A300. Obtain the rectangular frame corresponding to the area where each small target object is located in the first enhanced image, and enhance the resolution corresponding to the rectangular frame in the up, down, left, and right directions by a set percentage, and use the enhanced first enhanced image as the first enhanced image. Two enhanced images;

A400，将所述第二增强图像输入预构建的目标检测模型，获取所述场景样本图像中各小目标物体的预测检测结果；A400. Input the second enhanced image into a pre-built target detection model, and obtain the predicted detection results of each small target object in the scene sample image;

A500，基于所述预测检测结果、小目标物体检测结果的真值标签，计算损失值，更新所述目标检测模型的模型参数；A500, calculating a loss value based on the predicted detection result and the true value label of the small target object detection result, and updating the model parameters of the target detection model;

A600，循环A100-A500，直至得到训练好的目标检测模型。A600, cycle A100-A500 until a trained target detection model is obtained.

在一些优选的实施方式中，对矩形框在上下左右方向对应的分辨率进行设定百分比的增强，将增强后的第一增强图像作为第二增强图像，其方法为：In some preferred embodiments, the resolution corresponding to the rectangular frame in the up, down, left, and right directions is enhanced by a set percentage, and the enhanced first enhanced image is used as the second enhanced image. The method is as follows:

Figure 707452DEST_PATH_IMAGE010
Figure 707452DEST_PATH_IMAGE010

Figure 486053DEST_PATH_IMAGE011
Figure 486053DEST_PATH_IMAGE011

Figure 150252DEST_PATH_IMAGE012
Figure 150252DEST_PATH_IMAGE012

Figure 406921DEST_PATH_IMAGE013
Figure 406921DEST_PATH_IMAGE013

其中，

Figure 512280DEST_PATH_IMAGE014
、
Figure 586416DEST_PATH_IMAGE015
分别表示第一增强图像中各小目标物体所在区域对应的矩形框左 上角坐标，
Figure 82119DEST_PATH_IMAGE016
、
Figure 532692DEST_PATH_IMAGE017
分别表示第一增强图像中各小目标物体所在区域对应的矩形框的宽、 高，
Figure 695820DEST_PATH_IMAGE018
、
Figure 612960DEST_PATH_IMAGE019
表示第一增强图像的宽、高，
Figure 986173DEST_PATH_IMAGE020
、
Figure 850224DEST_PATH_IMAGE021
分别表示第一增强图像中各小目标物体所 在区域对应的矩形框增强后左上角坐标，
Figure 664596DEST_PATH_IMAGE022
、
Figure 346113DEST_PATH_IMAGE023
分别表示第一增强图像中各小目标物体 所在区域对应的矩形框增强后的宽、高，
Figure 816409DEST_PATH_IMAGE024
、
Figure 346135DEST_PATH_IMAGE025
表示设定的第二百分比、第三百分比对应的 数值。 in,
Figure 512280DEST_PATH_IMAGE014
,
Figure 586416DEST_PATH_IMAGE015
Respectively represent the coordinates of the upper left corner of the rectangular frame corresponding to the area where each small target object is located in the first enhanced image,
Figure 82119DEST_PATH_IMAGE016
,
Figure 532692DEST_PATH_IMAGE017
Respectively represent the width and height of the rectangular frame corresponding to the area where each small target object is located in the first enhanced image,
Figure 695820DEST_PATH_IMAGE018
,
Figure 612960DEST_PATH_IMAGE019
Indicates the width and height of the first enhanced image,
Figure 986173DEST_PATH_IMAGE020
,
Figure 850224DEST_PATH_IMAGE021
Respectively represent the coordinates of the upper left corner of the rectangular frame corresponding to the area where each small target object is located in the first enhanced image after enhancement,
Figure 664596DEST_PATH_IMAGE022
,
Figure 346113DEST_PATH_IMAGE023
respectively represent the enhanced width and height of the rectangular frame corresponding to the area where each small target object is located in the first enhanced image,
Figure 816409DEST_PATH_IMAGE024
,
Figure 346135DEST_PATH_IMAGE025
Indicates the values corresponding to the set second percentage and third percentage.

在一些优选的实施方式中，对所述输入图像进行填充，其方法为：In some preferred implementation manners, the input image is filled, and the method is as follows:

Figure 280593DEST_PATH_IMAGE026
Figure 280593DEST_PATH_IMAGE026

Figure 742798DEST_PATH_IMAGE027
Figure 742798DEST_PATH_IMAGE027

其中，

Figure 825023DEST_PATH_IMAGE028
、
Figure 93194DEST_PATH_IMAGE029
表示输入图像的宽、高，
Figure 85421DEST_PATH_IMAGE030
、
Figure 577582DEST_PATH_IMAGE031
表示输入图像右侧、下侧填充 的宽度，
Figure 288049DEST_PATH_IMAGE032
、
Figure 94331DEST_PATH_IMAGE033
表示输入图像在水平方向、垂直方向上分块的步长。 in,
Figure 825023DEST_PATH_IMAGE028
,
Figure 93194DEST_PATH_IMAGE029
Indicates the width and height of the input image,
Figure 85421DEST_PATH_IMAGE030
,
Figure 577582DEST_PATH_IMAGE031
Indicates the width of the right and bottom padding of the input image,
Figure 288049DEST_PATH_IMAGE032
,
Figure 94331DEST_PATH_IMAGE033
Indicates the step size of the input image in the horizontal direction and vertical direction.

在一些优选的实施方式中，按照卷积的方式对填充后的输入图像进行分块，其方法为：In some preferred embodiments, the padded input image is divided into blocks in a convolutional manner, and the method is as follows:

Figure 65698DEST_PATH_IMAGE034
Figure 65698DEST_PATH_IMAGE034

Figure 135285DEST_PATH_IMAGE035
Figure 135285DEST_PATH_IMAGE035

其中，

Figure 192103DEST_PATH_IMAGE036
、
Figure 739759DEST_PATH_IMAGE037
分别表示输入图像在水平方向、竖直方向上分块的块数。 in,
Figure 192103DEST_PATH_IMAGE036
,
Figure 739759DEST_PATH_IMAGE037
Respectively represent the number of blocks in the horizontal direction and vertical direction of the input image.

在一些优选的实施方式中，对输入图像中各图像块中小目标物体对应的矩形区域进行回归处理，其方法为：In some preferred embodiments, regression processing is performed on the rectangular area corresponding to the small target object in each image block in the input image, and the method is as follows:

获取输入图像中各图像块中小目标物体对应的矩形区域的坐标：（

Figure 503315DEST_PATH_IMAGE038
，
Figure 602858DEST_PATH_IMAGE039
，
Figure 22338DEST_PATH_IMAGE040
，
Figure 498319DEST_PATH_IMAGE041
），其中，
Figure 116382DEST_PATH_IMAGE038
，
Figure 262193DEST_PATH_IMAGE039
为输入图像中各图像块中小目标物体对应的矩形区域左 上角的横坐标、纵坐标，
Figure 28024DEST_PATH_IMAGE040
，
Figure 183061DEST_PATH_IMAGE041
为输入图像中各图像块中小目标物体对应的矩形 区域的宽、高，
Figure 655631DEST_PATH_IMAGE042
表示图像块的编号，1<=
Figure 365486DEST_PATH_IMAGE043
<=n*m； Obtain the coordinates of the rectangular area corresponding to the small target object in each image block in the input image: (
Figure 503315DEST_PATH_IMAGE038
,
Figure 602858DEST_PATH_IMAGE039
,
Figure 22338DEST_PATH_IMAGE040
,
Figure 498319DEST_PATH_IMAGE041
),in,
Figure 116382DEST_PATH_IMAGE038
,
Figure 262193DEST_PATH_IMAGE039
is the abscissa and ordinate of the upper left corner of the rectangular area corresponding to the small target object in each image block in the input image,
Figure 28024DEST_PATH_IMAGE040
,
Figure 183061DEST_PATH_IMAGE041
is the width and height of the rectangular area corresponding to the small target object in each image block in the input image,
Figure 655631DEST_PATH_IMAGE042
Indicates the number of the image block, 1<=
Figure 365486DEST_PATH_IMAGE043
<=n*m;

对（

Figure 493979DEST_PATH_IMAGE038
，
Figure 311762DEST_PATH_IMAGE039
，
Figure 904417DEST_PATH_IMAGE040
，
Figure 657610DEST_PATH_IMAGE041
）进行回归处理，得到回归后的坐标为（
Figure 132453DEST_PATH_IMAGE044
，
Figure 629294DEST_PATH_IMAGE045
，
Figure 76456DEST_PATH_IMAGE046
，
Figure 125183DEST_PATH_IMAGE047
）： right(
Figure 493979DEST_PATH_IMAGE038
,
Figure 311762DEST_PATH_IMAGE039
,
Figure 904417DEST_PATH_IMAGE040
,
Figure 657610DEST_PATH_IMAGE041
) for regression processing, and the coordinates after regression are (
Figure 132453DEST_PATH_IMAGE044
,
Figure 629294DEST_PATH_IMAGE045
,
Figure 76456DEST_PATH_IMAGE046
,
Figure 125183DEST_PATH_IMAGE047
):

Figure 228268DEST_PATH_IMAGE048
Figure 228268DEST_PATH_IMAGE048

Figure 387854DEST_PATH_IMAGE049
Figure 387854DEST_PATH_IMAGE049

Figure 689522DEST_PATH_IMAGE050
Figure 689522DEST_PATH_IMAGE050

Figure 518938DEST_PATH_IMAGE051
Figure 518938DEST_PATH_IMAGE051

其中，

Figure 233953DEST_PATH_IMAGE044
、
Figure 869334DEST_PATH_IMAGE045
分别表示输入图像中各图像块中小目标物体对应的矩形区域回归后 左上角的横坐标、纵坐标，
Figure 494350DEST_PATH_IMAGE046
、
Figure 884880DEST_PATH_IMAGE047
分别表示输入图像中各图像块中小目标物体 对应的矩形区域回归后的宽、高，N表示输入图像中小目标物体的宽高相对于回归后的小目 标物体宽高的缩放比例。 in,
Figure 233953DEST_PATH_IMAGE044
,
Figure 869334DEST_PATH_IMAGE045
Respectively represent the abscissa and ordinate of the upper left corner of the rectangular area corresponding to the small target object in each image block in the input image after regression,
Figure 494350DEST_PATH_IMAGE046
,
Figure 884880DEST_PATH_IMAGE047
Respectively represent the width and height of the rectangular area corresponding to the small target object in each image block in the input image after regression, and N represents the scaling ratio of the width and height of the small target object in the input image relative to the width and height of the small target object after regression.

本发明的第二方面，提出了一种基于图像分块的超分辨率增强小目标检测系统，包括：图像获取模块、分块标准尺寸计算模块、分块步长计算模块、图像分块模块、检测结果获取模块；In the second aspect of the present invention, a super-resolution enhanced small target detection system based on image segmentation is proposed, including: an image acquisition module, a block standard size calculation module, a block step calculation module, an image block module, Detection result acquisition module;

所述图像获取模块，配置为获取待进行小目标检测的场景图像，作为输入图像；The image acquisition module is configured to acquire a scene image to be detected for a small target as an input image;

所述分块标准尺寸计算模块，配置为获取预构建的目标检测模型其在训练时训练样本中小目标物体的平均宽高，并结合所述目标检测模型设定输入的宽高，计算所述输入图像分块时标准块的宽高；The block standard size calculation module is configured to obtain the average width and height of the small target objects in the training samples of the pre-built target detection model, and set the input width and height in combination with the target detection model to calculate the input The width and height of the standard block when the image is divided into blocks;

Figure 696979DEST_PATH_IMAGE001
Figure 696979DEST_PATH_IMAGE001

Figure 198367DEST_PATH_IMAGE002
Figure 198367DEST_PATH_IMAGE002

其中，

Figure 474628DEST_PATH_IMAGE003
、
Figure 645846DEST_PATH_IMAGE004
表示训练样本中小目标物体的平均宽、高，
Figure 350102DEST_PATH_IMAGE005
、
Figure 327285DEST_PATH_IMAGE006
表示所述目标 检测模型设定输入的宽、高，
Figure 926894DEST_PATH_IMAGE007
、
Figure 393647DEST_PATH_IMAGE008
表示所述输入图像分块时标准块的宽、高，
Figure 180337DEST_PATH_IMAGE009
表示设定 的第一百分比数值； in,
Figure 474628DEST_PATH_IMAGE003
,
Figure 645846DEST_PATH_IMAGE004
Indicates the average width and height of small target objects in the training samples,
Figure 350102DEST_PATH_IMAGE005
,
Figure 327285DEST_PATH_IMAGE006
Indicates the width and height of the target detection model setting input,
Figure 926894DEST_PATH_IMAGE007
,
Figure 393647DEST_PATH_IMAGE008
Indicates the width and height of the standard block when the input image is divided into blocks,
Figure 180337DEST_PATH_IMAGE009
Indicates the set first percentage value;

所述分块步长计算模块，配置为分别根据输入图像分块时标准块的宽减去设定水平方向的重叠值、输入图像分块时标准块的高减去设定垂直方向的重叠值，得到所述输入图像在水平方向、垂直方向上分块的步长；The block step calculation module is configured to subtract the overlapping value in the horizontal direction from the width of the standard block when the input image is divided into blocks, and subtract the overlapping value in the vertical direction from the height of the standard block when the input image is divided into blocks. , to obtain the block step size of the input image in the horizontal direction and the vertical direction;

所述图像分块模块，配置为结合所述分块步长计算模块得到的输入图像在水平方向、垂直方向上分块的步长，对所述输入图像进行填充，并按照卷积的方式对填充后的输入图像进行分块，得到所述输入图像分块后各图像块及各图像块的起始坐标在所述输入图像中的坐标；The image block module is configured to fill the input image in combination with the step size of the input image obtained by the block step calculation module in the horizontal direction and the vertical direction, and perform convolution on the input image. The filled input image is divided into blocks to obtain the coordinates of each image block and the starting coordinates of each image block in the input image after the input image is divided into blocks;

所述检测结果获取模块，配置为采用预训练的超分别率模型对所述图像分块模块得到的各图像块进行图像增强，增强后，输入训练好的目标检测模型，得到所述输入图像中各图像块中小目标物体对应的矩形区域，并进行回归、非极大值抑制处理，进而得到检测结果。The detection result acquisition module is configured to use a pre-trained super-resolution model to perform image enhancement on each image block obtained by the image block module, after enhancement, input the trained target detection model to obtain the input image The rectangular area corresponding to the small target object in each image block is processed by regression and non-maximum value suppression, and then the detection result is obtained.

本发明的第三方面，提出了一种电子设备，包括：至少一个处理器；以及与至少一个所述处理器通信连接的存储器；其中，所述存储器存储有可被所述处理器执行的指令，所述指令用于被所述处理器执行以实现上述的基于图像分块的超分辨率增强小目标检测方法。In a third aspect of the present invention, an electronic device is proposed, including: at least one processor; and a memory connected to at least one processor in communication; wherein, the memory stores instructions executable by the processor , the instructions are used to be executed by the processor to implement the above image block-based super-resolution enhanced small target detection method.

本发明的第四方面，提出了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机指令，所述计算机指令用于被所述计算机执行以实现上述的基于图像分块的超分辨率增强小目标检测方法。According to the fourth aspect of the present invention, a computer-readable storage medium is proposed, the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to realize the above-mentioned image segmentation-based super Resolution-enhanced small object detection methods.

本发明的有益效果：Beneficial effects of the present invention:

本发明提高了小目标检测的准确率。The invention improves the accuracy rate of small target detection.

1）在目标模型检测模型训练阶段，训练超分辨率增强模型，并对目标检测算法的训练样本进行增强，增强后，采用基于上下文信息的方式对场景样本图像中的小目标进行标注，进而对目标模型检测模型，提升模型的检测精度；1) In the training stage of the target model detection model, train the super-resolution enhancement model, and enhance the training samples of the target detection algorithm. Target model detection model to improve the detection accuracy of the model;

2）在实际的检测过程中采用合适比例将图像分块、增强（使图像块中的小目标更加清晰，特征更明显）、检测，核心思想是通过图像分块将小目标转化为正常目标大小，并且保证图像块比例接近于目标检测算法标注输入大小，避免因输入图像归一化带来的图像畸变和小目标信息损失，从而达到小目标特征能快速被深度神经网络提取的效果，以实现各类目标检测算法对小目标检测效果的提升。2) In the actual detection process, the image is divided into blocks, enhanced (to make the small target in the image block clearer and the features more obvious) and detected with an appropriate ratio. The core idea is to convert the small target into a normal target size through image block , and ensure that the image block ratio is close to the target detection algorithm label input size, to avoid image distortion and small target information loss caused by the normalization of the input image, so as to achieve the effect that the small target features can be quickly extracted by the deep neural network, in order to achieve Various target detection algorithms improve the detection effect of small targets.

附图说明Description of drawings

通过阅读参照以下附图所做的对非限制性实施例所做的详细描述，本申请的其他特征、目的和优点将会变得更明显。Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings.

图1是本发明一种实施例的基于图像分块的超分辨率增强小目标检测方法的流程示意图；Fig. 1 is a schematic flow chart of an image block-based super-resolution enhanced small target detection method according to an embodiment of the present invention;

图2是本发明一种实施例的基于图像分块的超分辨率增强小目标检测系统的框架示意图；Fig. 2 is a schematic framework diagram of a super-resolution enhanced small target detection system based on image segmentation according to an embodiment of the present invention;

图3是本发明一种实施例的目标检测模型的训练以及检测过程的流程示意图；Fig. 3 is a schematic flow chart of the training and detection process of the target detection model of an embodiment of the present invention;

图4是本发明一种实施例的基于图像分块的超分辨率增强小目标检测方法进行小目标检测的流程示意图；Fig. 4 is a schematic flow diagram of a small target detection method based on an image block-based super-resolution enhanced small target detection method according to an embodiment of the present invention;

图5是本发明一种实施例的适于用来实现本申请实施例的电子设备的计算机系统的结构示意图。FIG. 5 is a schematic structural diagram of a computer system suitable for implementing the electronic device of the embodiment of the present application according to an embodiment of the present invention.

具体实施方式detailed description

为使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than Full examples. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

下面结合附图和实施例对本申请作进一步的详细说明。可以理解的是，此处所描述的具体实施例仅用于解释相关发明，而非对该发明的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与有关发明相关的部分。The application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain related inventions, not to limit the invention. It should also be noted that, for the convenience of description, only the parts related to the related invention are shown in the drawings.

需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

本发明的基于图像分块的超分辨率增强小目标检测方法，如图1所示，包括以下步骤：The super-resolution enhanced small target detection method based on image segmentation of the present invention, as shown in Figure 1, comprises the following steps:

S100，获取待进行小目标检测的场景图像，作为输入图像；S100, acquiring a scene image to be subjected to small target detection as an input image;

S200，获取预构建的目标检测模型其在训练时训练样本中小目标物体的平均宽高，并结合所述目标检测模型设定输入的宽高，计算所述输入图像分块时标准块的宽高；S200, obtain the average width and height of the small target objects in the training samples of the pre-built target detection model, and set the input width and height in combination with the target detection model, and calculate the width and height of the standard block when the input image is divided into blocks ;

Figure 961212DEST_PATH_IMAGE001
Figure 961212DEST_PATH_IMAGE001

Figure 539961DEST_PATH_IMAGE002
Figure 539961DEST_PATH_IMAGE002

其中，

Figure 52981DEST_PATH_IMAGE003
、
Figure 451602DEST_PATH_IMAGE004
表示训练样本中小目标物体的平均宽、高，
Figure 770588DEST_PATH_IMAGE005
、
Figure 79209DEST_PATH_IMAGE006
表示所述目标 检测模型设定输入的宽、高，
Figure 887765DEST_PATH_IMAGE007
、
Figure 711365DEST_PATH_IMAGE008
表示所述输入图像分块时标准块的宽、高，
Figure 771725DEST_PATH_IMAGE009
表示设定 的第一百分比数值； in,
Figure 52981DEST_PATH_IMAGE003
,
Figure 451602DEST_PATH_IMAGE004
Indicates the average width and height of small target objects in the training samples,
Figure 770588DEST_PATH_IMAGE005
,
Figure 79209DEST_PATH_IMAGE006
Indicates the width and height of the target detection model setting input,
Figure 887765DEST_PATH_IMAGE007
,
Figure 711365DEST_PATH_IMAGE008
Indicates the width and height of the standard block when the input image is divided into blocks,
Figure 771725DEST_PATH_IMAGE009
Indicates the set first percentage value;

S300，分别根据输入图像分块时标准块的宽减去设定水平方向的重叠值、输入图像分块时标准块的高减去设定垂直方向的重叠值，得到所述输入图像在水平方向、垂直方向上分块的步长；S300. According to the width of the standard block when the input image is divided into blocks minus the overlap value set in the horizontal direction, and the height of the standard block when the input image is divided into blocks minus the overlap value set in the vertical direction, obtain the input image in the horizontal direction , the block step size in the vertical direction;

S400，结合S300得到的输入图像在水平方向、垂直方向上分块的步长，对所述输入图像进行填充，并按照卷积的方式对填充后的输入图像进行分块，得到所述输入图像分块后各图像块及各图像块的起始坐标在所述输入图像中的坐标；S400, combining the horizontal and vertical block step sizes of the input image obtained in S300, filling the input image, and performing convolution on the filled input image to obtain the input image Coordinates of each image block and the starting coordinates of each image block in the input image after being divided into blocks;

S500，采用预训练的超分别率模型对S400得到的各图像块进行图像增强，增强后，输入训练好的目标检测模型，得到所述输入图像中各图像块中小目标物体对应的矩形区域，并进行回归、非极大值抑制处理，进而得到检测结果。S500, using the pre-trained super-resolution model to perform image enhancement on each image block obtained in S400, after the enhancement, input the trained target detection model to obtain a rectangular area corresponding to the small target object in each image block in the input image, and Perform regression and non-maximum suppression processing to obtain the detection results.

为了更清晰地对本发明基于图像分块的超分辨率增强小目标检测方法进行说明，下面结合附图3对本发明方法一种实施例中各步骤进行展开详述。In order to describe the image block-based super-resolution enhanced small target detection method of the present invention more clearly, the steps in an embodiment of the method of the present invention will be described in detail below in conjunction with FIG. 3 .

在下述实施例中，先对目标检测模型的训练过程进行详述，再对通过一种基于图像分块的超分辨率增强小目标检测方法进行小目标检测的过程进行详述。In the following embodiments, the training process of the target detection model is first described in detail, and then the process of small target detection through a super-resolution enhanced small target detection method based on image segmentation is described in detail.

1、目标检测模型的训练过程1. The training process of the target detection model

A100，获取训练样本，构建训练集；所述训练样本包括场景样本图像及其对应的小目标物体检测结果的真值标签；A100, obtaining a training sample and constructing a training set; the training sample includes a scene sample image and a true value label of a corresponding small target object detection result;

在本实施例中，获取场景样本图像及其对应的小目标物体检测结果的真值标签，作为训练样本，构建训练集。In this embodiment, the scene sample images and the corresponding ground truth labels of the small target object detection results are obtained as training samples to construct a training set.

A200，通过预训练的超分辨率网络对所述场景样本图像进行图像增强，得到第一增强图像；A200, performing image enhancement on the scene sample image through a pre-trained super-resolution network to obtain a first enhanced image;

在本实施例中，先采用公共数据集或自建数据集训练超分辨率网络，设置超分辨倍数为N（即输入图像中小目标物体的宽高相对于回归后的小目标物体宽高的缩放比例），这样，对场景样本图像的分辨率在水平和竖直方向分别有N倍的增强。超分别率网络预训练后，对场景样本图像进行图像增强，将增强后的场景样本图像作为第一增强图像。具体如公式（1）（2）所示：In this embodiment, the public data set or self-built data set is used to train the super-resolution network first, and the super-resolution multiple is set to N (that is, the width and height of the small target object in the input image is scaled relative to the width and height of the small target object after regression In this way, the resolution of the scene sample image is enhanced by N times in the horizontal and vertical directions respectively. After the pre-training of the super-resolution network, image enhancement is performed on the scene sample image, and the enhanced scene sample image is used as the first enhanced image. Specifically, as shown in formula (1) (2):

Figure 325066DEST_PATH_IMAGE052
（1）
Figure 325066DEST_PATH_IMAGE052
(1)

Figure 179889DEST_PATH_IMAGE053
（2）
Figure 179889DEST_PATH_IMAGE053
(2)

其中，

Figure 553102DEST_PATH_IMAGE054
、
Figure 213890DEST_PATH_IMAGE055
为训练集中的场景样本图像的宽、高，
Figure 497104DEST_PATH_IMAGE018
、
Figure 647463DEST_PATH_IMAGE019
为第一增强图像的宽、高。 in,
Figure 553102DEST_PATH_IMAGE054
,
Figure 213890DEST_PATH_IMAGE055
is the width and height of the scene sample images in the training set,
Figure 497104DEST_PATH_IMAGE018
,
Figure 647463DEST_PATH_IMAGE019
are the width and height of the first enhanced image.

A300，获取所述第一增强图像中各小目标物体所在区域对应的矩形框，并对矩形框在上下左右方向对应的分辨率进行设定百分比的增强，将增强后的第一增强图像作为第二增强图像；A300. Obtain the rectangular frame corresponding to the area where each small target object is located in the first enhanced image, and enhance the resolution corresponding to the rectangular frame in the up, down, left, and right directions by a set percentage, and use the enhanced first enhanced image as the first enhanced image. Two enhanced images;

在本实施例中，采用引入Context（上下文）信息的方式标注第一增强图像中的小 目标，具体方法是：在第一增强图像上框出小目标物体所在区域的矩形框，矩形框的左上角 坐标为

Figure 180075DEST_PATH_IMAGE056
，宽、高为
Figure 847817DEST_PATH_IMAGE016
、
Figure 113101DEST_PATH_IMAGE017
。 In this embodiment, the small target in the first enhanced image is marked by introducing Context information. The specific method is: frame a rectangular frame in the area where the small target object is located on the first enhanced image, and the upper left of the rectangular frame The angular coordinates are
Figure 180075DEST_PATH_IMAGE056
, the width and height are
Figure 847817DEST_PATH_IMAGE016
,
Figure 113101DEST_PATH_IMAGE017
.

将小目标物体所在区域的矩形框在上下左右分辨率往外浮动（即增强）

Figure 309727DEST_PATH_IMAGE057
（
Figure 657532DEST_PATH_IMAGE058
在 本发明中一般优选取15~20）。浮动后的矩形左上角坐标为
Figure 925702DEST_PATH_IMAGE059
，宽、高为
Figure 917929DEST_PATH_IMAGE022
、
Figure 410090DEST_PATH_IMAGE023
，但 浮动后的小目标物体不宜超过原图像的
Figure 120557DEST_PATH_IMAGE060
（
Figure 926839DEST_PATH_IMAGE025
在本发明中一般优选取15~25），而且不能越 界。具体如公式（3）、（4）、（5）、（6）所示： Float the rectangular frame of the area where the small target object is located in the upper, lower, left, and right resolutions (that is, enhance)
Figure 309727DEST_PATH_IMAGE057
(
Figure 657532DEST_PATH_IMAGE058
In the present invention, it is generally preferred to take 15~20). The coordinates of the upper left corner of the floating rectangle are
Figure 925702DEST_PATH_IMAGE059
, the width and height are
Figure 917929DEST_PATH_IMAGE022
,
Figure 410090DEST_PATH_IMAGE023
, but the floating small target object should not exceed the original image
Figure 120557DEST_PATH_IMAGE060
(
Figure 926839DEST_PATH_IMAGE025
In the present invention, it is generally preferred to take 15~25), and it cannot exceed the limit. Specifically, as shown in formulas (3), (4), (5), and (6):

Figure 632627DEST_PATH_IMAGE010
（3）
Figure 632627DEST_PATH_IMAGE010
(3)

Figure 702214DEST_PATH_IMAGE011
（4）
Figure 702214DEST_PATH_IMAGE011
(4)

Figure 431135DEST_PATH_IMAGE012
（5）
Figure 431135DEST_PATH_IMAGE012
(5)

Figure 369004DEST_PATH_IMAGE013
（6）
Figure 369004DEST_PATH_IMAGE013
(6)

其中，

Figure 804665DEST_PATH_IMAGE024
、
Figure 169787DEST_PATH_IMAGE025
表示设定的第二百分比、第三百分比对应的数值。 in,
Figure 804665DEST_PATH_IMAGE024
,
Figure 169787DEST_PATH_IMAGE025
Indicates the values corresponding to the set second percentage and third percentage.

A400，将所述第二增强图像输入预构建的目标检测模型，获取所述场景样本图像中各小目标物体的预测检测结果；A400. Input the second enhanced image into a pre-built target detection model, and obtain the predicted detection results of each small target object in the scene sample image;

在本实施例中，将第一增强图像中标记（即分辨率增强）的小目标物体即第二增强图像，输入预构建的目标检测模型（本发明中优选设置为SSD模型），得到场景样本图像中各小目标物体的预测检测结果。In this embodiment, the small target object marked (i.e. resolution enhancement) in the first enhanced image, that is, the second enhanced image, is input into the pre-built target detection model (preferably set as the SSD model in the present invention), and the scene sample is obtained Predicted detection results for each small target object in the image.

A500，基于所述预测检测结果、小目标物体检测结果的真值标签，计算损失值，更新所述目标检测模型的模型参数；A500, calculating a loss value based on the predicted detection result and the true value label of the small target object detection result, and updating the model parameters of the target detection model;

在本实施例中，基于场景样本图像中各小目标物体的预测检测结果、结合小目标物体检测结果的真值标签，计算损失值，更新模型参数。In this embodiment, based on the predicted detection results of each small target object in the scene sample image and the true value label combined with the detection results of the small target object, the loss value is calculated and the model parameters are updated.

A600，循环A100-A500，直至得到训练好的目标检测模型。A600, cycle A100-A500 until a trained target detection model is obtained.

在本实施例中，循环对目标检测模型，直至得到训练好的目标检测模型。In this embodiment, the target detection model is cycled until a trained target detection model is obtained.

2、基于图像分块的超分辨率增强小目标检测方法，如图4所示2. Super-resolution enhanced small target detection method based on image segmentation, as shown in Figure 4

S100，获取待进行小目标检测的场景图像，作为输入图像；S100, acquiring a scene image to be subjected to small target detection as an input image;

在本实施例中，先获取待检测的场景图像。In this embodiment, the scene image to be detected is acquired first.

S200，获取预构建的目标检测模型其在训练时训练样本中小目标物体的平均宽高，并结合所述目标检测模型设定输入的宽高，计算所述输入图像分块时标准块的宽高；S200, obtain the average width and height of the small target objects in the training samples of the pre-built target detection model, and set the input width and height in combination with the target detection model, and calculate the width and height of the standard block when the input image is divided into blocks ;

在本实施例中，先获取预构建的目标检测模型其在训练时训练样本中小目标物体 的平均宽高，具体为：对训练集中标注的小目标物体设置整数编号

Figure 386005DEST_PATH_IMAGE061
（1<=
Figure 737352DEST_PATH_IMAGE061
<=
Figure 417732DEST_PATH_IMAGE062
），统计标注 的小目标物体的原始大小，并求其宽和高的平均值
Figure 829122DEST_PATH_IMAGE003
和
Figure 532636DEST_PATH_IMAGE004
，具体计算方法如下： In this embodiment, first obtain the average width and height of the small target objects in the training samples of the pre-built target detection model during training, specifically: set an integer number for the small target objects marked in the training set
Figure 386005DEST_PATH_IMAGE061
(1<=
Figure 737352DEST_PATH_IMAGE061
<=
Figure 417732DEST_PATH_IMAGE062
), calculate the original size of the marked small target object, and find the average of its width and height
Figure 829122DEST_PATH_IMAGE003
with
Figure 532636DEST_PATH_IMAGE004
, the specific calculation method is as follows:

Figure 812307DEST_PATH_IMAGE063
（7）
Figure 812307DEST_PATH_IMAGE063
(7)

Figure 818965DEST_PATH_IMAGE064
（8）
Figure 818965DEST_PATH_IMAGE064
(8)

其中，

Figure 401256DEST_PATH_IMAGE065
、
Figure 592066DEST_PATH_IMAGE065
表示第
Figure 409849DEST_PATH_IMAGE061
个小目标物体的宽、高。 in,
Figure 401256DEST_PATH_IMAGE065
,
Figure 592066DEST_PATH_IMAGE065
Indicates the first
Figure 409849DEST_PATH_IMAGE061
The width and height of a small target object.

然后基于目标检测模型其在训练时训练样本中小目标物体的原始宽高、平均宽高，并结合目标检测模型设定输入的宽高，计算所述输入图像分块时标准块的宽高，如公式（9）、（10）所示：Then based on the target detection model, the original width and height and average width and height of the small target objects in the training sample during training, and combined with the target detection model to set the input width and height, calculate the width and height of the standard block when the input image is divided into blocks, such as Formulas (9), (10) show:

Figure 940188DEST_PATH_IMAGE001
（9）
Figure 940188DEST_PATH_IMAGE001
(9)

Figure 552435DEST_PATH_IMAGE002
（10）
Figure 552435DEST_PATH_IMAGE002
(10)

其中，

Figure 230541DEST_PATH_IMAGE066
、
Figure 727381DEST_PATH_IMAGE004
表示训练样本中小目标物体的平均宽、高，
Figure 236860DEST_PATH_IMAGE005
、
Figure 160953DEST_PATH_IMAGE006
表示所述目标 检测模型设定输入的宽、高，即
Figure 60776DEST_PATH_IMAGE005
、
Figure 220362DEST_PATH_IMAGE006
与采用的目标检测模型有关，例如SSD模型(
Figure 725293DEST_PATH_IMAGE005
，
Figure 944922DEST_PATH_IMAGE067
=（300，300），如果采用目标检测模型适合任何比例的图像，那么
Figure 332041DEST_PATH_IMAGE005
、
Figure 170684DEST_PATH_IMAGE006
分别取256或 400，
Figure 654755DEST_PATH_IMAGE007
、
Figure 920651DEST_PATH_IMAGE008
表示所述输入图像分块时标准块的宽、高，
Figure 795066DEST_PATH_IMAGE009
表示设定的第一百分比数值，含 义为小目标物体尺寸相对原始图像尺寸的最大比例，本发明中优先设置为10%。 in,
Figure 230541DEST_PATH_IMAGE066
,
Figure 727381DEST_PATH_IMAGE004
Indicates the average width and height of small target objects in the training samples,
Figure 236860DEST_PATH_IMAGE005
,
Figure 160953DEST_PATH_IMAGE006
Indicates the width and height of the target detection model setting input, namely
Figure 60776DEST_PATH_IMAGE005
,
Figure 220362DEST_PATH_IMAGE006
It is related to the target detection model adopted, such as the SSD model (
Figure 725293DEST_PATH_IMAGE005
,
Figure 944922DEST_PATH_IMAGE067
= (300, 300), if the target detection model is suitable for any scale image, then
Figure 332041DEST_PATH_IMAGE005
,
Figure 170684DEST_PATH_IMAGE006
Take 256 or 400 respectively,
Figure 654755DEST_PATH_IMAGE007
,
Figure 920651DEST_PATH_IMAGE008
Indicates the width and height of the standard block when the input image is divided into blocks,
Figure 795066DEST_PATH_IMAGE009
Indicates the set first percentage value, which means the maximum ratio of the size of the small target object to the size of the original image. In the present invention, it is preferably set to 10%.

在上面公式中，10%是小目标物体相对原图最高比例，以这样的方式保证预测通过图像分块得到的子图像中，目标占比尽可能高于10%和图像块比例接近目标检测标准输入大小。In the above formula, 10% is the highest proportion of small target objects relative to the original image. In this way, it is guaranteed that in the sub-image obtained by predicting image blocks, the proportion of the target is as high as 10% and the proportion of the image block is close to the target detection standard. Enter a size.

S300，分别根据输入图像分块时标准块的宽减去设定水平方向的重叠值、输入图像分块时标准块的高减去设定垂直方向的重叠值，得到所述输入图像在水平方向、垂直方向上分块的步长；S300. According to the width of the standard block when the input image is divided into blocks minus the overlap value set in the horizontal direction, and the height of the standard block when the input image is divided into blocks minus the overlap value set in the vertical direction, obtain the input image in the horizontal direction , the block step size in the vertical direction;

在本实施例中，计算图像分块水平和垂直方向分块的步长

Figure 30875DEST_PATH_IMAGE032
和
Figure 510398DEST_PATH_IMAGE033
：采用重叠分 块的方法将预测图像（即场景图像）按S200标准块的宽高分块，设置水平方向上的重叠大小 （即重叠值）
Figure 743933DEST_PATH_IMAGE068
和竖直方向的重叠值
Figure 436470DEST_PATH_IMAGE069
，
Figure 351337DEST_PATH_IMAGE068
和
Figure 810000DEST_PATH_IMAGE069
一般设置为
Figure 214436DEST_PATH_IMAGE054
、
Figure 266706DEST_PATH_IMAGE004
的2~4倍，主要用于 保留切块缝隙间的小目标物体。 In this embodiment, calculate the step size of the horizontal and vertical block of the image block
Figure 30875DEST_PATH_IMAGE032
with
Figure 510398DEST_PATH_IMAGE033
: Divide the predicted image (i.e. scene image) into blocks according to the width and height of the S200 standard block by overlapping blocks, and set the overlap size in the horizontal direction (i.e. the overlap value)
Figure 743933DEST_PATH_IMAGE068
and vertical overlap values
Figure 436470DEST_PATH_IMAGE069
,
Figure 351337DEST_PATH_IMAGE068
with
Figure 810000DEST_PATH_IMAGE069
Generally set to
Figure 214436DEST_PATH_IMAGE054
,
Figure 266706DEST_PATH_IMAGE004
2~4 times of that, it is mainly used to keep the small target objects in the gap between the cutting blocks.

输入图像在水平方向分块的步长：

Figure 109897DEST_PATH_IMAGE070
； The step size of the input image in the horizontal direction:
Figure 109897DEST_PATH_IMAGE070
;

输入图像在垂直方向上分块的步长：

Figure 298433DEST_PATH_IMAGE071
。 The step size of the block in the vertical direction of the input image:
Figure 298433DEST_PATH_IMAGE071
.

S400，结合S300得到的输入图像在水平方向、垂直方向上分块的步长，对所述输入图像进行填充，并按照卷积的方式对填充后的输入图像进行分块，得到所述输入图像分块后各图像块及各图像块的起始坐标在所述输入图像中的坐标；S400, combining the horizontal and vertical block step sizes of the input image obtained in S300, filling the input image, and performing convolution on the filled input image to obtain the input image Coordinates of each image block and the starting coordinates of each image block in the input image after being divided into blocks;

采用S200和S300计算的参数按照从上至下，从左至右的方式均等分块，并不能保证预测图像被完全分块，在预测图像右侧和下侧可能会有残留部分无法被标准块覆盖到。The parameters calculated by S200 and S300 are equally divided into blocks from top to bottom and from left to right, which does not guarantee that the predicted image is completely divided into blocks, and there may be residual parts on the right and lower sides of the predicted image that cannot be divided into standard blocks. cover to.

在本实施例中，对宽和高为

Figure 873771DEST_PATH_IMAGE028
和
Figure 537970DEST_PATH_IMAGE029
的预测图像的右侧和下侧进行填充，右 侧和下侧填充宽度分别为
Figure 794639DEST_PATH_IMAGE030
和
Figure 227895DEST_PATH_IMAGE031
为： In this example, the width and height are
Figure 873771DEST_PATH_IMAGE028
with
Figure 537970DEST_PATH_IMAGE029
The right and bottom sides of the predicted image are filled, and the right and bottom padding widths are respectively
Figure 794639DEST_PATH_IMAGE030
with
Figure 227895DEST_PATH_IMAGE031
for:

Figure 974134DEST_PATH_IMAGE026
（11）
Figure 974134DEST_PATH_IMAGE026
(11)

Figure 996DEST_PATH_IMAGE072
（12）
Figure 996DEST_PATH_IMAGE072
(12)

采用S300获取的步长（

Figure 920410DEST_PATH_IMAGE032
，
Figure 83538DEST_PATH_IMAGE033
）按照卷积的方式对S400填充后的场景图像进行 分块。分块后水平方向分为
Figure 266258DEST_PATH_IMAGE036
块，竖直方向分为
Figure 373891DEST_PATH_IMAGE037
块。 Using the step size obtained by S300 (
Figure 920410DEST_PATH_IMAGE032
,
Figure 83538DEST_PATH_IMAGE033
) divides the scene image filled by S400 into blocks according to the convolution method. After partitioning, the horizontal direction is divided into
Figure 266258DEST_PATH_IMAGE036
blocks, vertically divided into
Figure 373891DEST_PATH_IMAGE037
Piece.

Figure 237942DEST_PATH_IMAGE034
（13）
Figure 237942DEST_PATH_IMAGE034
(13)

Figure 645790DEST_PATH_IMAGE035
（14）
Figure 645790DEST_PATH_IMAGE035
(14)

其中，

Figure 733831DEST_PATH_IMAGE036
、
Figure 204127DEST_PATH_IMAGE037
分别表示输入图像在水平方向、竖直方向上分块的块数。 in,
Figure 733831DEST_PATH_IMAGE036
,
Figure 204127DEST_PATH_IMAGE037
Respectively represent the number of blocks in the horizontal direction and vertical direction of the input image.

最后对图像分块得到的

Figure 987713DEST_PATH_IMAGE036
*
Figure 125434DEST_PATH_IMAGE037
块图像设置整数编号k(1<=k<=n*m)，并记录第k个图 像块起始点在原场景图像的坐标（
Figure 384377DEST_PATH_IMAGE073
，
Figure 466602DEST_PATH_IMAGE074
）。 Finally, the image is divided into blocks to obtain
Figure 987713DEST_PATH_IMAGE036
*
Figure 125434DEST_PATH_IMAGE037
The block image sets the integer number k (1<=k<=n*m), and records the coordinates of the starting point of the kth image block in the original scene image (
Figure 384377DEST_PATH_IMAGE073
,
Figure 466602DEST_PATH_IMAGE074
).

S500，采用预训练的超分别率模型对S400得到的各图像块进行图像增强，增强后，输入训练好的目标检测模型，得到所述输入图像中各图像块中小目标物体对应的矩形区域，并进行回归、非极大值抑制处理，进而得到检测结果。S500, using the pre-trained super-resolution model to perform image enhancement on each image block obtained in S400, after the enhancement, input the trained target detection model to obtain a rectangular area corresponding to the small target object in each image block in the input image, and Perform regression and non-maximum suppression processing to obtain the detection results.

在本实施例中，先采用超分辨率模型对编号为k的图像块逐一进行增强。增强后的图像分辨率与原图像块相比，宽和高分别增强了N倍。In this embodiment, the image block numbered k is enhanced one by one by using the super-resolution model. Compared with the original image block, the enhanced image resolution is enhanced by N times in width and height respectively.

然后采用训练好的目标检测模型对增强后的各图像块逐一进行检测，预测到第k 个图像块的小目标物体的矩形区域为（

Figure 672456DEST_PATH_IMAGE038
，
Figure 789316DEST_PATH_IMAGE039
，
Figure 484740DEST_PATH_IMAGE040
，
Figure 195207DEST_PATH_IMAGE041
），
Figure 798226DEST_PATH_IMAGE038
，
Figure 910539DEST_PATH_IMAGE039
为左上角横和 纵坐标，
Figure 511284DEST_PATH_IMAGE040
，
Figure 833681DEST_PATH_IMAGE041
为图像块中小目标物体宽和高。并预测得到的所有小目标物体的 坐标在原图上进行回归。回归后的坐标为（
Figure 115758DEST_PATH_IMAGE044
，
Figure 207211DEST_PATH_IMAGE045
，
Figure 244437DEST_PATH_IMAGE046
，
Figure 663917DEST_PATH_IMAGE047
）： Then use the trained target detection model to detect the enhanced image blocks one by one, and predict the rectangular area of the small target object of the kth image block as (
Figure 672456DEST_PATH_IMAGE038
,
Figure 789316DEST_PATH_IMAGE039
,
Figure 484740DEST_PATH_IMAGE040
,
Figure 195207DEST_PATH_IMAGE041
),
Figure 798226DEST_PATH_IMAGE038
,
Figure 910539DEST_PATH_IMAGE039
are the horizontal and vertical coordinates of the upper left corner,
Figure 511284DEST_PATH_IMAGE040
,
Figure 833681DEST_PATH_IMAGE041
It is the width and height of the small target object in the image block. And the predicted coordinates of all small target objects are regressed on the original image. The coordinates after regression are (
Figure 115758DEST_PATH_IMAGE044
,
Figure 207211DEST_PATH_IMAGE045
,
Figure 244437DEST_PATH_IMAGE046
,
Figure 663917DEST_PATH_IMAGE047
):

Figure 874319DEST_PATH_IMAGE048
（15）
Figure 874319DEST_PATH_IMAGE048
(15)

Figure 492382DEST_PATH_IMAGE049
（16）
Figure 492382DEST_PATH_IMAGE049
(16)

Figure 903772DEST_PATH_IMAGE050
（17）
Figure 903772DEST_PATH_IMAGE050
(17)

Figure 669602DEST_PATH_IMAGE051
（18）
Figure 669602DEST_PATH_IMAGE051
(18)

最后，采用非极大值抑制（NMS）的方法对回归后的目标进行合并，剔除第k（k<n*m）个图像块边缘部分与相邻图像块之间重叠部分同时检测到的目标，得到检测结果。Finally, the non-maximum suppression (NMS) method is used to merge the regressed targets, and the target detected at the same time as the edge part of the kth (k<n*m) image block and the overlap between the adjacent image blocks is eliminated , get the detection result.

本发明第二实施例的一种基于图像分块的超分辨率增强小目标检测系统，如图2所示，包括：图像获取模块100、分块标准尺寸计算模块200、分块步长计算模块300、图像分块模块400、检测结果获取模块500；A super-resolution enhanced small target detection system based on image segmentation according to the second embodiment of the present invention, as shown in FIG. 2 , includes: an image acquisition module 100, a block standard size calculation module 200, and a block step size calculation module 300, image block module 400, detection result acquisition module 500;

所述图像获取模块100，配置为获取待进行小目标检测的场景图像，作为输入图像；The image acquisition module 100 is configured to acquire a scene image to be detected for a small target as an input image;

所述分块标准尺寸计算模块200，配置为获取预构建的目标检测模型其在训练时训练样本中小目标物体的平均宽高，并结合所述目标检测模型设定输入的宽高，计算所述输入图像分块时标准块的宽高；The block standard size calculation module 200 is configured to obtain the average width and height of the small target objects in the training samples of the pre-built target detection model, and set the input width and height in combination with the target detection model, and calculate the The width and height of the standard block when the input image is divided into blocks;

Figure 559061DEST_PATH_IMAGE001
Figure 559061DEST_PATH_IMAGE001

Figure 297210DEST_PATH_IMAGE002
Figure 297210DEST_PATH_IMAGE002

其中，

Figure 7064DEST_PATH_IMAGE003
、
Figure 135557DEST_PATH_IMAGE004
表示训练样本中小目标物体的平均宽、高，
Figure 953341DEST_PATH_IMAGE005
、
Figure 280417DEST_PATH_IMAGE006
表示所述目标 检测模型设定输入的宽、高，
Figure 33609DEST_PATH_IMAGE007
、
Figure 774032DEST_PATH_IMAGE008
表示所述输入图像分块时标准块的宽、高，
Figure 67610DEST_PATH_IMAGE009
表示设定 的第一百分比数值； in,
Figure 7064DEST_PATH_IMAGE003
,
Figure 135557DEST_PATH_IMAGE004
Indicates the average width and height of small target objects in the training samples,
Figure 953341DEST_PATH_IMAGE005
,
Figure 280417DEST_PATH_IMAGE006
Indicates the width and height of the target detection model setting input,
Figure 33609DEST_PATH_IMAGE007
,
Figure 774032DEST_PATH_IMAGE008
Indicates the width and height of the standard block when the input image is divided into blocks,
Figure 67610DEST_PATH_IMAGE009
Indicates the set first percentage value;

所述分块步长计算模块300，配置为分别根据输入图像分块时标准块的宽减去设定水平方向的重叠值、输入图像分块时标准块的高减去设定垂直方向的重叠值，得到所述输入图像在水平方向、垂直方向上分块的步长；The block step calculation module 300 is configured to subtract the overlapping value in the horizontal direction from the width of the standard block when the input image is divided into blocks, and subtract the overlapping value in the vertical direction from the height of the standard block when the input image is divided into blocks. value to obtain the step size of the input image in the horizontal direction and the vertical direction;

所述图像分块模块400，配置为结合所述分块步长计算模块300得到的输入图像在水平方向、垂直方向上分块的步长，对所述输入图像进行填充，并按照卷积的方式对填充后的输入图像进行分块，得到所述输入图像分块后各图像块及各图像块的起始坐标在所述输入图像中的坐标；The image block module 400 is configured to fill the input image in combination with the block step size of the input image obtained by the block step calculation module 300 in the horizontal direction and the vertical direction, and fill the input image according to the convolution step size. The method divides the filled input image into blocks, and obtains the coordinates of each image block and the starting coordinates of each image block in the input image after the input image is divided into blocks;

所述检测结果获取模块500，配置为采用预训练的超分别率模型对所述图像分块模块400得到的各图像块进行图像增强，增强后，输入训练好的目标检测模型，得到所述输入图像中各图像块中小目标物体对应的矩形区域，并进行回归、非极大值抑制处理，进而得到检测结果。The detection result acquisition module 500 is configured to use a pre-trained super-resolution model to perform image enhancement on each image block obtained by the image block module 400, after enhancement, input the trained target detection model to obtain the input The rectangular area corresponding to the small target object in each image block in the image is processed by regression and non-maximum value suppression, and then the detection result is obtained.

需要说明的是，上述实施例提供的基于图像分块的超分辨率增强小目标检测系统，仅以上述各功能模块的划分进行举例说明，在实际应用中，可以根据需要而将上述功能分配由不同的功能模块来完成，即将本发明实施例中的模块或者步骤再分解或者组合，例如，上述实施例的模块可以合并为一个模块，也可以进一步拆分成多个子模块，以完成以上描述的全部或者部分功能。对于本发明实施例中涉及的模块、步骤的名称，仅仅是为了区分各个模块或者步骤，不视为对本发明的不当限定。It should be noted that the image block-based super-resolution enhanced small target detection system provided by the above embodiment is only illustrated by the division of the above-mentioned functional modules. In practical applications, the above-mentioned functions can be allocated by different functional modules, that is, to decompose or combine the modules or steps in the embodiments of the present invention. For example, the modules in the above embodiments can be combined into one module, or can be further split into multiple sub-modules to complete the above-described full or partial functionality. The names of the modules and steps involved in the embodiments of the present invention are only used to distinguish each module or step, and are not regarded as improperly limiting the present invention.

本发明第三实施例的一种电子设备，包括：至少一个处理器；以及与至少一个所述处理器通信连接的存储器；其中，所述存储器存储有可被所述处理器执行的指令，所述指令用于被所述处理器执行以实现上述的基于图像分块的超分辨率增强小目标检测方法。An electronic device according to a third embodiment of the present invention includes: at least one processor; and a memory communicatively connected to at least one of the processors; wherein, the memory stores instructions executable by the processor, so The above instructions are used to be executed by the processor to implement the above image block-based super-resolution enhanced small target detection method.

本发明第四实施例的一种计算机可读存储介质，所述计算机可读存储介质存储有计算机指令，所述计算机指令用于被所述计算机执行以实现上述的基于图像分块的超分辨率增强小目标检测方法。A computer-readable storage medium according to the fourth embodiment of the present invention, the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to realize the above-mentioned image block-based super-resolution Enhanced small object detection methods.

所述技术领域的技术人员可以清楚的了解到，为描述的方便和简洁，上述描述的电子设备、计算机可读存储介质的具体工作过程及有关说明，可以参考前述方法实例中的对应过程，在此不再赘述。Those skilled in the technical field can clearly understand that for the convenience and brevity of the description, the specific working process and related instructions of the electronic device and the computer-readable storage medium described above can refer to the corresponding process in the aforementioned method example. This will not be repeated here.

下面参考图5，其示出了适于用来实现本申请方法、系统、装置实施例的服务器的计算机系统的结构示意图。图5示出的服务器仅仅是一个示例，不应对本申请实施例的功能和使用范围带来任何限制。Referring now to FIG. 5 , it shows a schematic structural diagram of a server computer system suitable for implementing the method, system, and device embodiments of the present application. The server shown in FIG. 5 is only an example, and should not limit the functions and scope of use of this embodiment of the present application.

如图5所示，计算机系统包括中央处理单元(CPU，Central Processing Unit)501，其可以根据存储在只读存储器(ROM，Read Only Memory)502中的程序或者从存储部分508加载到随机访问存储器(RAM，Random Access Memory)503中的程序而执行各种适当的动作和处理。在RAM 503中，还存储有系统操作所需的各种程序和数据。CPU501、ROM 502以及RAM503通过总线504彼此相连。输入/输出(I/O，Input/Output)接口505也连接至总线504。As shown in Figure 5, the computer system includes a central processing unit (CPU, Central Processing Unit) 501, which can be stored in a program in a read only memory (ROM, Read Only Memory) 502 or loaded into a random access memory from a storage section 508 (RAM, Random Access Memory) 503 to execute various appropriate actions and processes. In RAM 503, various programs and data necessary for system operation are also stored. The CPU 501 , ROM 502 , and RAM 503 are connected to each other via a bus 504 . An input/output (I/O, Input/Output) interface 505 is also connected to the bus 504 .

以下部件连接至I/O接口505：包括键盘、鼠标等的输入部分506；包括诸如阴极射线管(CRT，Cathode Ray Tube)、液晶显示器(LCD，Liquid Crystal Display)等以及扬声器等的输出部分507；包括硬盘等的存储部分508；以及包括诸如LAN(局域网，Local AreaNetwork)卡、调制解调器等的网络接口卡的通讯部分509。通讯部分509经由诸如因特网的网络执行通信处理。驱动器510也根据需要连接至I/O接口505。可拆卸介质511，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器510上，以便于从其上读出的计算机程序根据需要被安装入存储部分508。The following components are connected to the I/O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT, Cathode Ray Tube), a liquid crystal display (LCD, Liquid Crystal Display), etc., and a speaker ; a storage section 508 including a hard disk or the like; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, or the like. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I/O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 510 as necessary so that a computer program read therefrom is installed into the storage section 508 as necessary.

特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通讯部分509从网络上被下载和安装，和/或从可拆卸介质511被安装。在该计算机程序被中央处理单元(CPU501执行时，执行本申请的方法中限定的上述功能。需要说明的是，本申请上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本申请中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本申请中，计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、电线、光缆、RF等等，或者上述的任意合适的组合。In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication portion 509 and/or installed from removable media 511 . When the computer program is executed by the central processing unit (CPU501), the above-mentioned functions defined in the method of the present application are executed. It should be noted that the computer-readable medium mentioned above in the present application can be a computer-readable signal medium or a computer-readable storage medium Or any combination of the above two. Computer-readable storage media can be, for example—but not limited to—electric, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any combination of the above. Computer More specific examples of readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable Read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this application, the computer-readable storage medium can be Is any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can be included in the baseband or propagated as part of a carrier wave A data signal, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media can also Any computer-readable medium, other than a computer-readable storage medium, that can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. Contained on a computer-readable medium Program code may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the above.

可以以一种或多种程序设计语言或其组合来编写用于执行本申请的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。Computer program code for performing the operations of the present application may be written in one or more programming languages or combinations thereof, including object-oriented programming languages—such as Java, Smalltalk, C++, and conventional Procedural Programming Language - such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider). Internet connection).

附图中的流程图和框图，图示了按照本申请各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.

术语“第一”、 “第二”等是用于区别类似的对象，而不是用于描述或表示特定的顺序或先后次序。The terms "first", "second", etc. are used to distinguish similar items, and are not used to describe or represent a specific order or sequence.

术语“包括”或者任何其它类似用语旨在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备/装置不仅包括那些要素，而且还包括没有明确列出的其它要素，或者还包括这些过程、方法、物品或者设备/装置所固有的要素。The term "comprising" or any other similar term is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus/apparatus comprising a set of elements includes not only those elements but also other elements not expressly listed, or Also included are elements inherent in these processes, methods, articles, or devices/devices.

至此，已经结合附图所示的优选实施方式描述了本发明的技术方案，但是，本领域技术人员容易理解的是，本发明的保护范围显然不局限于这些具体实施方式。在不偏离本发明的原理的前提下，本领域技术人员可以对相关技术特征作出等同的更改或替换，这些更改或替换之后的技术方案都将落入本发明的保护范围之内。So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings, but those skilled in the art will easily understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to relevant technical features, and the technical solutions after these changes or substitutions will all fall within the protection scope of the present invention.

Claims (7)

1. A super-resolution enhanced small target detection method based on image blocking is characterized by comprising the following steps:
s100, acquiring a scene image to be subjected to small target detection as an input image;
s200, acquiring the average width and height of small target objects in a training sample of a pre-constructed target detection model during training, setting the input width and height by combining the target detection model, and calculating the width and height of a standard block when the input image is blocked;
Figure FDA0003902738350000011
Figure FDA0003902738350000012
wherein Aw and Ah represent the average width and height of small target objects in a training sample, sw and Sh represent the width and height of the target detection model setting input, bw and Bh represent the width and height of a standard block when the input image is blocked, and q represents a set first percentage value;
s300, respectively subtracting a set horizontal direction overlapping value from the width of a standard block when the input image is blocked and subtracting a set vertical direction overlapping value from the height of the standard block when the input image is blocked to obtain the blocking step length of the input image in the horizontal direction and the vertical direction;
s400, combining the step length of the input image obtained in the step S300 in the horizontal direction and the vertical direction, filling the input image, and blocking the filled input image according to a convolution mode to obtain the coordinates of each image block and the initial coordinate of each image block in the input image after the input image is blocked;
s500, performing image enhancement on each image block obtained in the S400 by adopting a pre-trained super-resolution model, inputting a trained target detection model after the image enhancement, obtaining a rectangular region corresponding to a small target object in each image block in the input image, and performing regression and non-maximum suppression processing to further obtain a detection result;
wherein, the input image is filled, and the method comprises the following steps:
Pw＝Tw-(Pre w -Tw)mod Sw
Ph＝Th-(Pre h -Th)mod Sh
wherein, pre w 、Pre h Indicating the width and height of an input image, pw and Ph indicating the width of the right side and the lower side of the input image, tw and Th indicating the step size of the input image in the horizontal direction and the vertical direction;
partitioning the filled input image in a convolution mode, wherein the method comprises the following steps:
Figure FDA0003902738350000021
Figure FDA0003902738350000022
where n and m respectively represent the number of blocks of the input image partitioned in the horizontal direction and the vertical direction.
2. The method for detecting the super-resolution enhanced small target based on the image blocks as claimed in claim 1, wherein the training method of the target detection model is as follows:
a100, obtaining a training sample and constructing a training set; the training samples comprise scene sample images and truth value labels of small target object detection results corresponding to the scene sample images;
a200, performing image enhancement on the scene sample image through a pre-trained super-resolution network to obtain a first enhanced image;
a300, acquiring a rectangular frame corresponding to the area where each small target object is located in the first enhanced image, performing set percentage enhancement on the resolution corresponding to the rectangular frame in the up-down left-right direction, and taking the enhanced first enhanced image as a second enhanced image;
a400, inputting the second enhanced image into a pre-constructed target detection model, and obtaining a prediction detection result of each small target object in the scene sample image;
a500, calculating a loss value based on the predicted detection result and a true value label of the small target object detection result, and updating a model parameter of the target detection model;
and A600, circulating A100-A500 until a trained target detection model is obtained.
3. The method for detecting the super-resolution enhanced small target based on the image blocks as claimed in claim 2, wherein the resolution corresponding to the rectangular frame in the up, down, left and right directions is enhanced by a set percentage, and the enhanced first enhanced image is used as the second enhanced image, and the method comprises:
Nx＝MAX(Lx*(1-f％)，0)
Ny＝MAX(Ly*(1-f％)，0)
Nw＝MIN(Lw*(1+2f％)，W*p％)
Nh＝MIN(Lh*(1+2f％)，H*p％)
lx and Ly respectively represent coordinates of the upper left corner of a rectangular frame corresponding to a region where each small target object is located in the first enhanced image, lw and Lh respectively represent the width and the height of the rectangular frame corresponding to the region where each small target object is located in the first enhanced image, W and H represent the width and the height of the first enhanced image, nx and Ny respectively represent coordinates of the upper left corner of the rectangular frame corresponding to the region where each small target object is located in the first enhanced image after enhancement, nw and Nh respectively represent the width and the height of the rectangular frame corresponding to the region where each small target object is located in the first enhanced image after enhancement, and f and p represent numerical values corresponding to set second percentage and third percentage.
4. The method for detecting the super-resolution enhanced small target based on the image blocks according to claim 1, wherein the regression processing is performed on the rectangular region corresponding to the small target object in each image block in the input image, and the method comprises the following steps:
obtaining coordinates of a rectangular area corresponding to a small target object in each image block in an input image: (x) k ，y k ，TarW k ，TarH k ) Wherein x is k ，y k The abscissa and ordinate of the upper left corner of a rectangular area corresponding to a small target object in each image block in the input image, tarW k ，TarH k The width and the height of a rectangular region corresponding to a small target object in each image block in an input image are set, k represents the number of the image block, and 1 < = k < = n x m;
to (x) k ，y k ，TarW k ，TarH k ) Performing regression treatment to obtain coordinate (X) after regression k ，Y k ，MerW k ，MerH k )：
Figure FDA0003902738350000031
Figure FDA0003902738350000032
Figure FDA0003902738350000041
Figure FDA0003902738350000042
Wherein, X k 、Y k Respectively representing the abscissa and ordinate of the upper left corner after regression of the rectangular region corresponding to the small target object in each image block in the input image, merW k 、MerH k And N represents the scaling ratio of the width and the height of the small target object in the input image relative to the width and the height of the regressed small target object.
5. A super-resolution enhanced small target detection system based on image blocking is characterized by comprising: the device comprises an image acquisition module, a block standard size calculation module, a block step length calculation module, an image blocking module and a detection result acquisition module;
the image acquisition module is configured to acquire a scene image to be subjected to small target detection as an input image;
the block standard size calculation module is configured to acquire the average width and height of small target objects in a training sample of a pre-constructed target detection model during training, set the input width and height by combining the target detection model, and calculate the width and height of a standard block when the input image is blocked;
Figure FDA0003902738350000043
Figure FDA0003902738350000044
wherein Aw and Ah represent the average width and height of small target objects in a training sample, sw and Sh represent the width and height of the target detection model setting input, bw and Bh represent the width and height of a standard block when the input image is blocked, and q represents a set first percentage value;
the block step length calculation module is configured to respectively subtract a set horizontal direction overlap value from a standard block width when the input image is blocked and subtract a set vertical direction overlap value from a standard block height when the input image is blocked to obtain block step lengths of the input image in the horizontal direction and the vertical direction;
the image blocking module is configured to combine the blocking step lengths of the input image in the horizontal direction and the vertical direction, which are obtained by the blocking step length calculation module, to fill the input image, and block the filled input image in a convolution mode to obtain coordinates of each image block and the initial coordinate of each image block in the input image after the input image is blocked;
the detection result acquisition module is configured to perform image enhancement on each image block obtained by the image blocking module by adopting a pre-trained super-resolution model, input a trained target detection model after the image block is enhanced, obtain a rectangular region corresponding to a small target object in each image block in the input image, and perform regression and non-maximum suppression processing to obtain a detection result;
wherein, the input image is filled, and the method comprises the following steps:
Pw＝Tw-(Pre w -Tw)mod Sw
Ph＝Th-(Pre h -Th)mod Sh
wherein, pre w 、Pre h Indicating the width and height of an input image, pw and Ph indicating the width of the right side and the lower side of the input image, tw and Th indicating the step size of the input image in the horizontal direction and the vertical direction;
partitioning the filled input image in a convolution mode, wherein the method comprises the following steps:
Figure FDA0003902738350000051
Figure FDA0003902738350000052
where n and m respectively represent the number of blocks of the input image partitioned in the horizontal direction and the vertical direction.
6. An electronic device, comprising:
at least one processor; and a memory communicatively coupled to at least one of the processors;
wherein the memory stores instructions executable by the processor for execution by the processor to implement the image-patch based super-resolution enhanced small-target detection method of any one of claims 1-4.
7. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions for execution by the computer to implement the image-segmentation-based super-resolution-enhancement small-target detection method of any one of claims 1 to 4.
CN202210888803.XA 2022-07-27 2022-07-27 Super-resolution Enhanced Small Target Detection Method Based on Image Blocking Active CN115082775B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210888803.XA CN115082775B (en) 2022-07-27 2022-07-27 Super-resolution Enhanced Small Target Detection Method Based on Image Blocking

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210888803.XA CN115082775B (en) 2022-07-27 2022-07-27 Super-resolution Enhanced Small Target Detection Method Based on Image Blocking

Publications (2)

Publication Number Publication Date
CN115082775A CN115082775A (en) 2022-09-20
CN115082775B true CN115082775B (en) 2022-12-02

Family

ID=83243316

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210888803.XA Active CN115082775B (en) 2022-07-27 2022-07-27 Super-resolution Enhanced Small Target Detection Method Based on Image Blocking

Country Status (1)

Country Link
CN (1) CN115082775B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115330778B (en) * 2022-10-13 2023-03-10 浙江华是科技股份有限公司 Substation target detection network model training method and system
CN116935290B (en) * 2023-09-14 2023-12-12 南京邮电大学 Heterogeneous target detection method and system for high-resolution array camera in airport scene
CN117252927B (en) * 2023-11-20 2024-02-02 华中科技大学同济医学院附属协和医院 Catheter lower intervention target positioning method and system based on small target detection
CN117576521A (en) * 2024-01-16 2024-02-20 广州市易鸿智能装备股份有限公司 A method to improve the accuracy of industrial image detection models
CN118247711B (en) * 2024-04-12 2025-01-14 中科南京人工智能创新研究院 Method and system for detecting small target of transducer architecture

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113408367A (en) * 2021-05-30 2021-09-17 广州市云景信息科技有限公司 Black smoke ship identification method, device, medium and equipment
CN113435407A (en) * 2021-07-20 2021-09-24 广东电网有限责任公司 Small target identification method and device for power transmission system
CN113610087A (en) * 2021-06-30 2021-11-05 国网福建省电力有限公司信息通信分公司 Image small target detection method based on prior super-resolution and storage medium
WO2022100470A1 (en) * 2020-11-13 2022-05-19 Zhejiang Dahua Technology Co., Ltd. Systems and methods for target detection

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111539907B (en) * 2019-07-25 2023-09-12 毫末智行科技有限公司 Image processing method and device for target detection
CN110956126B (en) * 2019-11-27 2023-09-15 云南电网有限责任公司电力科学研究院 Small target detection method combined with super-resolution reconstruction
CN111524135B (en) * 2020-05-11 2023-12-26 安徽继远软件有限公司 Method and system for detecting defects of tiny hardware fittings of power transmission line based on image enhancement
CN114119646A (en) * 2021-11-02 2022-03-01 中山大学·深圳 Remote sensing image target detection method, device and medium based on foreground super-resolution

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022100470A1 (en) * 2020-11-13 2022-05-19 Zhejiang Dahua Technology Co., Ltd. Systems and methods for target detection
CN113408367A (en) * 2021-05-30 2021-09-17 广州市云景信息科技有限公司 Black smoke ship identification method, device, medium and equipment
CN113610087A (en) * 2021-06-30 2021-11-05 国网福建省电力有限公司信息通信分公司 Image small target detection method based on prior super-resolution and storage medium
CN113435407A (en) * 2021-07-20 2021-09-24 广东电网有限责任公司 Small target identification method and device for power transmission system

Also Published As

Publication number Publication date
CN115082775A (en) 2022-09-20

Similar Documents

Publication Publication Date Title
CN115082775B (en) Super-resolution Enhanced Small Target Detection Method Based on Image Blocking
CN106683052B (en) Defogging photographs and videos using visual artifact suppression
JP5452116B2 (en) High resolution video acquisition apparatus and method
WO2022105197A1 (en) Systems and methods for image detection
CN115661144A (en) Self-adaptive medical image segmentation method based on deformable U-Net
CN112651979A (en) Lung X-ray image segmentation method, system, computer equipment and storage medium
CN115018805B (en) Segmentation model training method, image segmentation method, device, equipment and medium
CN111951172A (en) An image optimization method, apparatus, device and storage medium
CN113221925A (en) Target detection method and device based on multi-scale image
WO2022247406A1 (en) Systems and methods for determining key frame images of video data
US20110057933A1 (en) Resolution Enhancement of Video Stream Based on Spatial and Temporal Correlation
CN106169174B (en) A method for image enlargement
CN114862889A (en) Road edge extraction method and device based on remote sensing image
CN115526850A (en) Training method and device for real-space decoder of refrigeration electron microscope and electronic equipment
CN118657946A (en) Fully polarimetric SAR image segmentation network model, segmentation method and training method
CN117292120B (en) Light-weight visible light insulator target detection method and system
CN117315406B (en) Sample image processing method, device and equipment
CN112052863B (en) Image detection method and device, computer storage medium and electronic equipment
CN117218481A (en) Fish identification method, device, equipment and storage medium
CN114745545A (en) Video frame insertion method, device, equipment and medium
CN113160258A (en) Method, system, server and storage medium for extracting building vector polygon
CN116777935B (en) Method and system for automatically segmenting the entire prostate gland based on deep learning
CN112651926A (en) Method and device for detecting cracks based on recursive attention mechanism
CN117152622B (en) Boundary optimization model training, boundary optimization method, device, equipment and medium
CN117830645B (en) Remote sensing basic model network rapid fine-tuning method and device based on parameter freezing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant