CN115082775B - Super-resolution enhanced small target detection method based on image blocking - Google Patents
Super-resolution enhanced small target detection method based on image blocking Download PDFInfo
- Publication number
- CN115082775B CN115082775B CN202210888803.XA CN202210888803A CN115082775B CN 115082775 B CN115082775 B CN 115082775B CN 202210888803 A CN202210888803 A CN 202210888803A CN 115082775 B CN115082775 B CN 115082775B
- Authority
- CN
- China
- Prior art keywords
- image
- input image
- small target
- block
- height
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 142
- 230000000903 blocking effect Effects 0.000 title claims abstract description 37
- 238000000034 method Methods 0.000 claims abstract description 41
- 238000012545 processing Methods 0.000 claims abstract description 12
- 230000001629 suppression Effects 0.000 claims abstract description 9
- 238000012549 training Methods 0.000 claims description 44
- 238000004364 calculation method Methods 0.000 claims description 16
- 238000003709 image segmentation Methods 0.000 claims description 11
- 238000000638 solvent extraction Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000010606 normalization Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000001846 repelling effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/40—Filling a planar surface by adding surface attributes, e.g. colour or texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the field of target detection, and particularly relates to a super-resolution enhanced small target detection method, a system and equipment based on image blocking, aiming at solving the problem of low detection accuracy of the existing small target detection method. The method comprises the following steps: acquiring a scene image to be subjected to small target detection as an input image; calculating the width and height of a standard block when an input image is blocked; obtaining the step length of the input image in the horizontal direction and the vertical direction; filling the input image, and blocking the filled input image to obtain each image block of the input image after blocking; and performing image enhancement on each obtained image block by adopting a pre-trained super-resolution model, inputting the trained target detection model after the image block is enhanced, obtaining a rectangular area corresponding to a small target object in each image block in the input image, and performing regression and non-maximum suppression processing to obtain a detection result. The invention improves the accuracy of small target detection.
Description
Technical Field
The invention belongs to the field of target detection, and particularly relates to a super-resolution enhanced small target detection method, system and device based on image blocking.
Background
In a computer vision task, small target object detection and semantic segmentation are always recognized problems, and compared with conventional target detection, the detection accuracy of a small target is only about 50% of that of a common target. In the MS COCO data set, an object or a target with an area smaller than 32 × 32 is regarded as a small object with respect to the original image by less than 10%, and in a common data set such as the inclusion COCO, the number of the small objects is more, taking the COCO as an example, a small object with a ratio of 41%, a medium-sized object with a ratio of 34%, and a large object with a ratio of 24%, these data sets are also from life. In many aspects of target detection, such as bird repelling in airports, satellite image target detection, automobile part detection and the like, small target detection is involved, and because no particularly effective method exists in small target detection scenes at present, manual intervention is mainly used in the application scenes, and finally, the result of small target detection is not only low in accuracy but also time-consuming and labor-consuming, so that the application scenes with high difficulty are dealt with. Based on the method, the small target detection algorithm model is enhanced and trained simply by adopting a proper strategy in the training stage, the image is partitioned, enhanced and detected in a prediction stage by adopting a proper proportion, the core idea is that the small target is converted into a normal target size through the image partition, the image block proportion is ensured to be close to the input size of the target detection algorithm label, the image distortion and the small target information loss caused by input image normalization are avoided, and the effect of improving the accuracy and the efficiency of small target detection is achieved.
Disclosure of Invention
In order to solve the above problems in the prior art, that is, to solve the problem that the existing small target detection method is low in detection accuracy, a first aspect of the present invention provides a super-resolution enhanced small target detection method based on image segmentation, the method comprising:
s100, acquiring a scene image to be subjected to small target detection as an input image;
s200, acquiring the average width and height of small target objects in a training sample of a pre-constructed target detection model during training, setting the input width and height by combining the target detection model, and calculating the width and height of a standard block when the input image is blocked;
wherein,
、
represents the average width and height of small target objects in the training sample,
、
representing the width and height of the target detection model setting input,
、
representing the width and height of a standard block when the input image is blocked,
a first percentage value representing a setting;
s300, respectively subtracting a set horizontal direction overlapping value from the width of a standard block when the input image is blocked and subtracting a set vertical direction overlapping value from the height of the standard block when the input image is blocked to obtain the blocking step length of the input image in the horizontal direction and the vertical direction;
s400, combining the step length of the input image obtained in the step S300 in the horizontal direction and the vertical direction, filling the input image, and blocking the filled input image according to a convolution mode to obtain the coordinates of each image block and the initial coordinate of each image block in the input image after the input image is blocked;
and S500, performing image enhancement on each image block obtained in the S400 by adopting a pre-trained super-resolution model, inputting the trained target detection model after the image enhancement to obtain a rectangular area corresponding to a small target object in each image block in the input image, and performing regression and non-maximum suppression processing to further obtain a detection result.
In some preferred embodiments, the target detection model is trained by:
a100, acquiring a training sample and constructing a training set; the training sample comprises a scene sample image and a true value label of a small target object detection result corresponding to the scene sample image;
a200, performing image enhancement on the scene sample image through a pre-trained super-resolution network to obtain a first enhanced image;
a300, acquiring a rectangular frame corresponding to the area where each small target object is located in the first enhanced image, enhancing the resolution of the rectangular frame in the up-down left-right direction by a set percentage, and taking the enhanced first enhanced image as a second enhanced image;
a400, inputting the second enhanced image into a pre-constructed target detection model, and obtaining a prediction detection result of each small target object in the scene sample image;
a500, calculating a loss value based on the predicted detection result and a true value label of the small target object detection result, and updating a model parameter of the target detection model;
and A600, circulating A100-A500 until a trained target detection model is obtained.
In some preferred embodiments, the resolution of the rectangular frame in the up, down, left and right directions is enhanced by a set percentage, and the enhanced first enhanced image is used as the second enhanced image, the method includes:
wherein,
、
respectively representing the coordinates of the upper left corner of a rectangular frame corresponding to the area of each small target object in the first enhanced image,
、
respectively showing the width and height of the rectangular frame corresponding to the region of each small target object in the first enhanced image,
、
representing the width and height of the first enhanced image,
、
respectively representing the coordinates of the enhanced upper left corner of the rectangular frame corresponding to the region of each small target object in the first enhanced image,
、
respectively showing the width and height of the enhanced rectangular frame corresponding to the region of each small target object in the first enhanced image,
、
and the numerical values corresponding to the set second percentage and the set third percentage are shown.
In some preferred embodiments, the input image is filled by:
wherein,
、
which represents the width and height of the input image,
、
indicating the width of the right and bottom side fill of the input image,
、
which represents the step size of the input image in the horizontal direction, the vertical direction, blocking.
In some preferred embodiments, the padded input image is segmented in a convolution manner by:
wherein,
、
the numbers of blocks of the input image are respectively indicated in the horizontal direction and the vertical direction.
In some preferred embodiments, the regression processing is performed on the rectangular region corresponding to the small target object in each image block in the input image, and the method includes:
obtaining coordinates of a rectangular area corresponding to a small target object in each image block in an input image: (
，
，
，
) Wherein
，
the abscissa and the ordinate of the upper left corner of the rectangular area corresponding to the small target object in each image block in the input image,
，
for the width and height of the rectangular area corresponding to the small target object in each image block in the input image,
number indicating image block, 1<=
<=n*m；
To (a)
，
，
，
) Performing regression treatment to obtain a coordinate after regression as (A)
，
，
，
）：
Wherein,
、
respectively representing the abscissa and ordinate of the upper left corner after the regression of the rectangular region corresponding to the small target object in each image block in the input image,
、
and N represents the scaling ratio of the width and the height of the small target object in the input image relative to the width and the height of the regressed small target object.
In a second aspect of the present invention, a super-resolution enhanced small target detection system based on image blocking is provided, including: the device comprises an image acquisition module, a block standard size calculation module, a block step length calculation module, an image blocking module and a detection result acquisition module;
the image acquisition module is configured to acquire a scene image to be subjected to small target detection as an input image;
the block standard size calculation module is configured to acquire the average width and height of small target objects in a training sample of a pre-constructed target detection model during training, set the input width and height by combining the target detection model, and calculate the width and height of a standard block when the input image is blocked;
wherein,
、
represents the average width and height of small target objects in the training sample,
、
represents the width and height of the target detection model setting input,
、
representing the width and height of a standard block when the input image is blocked,
a first percentage value representing a setting;
the block step length calculation module is configured to respectively subtract a set horizontal direction overlap value from a standard block width when the input image is blocked and subtract a set vertical direction overlap value from a standard block height when the input image is blocked to obtain block step lengths of the input image in the horizontal direction and the vertical direction;
the image blocking module is configured to fill the input image in combination with the blocking step lengths of the input image in the horizontal direction and the vertical direction, which are obtained by the blocking step length calculation module, and block the filled input image in a convolution mode to obtain coordinates of each image block and the initial coordinate of each image block in the input image after the input image is blocked;
the detection result acquisition module is configured to perform image enhancement on each image block obtained by the image blocking module by adopting a pre-trained super-resolution model, input the trained target detection model after the image block is enhanced, obtain a rectangular area corresponding to a small target object in each image block in the input image, perform regression and non-maximum suppression processing, and further obtain a detection result.
In a third aspect of the present invention, an electronic device is provided, including: at least one processor; and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor for execution by the processor to implement the above-described image-blocking-based super-resolution-enhanced small-target detection method.
In a fourth aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for being executed by the computer to implement the above-mentioned method for detecting a small target based on super-resolution enhancement by image segmentation.
The invention has the beneficial effects that:
the invention improves the accuracy of small target detection.
1) In the training stage of a target model detection model, a super-resolution enhancement model is trained, a training sample of a target detection algorithm is enhanced, and after enhancement, a small target in a scene sample image is marked in a mode based on context information, so that the detection precision of the model is improved for the target model detection model;
2) The method is characterized in that an image is partitioned, enhanced (small targets in image blocks are clearer and have more obvious characteristics) and detected in an actual detection process by adopting a proper proportion, the core idea is that the small targets are converted into normal target sizes through image partitioning, and the image block proportion is close to the labeled input size of a target detection algorithm, so that image distortion and small target information loss caused by input image normalization are avoided, the effect that small target characteristics can be extracted by a deep neural network quickly is achieved, and the small target detection effect is improved by various target detection algorithms.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings.
FIG. 1 is a flowchart illustrating a super-resolution enhanced small target detection method based on image segmentation according to an embodiment of the present invention;
FIG. 2 is a block diagram of a super-resolution enhanced small target detection system based on image segmentation according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating a process of training and detecting a target detection model according to an embodiment of the present invention;
FIG. 4 is a schematic flowchart of small target detection performed by the super-resolution enhanced small target detection method based on image segmentation according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
The super-resolution enhanced small target detection method based on image segmentation, disclosed by the invention, as shown in figure 1, comprises the following steps of:
s100, acquiring a scene image to be subjected to small target detection as an input image;
s200, acquiring the average width and height of small target objects in a training sample of a pre-constructed target detection model during training, setting the input width and height by combining the target detection model, and calculating the width and height of a standard block when the input image is blocked;
wherein,
、
represents the average width and height of small target objects in the training sample,
、
represents the width and height of the target detection model setting input,
、
representing the width and height of a standard block when the input image is blocked,
a first percentage value representing a setting;
s300, respectively subtracting a set horizontal direction overlapping value from the width of a standard block when the input image is blocked and subtracting a set vertical direction overlapping value from the height of the standard block when the input image is blocked to obtain the blocking step length of the input image in the horizontal direction and the vertical direction;
s400, combining the step length of the input image obtained in the step S300 in the horizontal direction and the vertical direction, filling the input image, and blocking the filled input image according to a convolution mode to obtain the coordinates of each image block and the initial coordinate of each image block in the input image after the input image is blocked;
and S500, performing image enhancement on each image block obtained in the step S400 by adopting a pre-trained super-resolution model, inputting the trained target detection model after the image enhancement, obtaining a rectangular area corresponding to a small target object in each image block in the input image, and performing regression and non-maximum suppression processing to further obtain a detection result.
In order to more clearly describe the super-resolution enhanced small target detection method based on image segmentation, the following will describe in detail the steps in an embodiment of the method of the present invention with reference to fig. 3.
In the following embodiments, the training process of the target detection model is detailed, and then the process of detecting a small target by a super-resolution enhanced small target detection method based on image segmentation is detailed.
1. Training process for target detection model
A100, obtaining a training sample and constructing a training set; the training samples comprise scene sample images and truth value labels of small target object detection results corresponding to the scene sample images;
in this embodiment, a true value label of a scene sample image and a corresponding small target object detection result is obtained and used as a training sample to construct a training set.
A200, performing image enhancement on the scene sample image through a pre-trained super-resolution network to obtain a first enhanced image;
in this embodiment, a public dataset or a self-established dataset is used to train a super-resolution network, and a super-resolution multiple is set to be N (i.e., a scaling ratio of the width and the height of a small target object in an input image to the width and the height of a regressed small target object), so that the resolution of a scene sample image is enhanced by N times in the horizontal and vertical directions, respectively. After the super-resolution network is pre-trained, image enhancement is carried out on the scene sample image, and the enhanced scene sample image is used as a first enhanced image. The method is specifically shown in formulas (1) and (2):
wherein,
、
for the width and height of the scene sample images in the training set,
、
the width and height of the first enhanced image.
A300, acquiring a rectangular frame corresponding to the area where each small target object is located in the first enhanced image, performing set percentage enhancement on the resolution corresponding to the rectangular frame in the up-down left-right direction, and taking the enhanced first enhanced image as a second enhanced image;
in this embodiment, an import Context is usedLabeling small targets in the first enhanced image in an information mode, wherein the specific method comprises the following steps: a rectangular frame of the area where the small target object is located is framed on the first enhanced image, and the coordinate of the upper left corner of the rectangular frame is
Width and height of
、
。
The rectangular frame of the area where the small target object is located is floated outwards (namely enhanced) in the upper, lower, left and right resolutions
（
In the present invention, it is generally preferred to take 15 to 20). The coordinate of the upper left corner of the rectangle after floating is
Width and height of
、
However, the small object after floating should not exceed the original image
（
In the present invention, it is generally preferably from 15 to 25), and the range cannot be exceeded. Specifically, the following formulas (3), (4), (5) and (6) show:
wherein,
、
and the numerical values corresponding to the set second percentage and the third percentage are shown.
A400, inputting the second enhanced image into a pre-constructed target detection model, and obtaining a prediction detection result of each small target object in the scene sample image;
in this embodiment, a second enhanced image, which is a small target object marked (i.e., resolution enhanced) in the first enhanced image, is input into a pre-constructed target detection model (preferably set as an SSD model in the present invention), and a predicted detection result of each small target object in the scene sample image is obtained.
A500, calculating a loss value based on the predicted detection result and a true value label of the small target object detection result, and updating a model parameter of the target detection model;
in this embodiment, a loss value is calculated based on the predicted detection result of each small target object in the scene sample image and the truth label of the detection result of the small target object, and the model parameters are updated.
And A600, circulating A100-A500 until a trained target detection model is obtained.
In this embodiment, the target detection model is cycled until a trained target detection model is obtained.
2. An image block-based super-resolution enhanced small target detection method is shown in FIG. 4
S100, acquiring a scene image to be subjected to small target detection as an input image;
in this embodiment, an image of a scene to be detected is obtained first.
S200, acquiring the average width and height of small target objects in a training sample of a pre-constructed target detection model during training, setting the input width and height by combining the target detection model, and calculating the width and height of a standard block when the input image is blocked;
in this embodiment, the average width and height of a small target object in a training sample of a pre-constructed target detection model is obtained first, and specifically: setting integer numbers for small target objects marked in training set
（1<=
<=
) Counting the original size of the labeled small target object, and averaging the width and height of the small target object
And
the specific calculation method is as follows:
wherein,
、
denotes the first
Width and height of small target objects.
Then, based on the original width and height and the average width and height of the small target object in the training sample of the target detection model during training, and setting the input width and height by combining the target detection model, calculating the width and height of the standard block when the input image is blocked, as shown in formulas (9) and (10):
wherein,
、
represents the average width and height of small target objects in the training sample,
、
representing width, height, of the target detection model setting input, i.e.
、
In relation to the target detection model employed, e.g. SSD model: (
，
= (300 ), if any scale of image is fit using object detection model
、
Respectively taking 256 to 400 of the total weight of the product,
、
representing the width and height of a standard block when the input image is blocked,
a first percentage value is set, meaning the maximum ratio of the size of the small target object to the size of the original image, and is preferably set to 10% in the present invention.
In the above formula, 10% is the highest ratio of small target objects to the original in such a way as to ensure that the target proportion is as high as possible above 10% and the image block ratio is close to the target detection standard input size in predicting the sub-images obtained by image blocking.
S300, respectively subtracting the overlap value in the set horizontal direction from the width of the standard block when the input image is blocked, and subtracting the overlap value in the set vertical direction from the height of the standard block when the input image is blocked to obtain the blocking step length of the input image in the horizontal direction and the vertical direction;
in this embodiment, the step size of horizontal and vertical direction blocks of image block is calculated
And
: the predicted image (i.e. scene image) is partitioned into blocks with width and height of S200 standard block by adopting the overlapping partitioning method, and the overlapping size (i.e. overlapping value) in the horizontal direction is set
Overlap value with vertical direction
，
And
is generally arranged as
、
2 to 4 times of the cutting gap, and is mainly used for reserving small target objects among the cutting gaps.
Step size of input image block in horizontal direction:
；
step size of input image blocking in vertical direction:
。
s400, combining the step length of the input image obtained in the step S300 in the horizontal direction and the vertical direction, filling the input image, and blocking the filled input image according to a convolution mode to obtain the coordinates of each image block and the initial coordinate of each image block in the input image after the input image is blocked;
the parameters calculated in S200 and S300 are equally partitioned in a top-to-bottom and left-to-right manner, which cannot guarantee that the predicted image is completely partitioned, and there may be residual portions on the right side and the lower side of the predicted image that cannot be covered by the standard block.
In this embodiment, the pair of width and height is
And
is filled in at the right and lower sides of the predicted image, the right and lower sides are filled with widths of
And
comprises the following steps:
step size obtained by S300: (
，
) And partitioning the scene image filled in the S400 mode in a convolution mode. Divided into blocks and then horizontally divided into
Blocks, vertically divided into
And (5) blocking.
Wherein,
、
the numbers of blocks of the input image are respectively indicated in the horizontal direction and the vertical direction.
Finally obtained by blocking the image
*
The block image sets an integer number k (1)<=k<= n × m), and records the coordinates of the starting point of the k-th image block in the original scene image: (
，
）。
And S500, performing image enhancement on each image block obtained in the S400 by adopting a pre-trained super-resolution model, inputting the trained target detection model after the image enhancement to obtain a rectangular area corresponding to a small target object in each image block in the input image, and performing regression and non-maximum suppression processing to further obtain a detection result.
In this embodiment, a super-resolution model is first used to enhance the image blocks numbered k one by one. The enhanced image resolution is N times wider and taller than the original image block.
Then, the trained target detection model is adopted to detect the enhanced image blocks one by one, and the rectangular area of the small target object of the kth image block is predicted to be (1)
，
，
，
），
，
The horizontal and vertical coordinates of the upper left corner,
，
the width and the height of small target objects in the image block are obtained. And predicting the coordinates of all the small target objects to perform regression on the original image. The regressed coordinates are (
，
，
，
）：
And finally, merging the regressed targets by adopting a non-maximum value suppression (NMS) method, and rejecting the targets detected at the same time by the overlapping part between the k (k < n x m) th image block edge part and the adjacent image block to obtain a detection result.
A super-resolution enhanced small target detection system based on image blocking according to a second embodiment of the present invention, as shown in fig. 2, includes: the device comprises an image acquisition module 100, a block standard size calculation module 200, a block step calculation module 300, an image blocking module 400 and a detection result acquisition module 500;
the image acquisition module 100 is configured to acquire a scene image to be subjected to small target detection as an input image;
the block standard size calculation module 200 is configured to obtain an average width and height of small target objects in a training sample of a pre-constructed target detection model during training, set an input width and height in combination with the target detection model, and calculate the width and height of a standard block when the input image is blocked;
wherein,
、
represents the average width and height of small target objects in the training sample,
、
represents the width and height of the target detection model setting input,
、
representing the width and height of a standard block when the input image is blocked,
a first percentage value representing a setting;
the block step calculation module 300 is configured to obtain step lengths of the input image in the horizontal direction and the vertical direction by subtracting the overlap value in the set horizontal direction from the width of the standard block when the input image is blocked and subtracting the overlap value in the set vertical direction from the height of the standard block when the input image is blocked;
the image blocking module 400 is configured to combine the blocking step lengths of the input image obtained by the blocking step length calculation module 300 in the horizontal direction and the vertical direction to fill the input image, and block the filled input image in a convolution manner to obtain coordinates of each image block and the initial coordinate of each image block in the input image after the input image is blocked;
the detection result obtaining module 500 is configured to perform image enhancement on each image block obtained by the image partitioning module 400 by using a pre-trained super-resolution model, input the trained target detection model after the image enhancement, obtain a rectangular region corresponding to a small target object in each image block in the input image, and perform regression and non-maximum suppression processing to obtain a detection result.
It should be noted that, the super-resolution enhanced small target detection system based on image segmentation provided in the foregoing embodiment is only illustrated by the division of the above functional modules, and in practical applications, the above functions may be allocated to different functional modules according to needs, that is, the modules or steps in the embodiments of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiments may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the above described functions. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
An electronic device of a third embodiment of the present invention includes: at least one processor; and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor for execution by the processor to implement the above-described image-blocking-based super-resolution-enhanced small-target detection method.
A computer-readable storage medium of a fourth embodiment of the present invention stores computer instructions for being executed by the computer to implement the above-mentioned method for detecting a small target based on super-resolution enhancement of image segmentation.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes and related descriptions of the electronic device and the computer-readable storage medium described above may refer to corresponding processes in the foregoing method examples, and are not described herein again.
Referring now to FIG. 5, there is illustrated a block diagram of a computer system suitable for use as a server in implementing embodiments of the method, system, and apparatus of the present application. The server shown in fig. 5 is only an example, and should not bring any limitation to the functions and the use range of the embodiments of the present application.
As shown in fig. 5, the computer system includes a Central Processing Unit (CPU) 501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data necessary for system operation are also stored. The CPU501, ROM 502, and RAM503 are connected to each other through a bus 504. An Input/Output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output section 507 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage portion 508 including a hard disk and the like; and a communication section 509 including a Network interface card such as a LAN (Local Area Network) card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.
In particular, the processes described above with reference to the flow diagrams may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511. More specific examples of a computer readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing Propagate or transport the program for use by or in connection with the instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (7)
1. A super-resolution enhanced small target detection method based on image blocking is characterized by comprising the following steps:
s100, acquiring a scene image to be subjected to small target detection as an input image;
s200, acquiring the average width and height of small target objects in a training sample of a pre-constructed target detection model during training, setting the input width and height by combining the target detection model, and calculating the width and height of a standard block when the input image is blocked;
wherein Aw and Ah represent the average width and height of small target objects in a training sample, sw and Sh represent the width and height of the target detection model setting input, bw and Bh represent the width and height of a standard block when the input image is blocked, and q represents a set first percentage value;
s300, respectively subtracting a set horizontal direction overlapping value from the width of a standard block when the input image is blocked and subtracting a set vertical direction overlapping value from the height of the standard block when the input image is blocked to obtain the blocking step length of the input image in the horizontal direction and the vertical direction;
s400, combining the step length of the input image obtained in the step S300 in the horizontal direction and the vertical direction, filling the input image, and blocking the filled input image according to a convolution mode to obtain the coordinates of each image block and the initial coordinate of each image block in the input image after the input image is blocked;
s500, performing image enhancement on each image block obtained in the S400 by adopting a pre-trained super-resolution model, inputting a trained target detection model after the image enhancement, obtaining a rectangular region corresponding to a small target object in each image block in the input image, and performing regression and non-maximum suppression processing to further obtain a detection result;
wherein, the input image is filled, and the method comprises the following steps:
Pw＝Tw-(Pre w -Tw)mod Sw
Ph＝Th-(Pre h -Th)mod Sh
wherein, pre w 、Pre h Indicating the width and height of an input image, pw and Ph indicating the width of the right side and the lower side of the input image, tw and Th indicating the step size of the input image in the horizontal direction and the vertical direction;
partitioning the filled input image in a convolution mode, wherein the method comprises the following steps:
where n and m respectively represent the number of blocks of the input image partitioned in the horizontal direction and the vertical direction.
2. The method for detecting the super-resolution enhanced small target based on the image blocks as claimed in claim 1, wherein the training method of the target detection model is as follows:
a100, obtaining a training sample and constructing a training set; the training samples comprise scene sample images and truth value labels of small target object detection results corresponding to the scene sample images;
a200, performing image enhancement on the scene sample image through a pre-trained super-resolution network to obtain a first enhanced image;
a300, acquiring a rectangular frame corresponding to the area where each small target object is located in the first enhanced image, performing set percentage enhancement on the resolution corresponding to the rectangular frame in the up-down left-right direction, and taking the enhanced first enhanced image as a second enhanced image;
a400, inputting the second enhanced image into a pre-constructed target detection model, and obtaining a prediction detection result of each small target object in the scene sample image;
a500, calculating a loss value based on the predicted detection result and a true value label of the small target object detection result, and updating a model parameter of the target detection model;
and A600, circulating A100-A500 until a trained target detection model is obtained.
3. The method for detecting the super-resolution enhanced small target based on the image blocks as claimed in claim 2, wherein the resolution corresponding to the rectangular frame in the up, down, left and right directions is enhanced by a set percentage, and the enhanced first enhanced image is used as the second enhanced image, and the method comprises:
Nx＝MAX(Lx*(1-f％)，0)
Ny＝MAX(Ly*(1-f％)，0)
Nw＝MIN(Lw*(1+2f％)，W*p％)
Nh＝MIN(Lh*(1+2f％)，H*p％)
lx and Ly respectively represent coordinates of the upper left corner of a rectangular frame corresponding to a region where each small target object is located in the first enhanced image, lw and Lh respectively represent the width and the height of the rectangular frame corresponding to the region where each small target object is located in the first enhanced image, W and H represent the width and the height of the first enhanced image, nx and Ny respectively represent coordinates of the upper left corner of the rectangular frame corresponding to the region where each small target object is located in the first enhanced image after enhancement, nw and Nh respectively represent the width and the height of the rectangular frame corresponding to the region where each small target object is located in the first enhanced image after enhancement, and f and p represent numerical values corresponding to set second percentage and third percentage.
4. The method for detecting the super-resolution enhanced small target based on the image blocks according to claim 1, wherein the regression processing is performed on the rectangular region corresponding to the small target object in each image block in the input image, and the method comprises the following steps:
obtaining coordinates of a rectangular area corresponding to a small target object in each image block in an input image: (x) k ，y k ，TarW k ，TarH k ) Wherein x is k ，y k The abscissa and ordinate of the upper left corner of a rectangular area corresponding to a small target object in each image block in the input image, tarW k ，TarH k The width and the height of a rectangular region corresponding to a small target object in each image block in an input image are set, k represents the number of the image block, and 1 < = k < = n x m;
to (x) k ，y k ，TarW k ，TarH k ) Performing regression treatment to obtain coordinate (X) after regression k ，Y k ，MerW k ，MerH k )：
Wherein, X k 、Y k Respectively representing the abscissa and ordinate of the upper left corner after regression of the rectangular region corresponding to the small target object in each image block in the input image, merW k 、MerH k And N represents the scaling ratio of the width and the height of the small target object in the input image relative to the width and the height of the regressed small target object.
5. A super-resolution enhanced small target detection system based on image blocking is characterized by comprising: the device comprises an image acquisition module, a block standard size calculation module, a block step length calculation module, an image blocking module and a detection result acquisition module;
the image acquisition module is configured to acquire a scene image to be subjected to small target detection as an input image;
the block standard size calculation module is configured to acquire the average width and height of small target objects in a training sample of a pre-constructed target detection model during training, set the input width and height by combining the target detection model, and calculate the width and height of a standard block when the input image is blocked;
wherein Aw and Ah represent the average width and height of small target objects in a training sample, sw and Sh represent the width and height of the target detection model setting input, bw and Bh represent the width and height of a standard block when the input image is blocked, and q represents a set first percentage value;
the block step length calculation module is configured to respectively subtract a set horizontal direction overlap value from a standard block width when the input image is blocked and subtract a set vertical direction overlap value from a standard block height when the input image is blocked to obtain block step lengths of the input image in the horizontal direction and the vertical direction;
the image blocking module is configured to combine the blocking step lengths of the input image in the horizontal direction and the vertical direction, which are obtained by the blocking step length calculation module, to fill the input image, and block the filled input image in a convolution mode to obtain coordinates of each image block and the initial coordinate of each image block in the input image after the input image is blocked;
the detection result acquisition module is configured to perform image enhancement on each image block obtained by the image blocking module by adopting a pre-trained super-resolution model, input a trained target detection model after the image block is enhanced, obtain a rectangular region corresponding to a small target object in each image block in the input image, and perform regression and non-maximum suppression processing to obtain a detection result;
wherein, the input image is filled, and the method comprises the following steps:
Pw＝Tw-(Pre w -Tw)mod Sw
Ph＝Th-(Pre h -Th)mod Sh
wherein, pre w 、Pre h Indicating the width and height of an input image, pw and Ph indicating the width of the right side and the lower side of the input image, tw and Th indicating the step size of the input image in the horizontal direction and the vertical direction;
partitioning the filled input image in a convolution mode, wherein the method comprises the following steps:
where n and m respectively represent the number of blocks of the input image partitioned in the horizontal direction and the vertical direction.
6. An electronic device, comprising:
at least one processor; and a memory communicatively coupled to at least one of the processors;
wherein the memory stores instructions executable by the processor for execution by the processor to implement the image-patch based super-resolution enhanced small-target detection method of any one of claims 1-4.
7. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions for execution by the computer to implement the image-segmentation-based super-resolution-enhancement small-target detection method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210888803.XA CN115082775B (en) | 2022-07-27 | 2022-07-27 | Super-resolution enhanced small target detection method based on image blocking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210888803.XA CN115082775B (en) | 2022-07-27 | 2022-07-27 | Super-resolution enhanced small target detection method based on image blocking |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115082775A CN115082775A (en) | 2022-09-20 |
| CN115082775B true CN115082775B (en) | 2022-12-02 |
Family
ID=83243316
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210888803.XA Active CN115082775B (en) | 2022-07-27 | 2022-07-27 | Super-resolution enhanced small target detection method based on image blocking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115082775B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115330778B (en) * | 2022-10-13 | 2023-03-10 | 浙江华是科技股份有限公司 | Substation target detection network model training method and system |
| CN116935290B (en) * | 2023-09-14 | 2023-12-12 | 南京邮电大学 | Heterogeneous target detection method and system for high-resolution array camera in airport scene |
| CN117252927B (en) * | 2023-11-20 | 2024-02-02 | 华中科技大学同济医学院附属协和医院 | Catheter lower intervention target positioning method and system based on small target detection |
| CN117576521A (en) * | 2024-01-16 | 2024-02-20 | 广州市易鸿智能装备股份有限公司 | Method for improving accuracy of industrial image detection model |
| CN118247711A (en) * | 2024-04-12 | 2024-06-25 | 中科南京人工智能创新研究院 | Method and system for detecting small target of transducer architecture |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113408367A (en) * | 2021-05-30 | 2021-09-17 | 广州市云景信息科技有限公司 | Black smoke ship identification method, device, medium and equipment |
| CN113435407A (en) * | 2021-07-20 | 2021-09-24 | 广东电网有限责任公司 | Small target identification method and device for power transmission system |
| CN113610087A (en) * | 2021-06-30 | 2021-11-05 | 国网福建省电力有限公司信息通信分公司 | Image small target detection method based on prior super-resolution and storage medium |
| WO2022100470A1 (en) * | 2020-11-13 | 2022-05-19 | Zhejiang Dahua Technology Co., Ltd. | Systems and methods for target detection |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111539907B (en) * | 2019-07-25 | 2023-09-12 | 毫末智行科技有限公司 | Image processing method and device for target detection |
| CN110956126B (en) * | 2019-11-27 | 2023-09-15 | 云南电网有限责任公司电力科学研究院 | Small target detection method combined with super-resolution reconstruction |
| CN111524135B (en) * | 2020-05-11 | 2023-12-26 | 安徽继远软件有限公司 | Method and system for detecting defects of tiny hardware fittings of power transmission line based on image enhancement |
| CN114119646A (en) * | 2021-11-02 | 2022-03-01 | 中山大学·深圳 | Remote sensing image target detection method, device and medium based on foreground overdivision |
-
2022
- 2022-07-27 CN CN202210888803.XA patent/CN115082775B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022100470A1 (en) * | 2020-11-13 | 2022-05-19 | Zhejiang Dahua Technology Co., Ltd. | Systems and methods for target detection |
| CN113408367A (en) * | 2021-05-30 | 2021-09-17 | 广州市云景信息科技有限公司 | Black smoke ship identification method, device, medium and equipment |
| CN113610087A (en) * | 2021-06-30 | 2021-11-05 | 国网福建省电力有限公司信息通信分公司 | Image small target detection method based on prior super-resolution and storage medium |
| CN113435407A (en) * | 2021-07-20 | 2021-09-24 | 广东电网有限责任公司 | Small target identification method and device for power transmission system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115082775A (en) | 2022-09-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115082775B (en) | Super-resolution enhanced small target detection method based on image blocking | |
| CN108446694B (en) | Target detection method and device | |
| CN111369581A (en) | Image processing method, device, equipment and storage medium | |
| CN110705576B (en) | Region contour determining method and device and image display equipment | |
| CN111340749B (en) | Image quality detection method, device, equipment and storage medium | |
| CN111444807B (en) | Target detection method, device, electronic equipment and computer readable medium | |
| CN110570435A (en) | method and device for carrying out damage segmentation on vehicle damage image | |
| EP4086840A1 (en) | Video denoising method and electronic device | |
| CN112053302A (en) | Denoising method and device for hyperspectral image and storage medium | |
| CN116310993A (en) | Target detection method, device, equipment and storage medium | |
| CN115222683A (en) | Method and system for distributing cleaning personnel, storage medium and electronic equipment | |
| CN111126248A (en) | Method and device for identifying shielded vehicle | |
| CN115937071A (en) | Image detection method, device, equipment and medium | |
| CN113724267B (en) | Breast ultrasonic image tumor segmentation method and device | |
| CN117315406B (en) | Sample image processing method, device and equipment | |
| CN112614108B (en) | Method and device for detecting nodules in thyroid ultrasound image based on deep learning | |
| CN109584165A (en) | A kind of antidote of digital picture, device, medium and electronic equipment | |
| CN116229419B (en) | Pedestrian detection method and device | |
| CN115131392B (en) | Space moving target detection tracking method based on space-based optical observation image | |
| CN114037933A (en) | Method and device for detecting tunnel throwing object, electronic equipment and storage medium | |
| CN112541915A (en) | Efficient cloth defect detection method, system and equipment for high-resolution images | |
| CN113111929B (en) | Template matching method and device, computer equipment and storage medium | |
| CN116363019B (en) | Image data enhancement method, system and device | |
| CN112434674B (en) | Method, system and equipment for detecting tiny human face based on virtual amplification of label box | |
| CN116091367B (en) | Blind deblurring method, device, equipment and medium for optical remote sensing image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |