CN116977777A - Small target detection data enhancement method and system based on unmanned aerial vehicle inspection scene - Google Patents
Small target detection data enhancement method and system based on unmanned aerial vehicle inspection scene Download PDFInfo
- Publication number
- CN116977777A CN116977777A CN202310736258.7A CN202310736258A CN116977777A CN 116977777 A CN116977777 A CN 116977777A CN 202310736258 A CN202310736258 A CN 202310736258A CN 116977777 A CN116977777 A CN 116977777A
- Authority
- CN
- China
- Prior art keywords
- image
- small target
- sub
- graph
- small
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000007689 inspection Methods 0.000 title claims abstract description 34
- 238000001514 detection method Methods 0.000 title claims abstract description 32
- 230000003190 augmentative effect Effects 0.000 claims abstract description 15
- 238000004519 manufacturing process Methods 0.000 claims abstract description 9
- 238000011176 pooling Methods 0.000 claims description 22
- 230000002708 enhancing effect Effects 0.000 claims description 12
- 238000012216 screening Methods 0.000 claims description 10
- 238000005070 sampling Methods 0.000 claims description 8
- 240000002989 Euphorbia neriifolia Species 0.000 claims description 5
- 238000002372 labelling Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 13
- 238000003860 storage Methods 0.000 description 7
- 238000004590 computer program Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 239000004973 liquid crystal related substance Substances 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000005520 cutting process Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013434 data augmentation Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Remote Sensing (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a small target detection data enhancement method based on an unmanned aerial vehicle inspection scene, which comprises the following steps: the method comprises the steps of manufacturing a small target sample data set, and obtaining 6 subgraphs corresponding to the small target image sample set; any sub-graph is spliced to generate an augmented target graph with the size twice that of the sub-graph, pixels at corresponding positions in 6 sub-graphs are randomly extracted to be used as jigsaw pixels, sub-graph splicing is completed, and a new graph formed by sub-graph splicing is recorded as a small target augmented graph; small object enhancement map paste. According to the method provided by the application, the enhancement of the small target image is realized, meanwhile, the small target is pasted in a grid mode, so that the small target samples are distributed more uniformly, the sample number of the small targets on a single image can be controlled through the adjustment of a plurality of parameters, the enhancement of the small targets in an unmanned aerial vehicle inspection scene is finally realized, and the detection effect of the small targets is improved.
Description
Technical Field
The application relates to the technical field of deep learning and unmanned aerial vehicle application, in particular to a small target detection data enhancement method and system based on an unmanned aerial vehicle inspection scene.
Background
In recent years, unmanned aerial vehicle replaces the manual work gradually and patrol and examine, and this not only can break away from production work and to the reliance of manpower, reduces labor expenditure and cost, can also promote the accuracy and the efficiency of patrol and examine work simultaneously, guarantee the safety of operation. And carrying a camera to shoot the ground environment in the unmanned aerial vehicle inspection process, transmitting data to a server side through image transmission or carrying out target detection by using airborne embedded equipment, and alarming an abnormal target.
The small target detection is a target with small pixel ratio in the image, the size is small, the feature information which can be extracted by the depth network is very limited, and the features of the small target such as the outline, the texture, the shape and the like in the image are often not obvious. Unlike the mature large-scale and medium-scale target detection in the prior art, the small target has the inherent defects of less semantic information, small coverage area and the like, so that the detection effect of the small target is not ideal. The common small target detection thought is mainly improved in aspects of data enhancement, improvement of input feature diagram size, stronger backbone network, multi-scale feature fusion and the like.
However, in the aspects of electric power and oil gas, the unmanned aerial vehicle inspection generally flies higher, the shot ground targets are smaller, and the missed inspection of the targets is easy to cause; for the scenes such as desert gobi, mountain basin, oil and gas pipeline vicinity and the like, few targets such as personnel or vehicles appear in most of the time, and the problem of few target samples is also caused; in addition, the difference between the shooting visual angle and monitoring during inspection of the unmanned aerial vehicle also leads to the fact that external open source data can not be well utilized during inspection of the unmanned aerial vehicle. Therefore, increasing the algorithmic effect from a data enhancement perspective is the preferred way to implement such scenarios.
At this stage, common data enhancement methods are typically: the method for enhancing the small target data in the visual angle of the unmanned aerial vehicle has no method for enhancing the small target samples in complex scenes such as desert gobi, mountain basin, near oil and gas pipelines and the like by translating, rotating, scale transforming and jigsaw images or adjusting the overall color and brightness of the images and adding noise and the like.
Disclosure of Invention
In view of the foregoing drawbacks or shortcomings in the prior art, it is desirable to provide a method for enhancing small target detection data in an unmanned aerial vehicle-based inspection scenario.
In a first aspect, an embodiment of the present application provides a method for enhancing small target detection data in an unmanned aerial vehicle inspection scene, where the method includes:
step S1: manufacturing a small target sample data set, sequentially screening targets with widths or heights smaller than 40 pixels from the manually marked data set, and expanding a cut area image after a preset boundary to serve as a small target image sample set;
step S2: 6 subgraphs corresponding to the small target image sample set are obtained;
step S3: any sub-graph is spliced to generate an augmented target graph with the size twice that of the sub-graph, pixels at corresponding positions in 6 sub-graphs are randomly extracted to be used as jigsaw pixels, sub-graph splicing is completed, and a new graph formed by sub-graph splicing is recorded as a small target augmented graph;
step S4: small object enhancement map paste.
In one embodiment, the sequentially screening targets with a width or height smaller than 40 pixels from the manually marked data set includes:
the artificially labeled dataset is defined as (cls, x) 1 ，y 1 ，x 2 ，y 2 ) Wherein cls is the target class, (x 1 ，y 1 ) For the upper left point of the target, (x) 2 ，y 2 ) Is the lower right point of the target;
according to formula (x 2 -x 1 )＜thor(y 2 -y 1 ) < th, where th is the threshold for the small target, 40 is taken here.
In one embodiment, the step S2 includes: s2-1: performing maximum pooling operation of filter=n× N, stride =n on a small target image to obtain a first sub-image, wherein N is greater than or equal to 2
S2-2: performing average pooling operation of filter=n× N, stride =n on the small target image to obtain a second sub-image;
s2-3: and taking the positions of four pixels (0, 0), (0, 1), (1, 0) and (1, 1) as starting points, and sampling the small target image at equal intervals by every other pixel in the row-column direction to obtain other corresponding sub-images.
In one embodiment, the method comprises the steps of: s2-1: performing maximum pooling operation of filter=2×2 and stride=2 on the small target image to obtain a first sub-image;
s2-2: and carrying out average pooling operation of filter=2×2 and stride=2 on the small target image to obtain a second sub-image.
In one embodiment, the amplified target image is an image a, and any one of the manually noted data sets is an image B, and the step S4 includes:
s4-1: dividing an image B into a grid of m x n;
s4-2: traversing each grid in turn, and taking a random point in the grid as a center point of the image A;
s4-3: judging whether the circumscribed rectangle of the image A in the image B is overlapped with the existing target area in the image B, if so, continuing to execute the step S4-2; if the circumscribed rectangle has a part outside the image B, continuing to execute the step S4-2; if no overlap exists, executing the next step;
s4-4: and pasting the image A at a position taking the random point as the center, and writing the information of the category, the size and the position of the image A in the image B into the labeling file of the image B.
In a second aspect, an embodiment of the present application provides a small target detection data enhancement system based on an unmanned aerial vehicle inspection scene, where the system includes:
the manufacturing module is used for manufacturing a small target sample data set, sequentially screening targets with the width or the height smaller than 40 pixels from the manually marked data set, and expanding a cut area image after a preset boundary to serve as a small target image sample set;
the acquisition module is used for acquiring 6 subgraphs corresponding to the small target image sample set;
the splicing module is used for splicing any sub-graph to generate an augmented target graph with the size twice that of the sub-graph, randomly extracting pixels at corresponding positions in 6 sub-graphs to serve as jigsaw pixels, completing sub-graph splicing, and marking a new graph formed by sub-graph splicing as a small target augmented graph;
and the pasting module is used for pasting the small target enhancement map.
In one embodiment, the sequentially screening targets with a width or height smaller than 40 pixels from the manually marked data set includes:
the artificially labeled dataset is defined as (cls, x) 1 ，y 1 ，x 2 ，y 2 ) Wherein cls is the target class, (x 1 ，y 1 ) For the upper left point of the target, (x) 2 y 2 ) Is the lower right point of the target;
according to formula (x 2 -x 1 )＜thor(y 2 -y 1 ) < th, where th is the threshold for the small target, 40 is taken here.
In one embodiment, the acquiring module includes:
a first operation module for performing maximum pooling operation of filter=n× N, stride =n on the small target image to obtain a first sub-graph, where N is greater than or equal to 2
The second operation module is configured to perform filter=n× N, stride =n averaging pooling operation on the small target image, and obtain a second sub-graph:
the sampling module is used for sampling the small target image at equal intervals by taking the positions of four pixels (0, 0), (0, 1), (1, 0) and (1, 1) as starting points respectively and every other pixel in the row-column direction to obtain corresponding other four subgraphs.
In one embodiment, the method comprises the steps of:
the first operation module is used for carrying out maximum pooling operation of filter=2×2 and stride=2 on the small target image to obtain a first sub-image;
and the second operation module is used for carrying out average pooling operation of filter=2×2 and stride=2 on the small target image to obtain a second sub-image.
In one embodiment, the augmented target image is an image a, and any one of the manually marked data sets is an image B, and the pasting module includes:
the dividing module is used for dividing the image B into grids of m x n;
the traversing module is used for traversing each grid in turn, and randomly taking a point in the grid as a center point of the image A;
the judging module is used for judging whether the circumscribed rectangle of the image A in the image B is overlapped with the existing target area in the image B, and if so, the step S4-2 is continuously executed; if the circumscribed rectangle has a part outside the image B, continuing to execute the step S4-2; if no overlap exists, executing the next step;
and the writing module is used for pasting the image A at a position with the random point as the center, and writing the information of the category, the size and the position of the image A in the image B into the annotation file of the image B.
The beneficial effects of the application include:
according to the small target detection data enhancement method based on the unmanned aerial vehicle inspection scene, targets with widths or heights smaller than 40 pixels are sequentially screened out from a manually marked data set, the region image is cut out to serve as a small target image sample set after the boundary is expanded, 6 subgraphs are extracted from the small target image, then the subgraphs are spliced to serve as a small target enhancement map, finally the small target enhancement map is pasted on any image in the sample set to form a new sample, and finally the effect of enhancing the small target image in the data set is achieved. The method realizes the enhancement of the small target image, simultaneously, the small target can be pasted in a grid mode to ensure that the small target samples are distributed more uniformly, the sample number of the small targets on a single image can be controlled through the adjustment of a plurality of parameters, the enhancement of the small targets in the unmanned aerial vehicle inspection scene is finally realized, and the detection effect of the small targets is improved. The method is suitable for the inspection scenes of unmanned aerial vehicles such as desert gobi, mountain basin, oil and gas pipeline vicinity and the like, and the small target detection effect is improved.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the accompanying drawings in which:
fig. 1 shows a flow diagram of a small target detection data enhancement method in an unmanned aerial vehicle inspection scene according to an embodiment of the present application;
FIG. 2 is a schematic diagram of small target image sample acquisition in the present application;
FIG. 3 is a representation of various coordinates obtained from a small target image sample in accordance with the present application;
FIG. 4 is a schematic diagram of six subgraphs acquisition in the present application;
FIG. 5 is a schematic diagram of the puzzle principle generated by combining six subgraphs in the application;
FIG. 6 is a graph showing the relationship between parameters related to the pasting of small target enhancement graphs in step 4 of the present application;
FIG. 7 is a graph of enhancement effect of the present application;
fig. 8 shows a schematic diagram of a computer system suitable for use in implementing the terminal device of an embodiment of the application.
Detailed Description
The application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and not limiting of the application. It should be noted that, for convenience of description, only the portions related to the application are shown in the drawings.
It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The application will be described in detail below with reference to the drawings in connection with embodiments.
As mentioned in the background art section, in recent years, unmanned aerial vehicles gradually replace manual inspection, and in the inspection process of unmanned aerial vehicles, cameras are carried on to shoot the ground environment, data are transmitted to a server side through image transmission or an airborne embedded device is used for object detection, and abnormal objects are warned.
However, unmanned aerial vehicle inspection generally flies higher in power and oil gas, and targets in photographed pictures are smaller. For the scenes such as desert gobi, mountain basin, oil and gas pipeline vicinity and the like, few targets such as personnel or vehicles appear in most of the time, so that the target samples are few; in addition, the difference between the shooting visual angle and monitoring during inspection of the unmanned aerial vehicle also leads to the fact that external open source data can not be well utilized during inspection of the unmanned aerial vehicle. Therefore, increasing the algorithmic effect from a data enhancement perspective is the preferred way to implement such scenarios.
In order to solve the above problems, an embodiment of the present application provides a method for enhancing small target detection data in an unmanned aerial vehicle inspection scene, which includes:
step S1: manufacturing a small target sample data set, sequentially screening targets with widths or heights smaller than 40 pixels from the manually marked data set, and expanding a cut area image after a preset boundary to serve as a small target image sample set;
step S2: 6 subgraphs corresponding to the small target image sample set are obtained;
step S3: any sub-graph is spliced to generate an augmented target graph with the size twice that of the sub-graph, pixels at corresponding positions in 6 sub-graphs are randomly extracted to be used as jigsaw pixels, sub-graph splicing is completed, and a new graph formed by sub-graph splicing is recorded as a small target augmented graph;
step S4: small object enhancement map paste.
Specifically, step S1 includes: sequentially screening targets with the width or the height smaller than 40 pixels from the manually marked data set, expanding the boundary, and then cutting out the region image to be used as a small target image sample set;
the step S2 comprises the following steps: respectively carrying out maximum pooling, average pooling and equidistant sampling on the small target image to obtain 6 corresponding subgraphs;
the step S3 comprises the following steps: for sub-graph splicing of any small target, generating an augmented target graph with the size being 2 times of the sub-graph, randomly taking pixels at corresponding positions in 6 sub-graphs of the small target as jigsaw pixels, completing sub-graph splicing, and marking a new graph formed by sub-graph splicing as a small target augmented graph;
the step S4 includes: the small target enhancement map pasting method is characterized in that a small target enhancement map is recorded as an image A, any map in a manually marked data set is recorded as an image B without losing generality, and the small target enhancement map pasting method comprises the following steps of:
(1) Dividing an image B into a grid of m x n;
(2) Traversing each grid in turn, and taking a random point in the grid as a center point of the image A with a certain probability rho;
(3) Judging whether the circumscribed rectangle of the image A in the image B is overlapped with the existing target area in the image B, if so, continuing to execute the step (2), if not, executing the step (4), and if part of the circumscribed rectangle is outside the image B, continuing to execute the step (2);
(4) And pasting the image A at the position taking the random point as the center, and writing the category, size and position information of the small target in the image A in the image B into the annotation file of the image B.
According to the technical scheme, targets with the width or the height smaller than 40 pixels are sequentially screened from the manually marked data set, the region image is cut out to serve as a small target image sample set after the boundary is expanded, 6 subgraphs are extracted from the small target image, then the subgraphs are spliced to serve as small target enhancement images, finally the small target enhancement images are pasted on any image in the sample set to form a new sample, and finally the effect of enhancing the small target image in the data set is achieved. The method realizes the enhancement of the small target image, simultaneously, the small target can be pasted in a grid mode to ensure that the small target samples are distributed more uniformly, the sample number of the small targets on a single image can be controlled through the adjustment of a plurality of parameters, the enhancement of the small targets in the unmanned aerial vehicle inspection scene is finally realized, and the detection effect of the small targets is improved. The method is suitable for the inspection scenes of unmanned aerial vehicles such as desert gobi, mountain basin, oil and gas pipeline vicinity and the like, and the small target detection effect is improved.
In some embodiments, in step S1, for a manually labeled dataset, the tag is always described directly or indirectly as (cls, x 1 ，y 1 ，x 2 ，y 2 ) Respectively expressed as object class cls, object upper left point (x 1 ，y 1 ) Target lower right point (x) 2 ，y 2 ). Judging whether the target is a small target or not according to whether the target meets the requirement of the formula 1 or not:
(x 2 -x 1 )＜thor(y 2 -y 1 ) < th (equation 1)
Where th is a threshold for judging as a small target, here 40. And (3) performing expansion boundary clipping on the small target, and not processing the target which does not meet the small target. The small target image acquisition process is shown in fig. 2, and the coordinate relationship is shown in fig. 3, and the center point (cx, cy) and the width and height (bw, bh) of the small target are acquired first:
the boundary is then randomly expanded by 120-150% of the previous width and height of the small object, and the expanded boundary region has a width and height (bw) in the image e ，bh e ) Upper left dotBottom right dot->The method comprises the following steps of:
wherein rand (1.2,1.5) represents randomly generating a number between 1.2 and 1.5. The region after the boundary expansion is as the region indicated by the broken line in fig. 2, if the region after the boundary expansion is out of range, the small target graph is abandoned; for meeting the requirementsA region, which is taken as a small target sample graph, and the target region of the small target graph is marked as
The expansion boundary of the target area does not affect the category of the target, so that the category of the target obtained after cutting is consistent with that obtained before cutting. And (3) storing all obtained small target graphs and corresponding marks to form a small target image sample set in the step S1.
In some embodiments, in step S2, fig. 4 is a schematic diagram of sub-graph acquisition, and 6 sub-graphs are obtained by:
(1) Performing maximum pooling operation of filter=2×2 and stride=2 on the small target image to obtain a first sub-image;
(2) Obtaining a second sub-graph by carrying out average pooling operation of filter=2×2 and stride=2 on the small target image;
(3) And taking the positions of four pixels (0, 0), (0, 1), (1, 0) and (1, 1) as starting points, and sampling the small target image at equal intervals by every other pixel in the row-column direction to obtain other corresponding sub-images.
In some embodiments, when the first and second subgraphs are obtained in step S2, the maximum pooling and the average pooling are not limited to using the kernel with filter=2x2, and a convolution kernel with filter= 3*3 or larger size may be used, so long as the subgraphs meeting the size requirement can be obtained finally.
In some embodiments, in step S3, the pixel point t (x, y) at any position in the small target enhancement map is randomly selected to obtain the pixel value at the position (x// 2, y// 2) corresponding to the 6 sub-graphs of the small target, where "///" indicates a downward rounding, and may also be selected according to a certain fixed rule. Fig. 5 is a schematic diagram of a jigsaw principle generated by merging sub-images, wherein x% 2=a, y% 2=b, and% represents integer remainder operation, and only 0 or 1 can be taken according to the values of a and b, and then the pixel values of the corresponding (x// 2, y// 2) positions in sub-images 1, 2, 3 and 4 are taken respectively according to the conditions that a and b are (0, 0), (0, 1), (1, 0), (1, 1 and 1).
In some embodiments, in step S4, the image B may be derived from an existing manually labeled sample set, or may directly select a background image without any target of interest, so as to increase the learning of the algorithm model on the background. In the grid dividing the image B into m×n, two parameters of m and n are empirical values, each grid is close to a square, the total number of grids m×n is not more than 30, a 1080P image and a 3×4 grid are taken as an example, the coordinate relationship is as shown in fig. 6, and mapping is realized according to the following steps:
(1) Dividing 1080P image B into 3 x 4 grids, and then dividing 1080P image B into 1920 pixels in width and 1080 pixels in height, and each grid has a height g h 360 pixels and 480 pixels wide gw, respectively;
(2) Each grid is traversed in turn in a line scanning mode, a uniformly distributed random number generation function rand (0, 1) with the range of 0-1 is set, if the random number is larger than the probability rho, the grid is skipped, and otherwise, a point is randomly arranged in the grid to serve as the center point of the image A.
For any one lattice (g i ，g j ) Wherein 0.ltoreq.g i ＜m、0≤g j And < n, then a random center point in the lattice is:
the corresponding coordinates of the upper left point and the lower right point are as follows:
(3) Judging whether the circumscribed rectangle of the image A in the image B is overlapped with the existing target area of the image B, if so, continuing to execute the step (2), if not, executing the step (4), and if part of the circumscribed rectangle is outside the image B, continuing to execute the step (2). Judging whether two rectangular frames overlap or not through classical IOU in target detection, and considering overlapping if the IOU is greater than 0.
(4) And pasting the image A at the position taking the random point as the center, and writing the information of the category, the size and the position of the image A in the image B into the labeling file of the image B.
The coordinates in image B of the object in image a that can be pasted are determined by:
wherein, the liquid crystal display device comprises a liquid crystal display device,in step S1, the type of the target is unchanged after mapping with respect to the coordinates of the upper left point of the actual coordinates of the small target.
In some embodiments, in step S4, the probability ρ is a fraction between 0 and 1, and in practice, the density of small object paste can be determined approximately by controlling three parameters of m, n, ρ:
wherein, the liquid crystal display device comprises a liquid crystal display device,the probability of overlapping or crossing the boundary of the image B with the original marked object in the set formed by the map A and the map B to be mapped can be obtained by statistics of the traversing data set.
Further, let the image data composed of the map to be mapped B be N pieces, then the number of data augmentation for the small object is:
N s_obj ＝N×p s_obj (equation 11)
Referring now to FIG. 8, there is illustrated a schematic diagram of a computer system 300 suitable for use in implementing a terminal device or server in accordance with an embodiment of the present application.
As shown in fig. 8, the computer system 300 includes a Central Processing Unit (CPU) 301 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 302 or a program loaded from a storage section 308 into a Random Access Memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the system 300 are also stored. The CPU 301, ROM 302, and RAM 303 are connected to each other through a bus 304. An input/output (I/O) interface 305 is also connected to bus 304.
The following components are connected to the I/O interface 305: an input section 306 including a keyboard, a mouse, and the like; an output portion 307 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 308 including a hard disk or the like; and a communication section 309 including a network interface card such as a LAN card, a modem, or the like. The communication section 309 performs communication processing via a network such as the internet. The drive 310 is also connected to the I/O interface 305 as needed. A removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed on the drive 310 as needed, so that a computer program read therefrom is installed into the storage section 308 as needed.
In particular, according to embodiments of the present disclosure, the processes described above with reference to fig. 1-2 may be implemented as computer software programs or provide related processing services in the form of HTTP interfaces. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the methods of fig. 1-2. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 309, and/or installed from the removable medium 311.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules involved in the embodiments of the present application may be implemented in software or in hardware. The described units or modules may also be provided in a processor, for example, as: a processor includes a first sub-region generation unit, a second sub-region generation unit, and a display region generation unit. The names of these units or modules do not constitute a limitation of the unit or module itself in some cases, and for example, the display area generating unit may also be described as "a unit for generating a display area of text from the first sub-area and the second sub-area".
As another aspect, the present application also provides a computer-readable storage medium, which may be a computer-readable storage medium contained in the foregoing apparatus in the foregoing embodiment; or may be a computer-readable storage medium, alone, that is not assembled into a device. The computer-readable storage medium stores one or more programs for use by one or more processors in performing the text generation method described in the present application as applied to transparent window envelopes.
The above description is only illustrative of the preferred embodiments of the present application and of the principles of the technology employed. It will be appreciated by persons skilled in the art that the scope of the application is not limited to the specific combinations of the features described above, but also covers other embodiments which may be formed by any combination of the features described above or their equivalents without departing from the spirit of the application. Such as the above-mentioned features and the technical features disclosed in the present application (but not limited to) having similar functions are replaced with each other.
Claims (10)
1. The method for enhancing the small target detection data based on the unmanned aerial vehicle inspection scene is characterized by comprising the following steps:
step S1: manufacturing a small target sample data set, sequentially screening targets with widths or heights smaller than 40 pixels from the manually marked data set, and expanding a cut area image after a preset boundary to serve as a small target image sample set;
step S2: 6 subgraphs corresponding to the small target image sample set are obtained;
step S3: any sub-graph is spliced to generate an augmented target graph with the size twice that of the sub-graph, pixels at corresponding positions in 6 sub-graphs are randomly extracted to be used as jigsaw pixels, sub-graph splicing is completed, and a new graph formed by sub-graph splicing is recorded as a small target augmented graph;
step S4: small object enhancement map paste.
2. The method for enhancing small target detection data in an unmanned aerial vehicle inspection scene according to claim 1, wherein the step of sequentially screening targets with widths or heights smaller than 40 pixels from the manually marked data set comprises the following steps:
the artificially labeled dataset is defined as (cls, x) 1 ，y 1 ，x 2 ，y 2 ) Wherein cls is the target class, (x 1 ，y 1 ) For the upper left point of the target, (x) 2 ，y 2 ) Is the lower right point of the target;
according to formula (x 2 -x 1 )＜thor(y 2 -y 1 ) < th, where th is the threshold for the small target, 40 is taken here.
3. The method for enhancing small target detection data in the unmanned aerial vehicle-based inspection scene according to claim 1, wherein the step S2 comprises:
s2-1: performing maximum pooling operation of filter=n× N, stride =n on a small target image to obtain a first sub-image, wherein N is greater than or equal to 2
S2-2: performing average pooling operation of filter=n× N, stride =n on the small target image to obtain a second sub-image;
s2-3: and taking the positions of four pixels (0, 0), (0, 1), (1, 0) and (1, 1) as starting points, and sampling the small target image at equal intervals by every other pixel in the row-column direction to obtain other corresponding sub-images.
4. The small target detection data enhancement method based on the unmanned aerial vehicle inspection scene as claimed in claim 3, comprising the following steps:
s2-1: performing maximum pooling operation of filter=2×2 and stride=2 on the small target image to obtain a first sub-image;
s2-2: and carrying out average pooling operation of filter=2×2 and stride=2 on the small target image to obtain a second sub-image.
5. The method for enhancing small target detection data in an unmanned aerial vehicle inspection scene according to claim 1, wherein the enhanced target image is an image a, and any one of the images in the manually marked data set is an image B, and the step S4 includes:
s4-1: dividing an image B into a grid of m x n;
s4-2: traversing each grid in turn, and taking a random point in the grid as a center point of the image A;
s4-3: judging whether the circumscribed rectangle of the image A in the image B is overlapped with the existing target area in the image B, if so, continuing to execute the step S4-2; if the circumscribed rectangle has a part outside the image B, continuing to execute the step S4-2; if no overlap exists, executing the next step;
s4-4: and pasting the image A at a position taking the random point as the center, and writing the information of the category, the size and the position of the image A in the image B into the labeling file of the image B.
6. The utility model provides a small target detection data enhancement system based on unmanned aerial vehicle patrols and examines scene which characterized in that, this system includes:
the manufacturing module is used for manufacturing a small target sample data set, sequentially screening targets with the width or the height smaller than 40 pixels from the manually marked data set, and expanding a cut area image after a preset boundary to serve as a small target image sample set;
the acquisition module is used for acquiring 6 subgraphs corresponding to the small target image sample set;
the splicing module is used for splicing any sub-graph to generate an augmented target graph with the size twice that of the sub-graph, randomly extracting pixels at corresponding positions in 6 sub-graphs to serve as jigsaw pixels, completing sub-graph splicing, and marking a new graph formed by sub-graph splicing as a small target augmented graph;
and the pasting module is used for pasting the small target enhancement map.
7. The system for enhancing small target detection data in an unmanned aerial vehicle-based inspection scene as claimed in claim 6, wherein the sequentially screening targets with a width or height less than 40 pixels from the manually labeled data set comprises:
the artificially labeled dataset is defined as (cls, x) 1 ，y 1 ，x 2 ，y 2 ) Wherein cls is the target class, (x 1 ，y 1 ) For the upper left point of the target, (x) 2 y 2 ) Is the lower right point of the target;
according to formula (x 2 -x 1 )＜thor(y 2 -y 1 ) < th, where th is the threshold for the small target, 40 is taken here.
8. The small target detection data enhancement system in an unmanned aerial vehicle-based inspection scenario of claim 6, wherein the acquisition module comprises:
a first operation module for performing maximum pooling operation of filter=n× N, stride =n on the small target image to obtain a first sub-graph, where N is greater than or equal to 2
The second operation module is used for carrying out filter=n× N, stride =n average pooling operation on the small target image to obtain a second sub-image;
the sampling module is used for sampling the small target image at equal intervals by taking the positions of four pixels (0, 0), (0, 1), (1, 0) and (1, 1) as starting points respectively and every other pixel in the row-column direction to obtain corresponding other four subgraphs.
9. The small target detection data enhancement system based on the unmanned aerial vehicle inspection scene of claim 8, comprising:
the first operation module is used for carrying out maximum pooling operation of filter=2×2 and stride=2 on the small target image to obtain a first sub-image;
and the second operation module is used for carrying out average pooling operation of filter=2×2 and stride=2 on the small target image to obtain a second sub-image.
10. The small target detection data enhancement system based on the unmanned aerial vehicle inspection scene according to claim 6, wherein the amplified target image is an image a, any one of the manually marked data sets is an image B, and the pasting module comprises:
the dividing module is used for dividing the image B into grids of m x n;
the traversing module is used for traversing each grid in turn, and randomly taking a point in the grid as a center point of the image A;
the judging module is used for judging whether the circumscribed rectangle of the image A in the image B is overlapped with the existing target area in the image B, and if so, the step S4-2 is continuously executed; if the circumscribed rectangle has a part outside the image B, continuing to execute the step S4-2; if no overlap exists, executing the next step;
and the writing module is used for pasting the image A at a position with the random point as the center, and writing the information of the category, the size and the position of the image A in the image B into the annotation file of the image B.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310736258.7A CN116977777A (en) | 2023-06-20 | 2023-06-20 | Small target detection data enhancement method and system based on unmanned aerial vehicle inspection scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310736258.7A CN116977777A (en) | 2023-06-20 | 2023-06-20 | Small target detection data enhancement method and system based on unmanned aerial vehicle inspection scene |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116977777A true CN116977777A (en) | 2023-10-31 |
Family
ID=88470315
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310736258.7A Pending CN116977777A (en) | 2023-06-20 | 2023-06-20 | Small target detection data enhancement method and system based on unmanned aerial vehicle inspection scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116977777A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117710467A (en) * | 2024-02-06 | 2024-03-15 | 天津云圣智能科技有限责任公司 | Unmanned plane positioning method, unmanned plane positioning equipment and aircraft |
-
2023
- 2023-06-20 CN CN202310736258.7A patent/CN116977777A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117710467A (en) * | 2024-02-06 | 2024-03-15 | 天津云圣智能科技有限责任公司 | Unmanned plane positioning method, unmanned plane positioning equipment and aircraft |
| CN117710467B (en) * | 2024-02-06 | 2024-05-28 | 天津云圣智能科技有限责任公司 | Unmanned plane positioning method, unmanned plane positioning equipment and aircraft |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11620439B2 (en) | Enhancing documents portrayed in digital images | |
| US8774555B2 (en) | Image defogging method and system | |
| US11393100B2 (en) | Automatically generating a trimap segmentation for a digital image by utilizing a trimap generation neural network | |
| US11651477B2 (en) | Generating an image mask for a digital image by utilizing a multi-branch masking pipeline with neural networks | |
| CN113505772B (en) | License plate image generation method and system based on generation countermeasure network | |
| CN116977777A (en) | Small target detection data enhancement method and system based on unmanned aerial vehicle inspection scene | |
| CN112614136A (en) | Infrared small target real-time instance segmentation method and device | |
| CN114648640B (en) | Target object monomer method, device, equipment and storage medium | |
| CN107636724A (en) | The adaptive demister of wavelength and method of image | |
| CN116109954B (en) | House potential safety hazard identification method, device and storage medium | |
| CN113052170A (en) | Small target license plate recognition method under unconstrained scene | |
| CN115439803A (en) | Smoke optical flow identification method based on deep learning model | |
| CN116188859A (en) | Tea disease unmanned aerial vehicle remote sensing monitoring method based on superdivision and detection network | |
| CN114299394A (en) | Intelligent interpretation method for remote sensing image | |
| CN114372931A (en) | Target object blurring method and device, storage medium and electronic equipment | |
| CN113537017A (en) | Optical remote sensing image airplane detection method and device based on cascade regression correction | |
| CN116798118A (en) | Abnormal behavior detection method based on TPH-yolov5 | |
| CN115953312A (en) | Joint defogging detection method and device based on single image and storage medium | |
| CN115115538A (en) | Video processing method, device, equipment and storage medium | |
| US11037311B2 (en) | Method and apparatus for augmenting data in monitoring video | |
| CN111191518A (en) | Double-spectrum target detection frame coordinate synchronization and mask layer drawing method | |
| CN111382645A (en) | Method and system for identifying expired buildings in electronic map | |
| US12039252B2 (en) | Enhancing documents portrayed in digital images | |
| JP6044130B2 (en) | Image region dividing apparatus, method, and program | |
| JP2001034755A (en) | Method and device for image processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |