CN114066788A - Balanced instance segmentation data synthesis method - Google Patents
Balanced instance segmentation data synthesis method Download PDFInfo
- Publication number
- CN114066788A CN114066788A CN202111245120.4A CN202111245120A CN114066788A CN 114066788 A CN114066788 A CN 114066788A CN 202111245120 A CN202111245120 A CN 202111245120A CN 114066788 A CN114066788 A CN 114066788A
- Authority
- CN
- China
- Prior art keywords
- image
- size
- label
- area
- paste
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 13
- 238000001308 synthesis method Methods 0.000 title claims abstract description 13
- 238000000034 method Methods 0.000 claims abstract description 16
- 238000005192 partition Methods 0.000 claims description 2
- 238000012549 training Methods 0.000 abstract description 3
- 238000001514 detection method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 2
- 239000002131 composite material Substances 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
Abstract
The invention discloses a balanced instance segmentation data synthesis method, which comprises the following steps: 1) constructing an object instance library by using the image and the label of the original data set; 2) reading an image and a label in an original data set, generating a foreground background mask image for the image according to the label, and uniformly generating 10 x 10 candidate points according to the size of the image; 3) setting a pasting size list, calculating an area with 10 multiplied by 10 candidate points as centers and a foreground background mask image according to the set pasting size list, and selecting an area which is not overlapped with the foreground and adding the area into a pasting area; 4) and selecting an object from the object instance library through class balance, zooming, pasting the object to a pasting area, and updating the label. The method realizes data enhancement by using an image synthesis method, has better applicability and diversity, can be applied to more difficult example segmentation tasks, has very small calculated amount and high operation speed, and basically does not increase the time for training a network.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a balanced instance segmentation data synthesis method.
Background
Object detection and instance segmentation are important research directions in deep learning, and objects in images are detected and identified. Specifically, the object detection is to frame the object in the image by a rectangular frame, and the instance segmentation is further to realize the object separation at the pixel level. Regardless of target detection or instance segmentation, the data set is an essential part required for training the network, but in practical application, the data set is always very limited, not only in quantity, but also in quality, there is an imbalance problem: unbalanced scale, unbalanced category and unbalanced distribution.
The mainstream method for solving the data set deficiency is data enhancement, the research and implementation of the data enhancement at present mainly comprises color transformation and geometric transformation, the development is carried out to the present, more remarkable innovations are not provided, a certain stagnation is involved, data synthesis is an emerging method in the data enhancement, and more feasible solutions are provided.
In combination with the above discussion, the invention provides a balanced example segmentation data synthesis method which has higher practical application value.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a balanced instance segmentation data synthesis method, which mainly utilizes images and labels of an original data set to construct an object instance library, synthesizes the original data set and the object instance library through an image processing technology, synthesizes new images and generates new labels, thereby achieving the effect of data enhancement and relieving the defects of the data set.
In order to achieve the purpose, the technical scheme provided by the invention is as follows: a balanced instance partition data synthesis method, comprising the steps of:
1) constructing an object instance library by using the image and the label of the original data set;
2) reading an image and a label in an original data set, generating a foreground background mask image for the image according to the label, and uniformly generating 10 x 10 candidate points according to the size of the image;
3) setting a pasting size list, calculating the area taking 10 multiplied by 10 candidate points in the step 2) as the center and the foreground background mask image according to the set pasting size list, and selecting the area which is not overlapped with the foreground and adding the area into the pasting area;
4) selecting an object from the object instance library in the step 1) through class balance, pasting the object to the pasting area in the step 3) after scaling, and updating the label.
Further, in step 1), an object instance library is constructed by using the image and the label of the original data set, a mask corresponding to each object in the image is obtained by analyzing the label, the object is separated from the original image through the mask, and the object is classified according to the category of the object.
Further, in step 2), reading an image and a label in the original data set, generating a foreground-background mask image for the image according to the label, and uniformly generating 10 × 10 candidate points according to the size of the image, including the following steps:
2.1) Generation of a foreground-background mask map
Acquiring an original image and a corresponding label, generating a single-channel image of the original image in length and width dimensions, setting each pixel value to be 0, namely, a foreground background mask image, acquiring a mask of an object in the label, setting the pixel value of the corresponding position of the foreground background mask image to be 1, setting an area with a final pixel value of 1 as a foreground, and setting an area with a pixel value of 0 as a background;
2.2) generating 10 × 10 candidate points
The length and width values of the original image are obtained, the length and the width are respectively and uniformly divided into 11 segments, namely 10 equally divided points are obtained, and the length and the width are respectively combined with the 10 equally divided points to finally obtain 10 multiplied by 10 candidate points.
Further, in step 3), a paste size list is set, and according to the set paste size list, an area satisfying the requirement is selected from the area with the 10 × 10 candidate points as the center in step 2) and added into the paste area, including the following steps:
3.1) set a paste size list
For relative balance of paste sizes, a paste size list is set to 150, 120, 90, 60, 30;
3.2) generating paste area
And sequentially selecting the size from the paste size list, taking the selected size as the side length of a square, taking the candidate point as the center, judging whether the candidate point meets the condition of no foreground, if no foreground exists, adding the paste area and selecting the next size, and if foreground exists, selecting the next candidate point until the paste size list and all candidate points are traversed.
Further, in step 4), the pasting areas in step 3) are sequentially processed, a category is randomly selected from the object instance library in step 1), an object is randomly selected from the category, if the size of the object is smaller than the size of the pasting area by a threshold value, an object is randomly selected again, the size of the object is reduced to a size corresponding to the size of the pasting area after the size requirement is met, the object is pasted to the pasting area, and a new label is added to the original label.
Compared with the prior art, the invention has the following advantages and beneficial effects:
1. the invention uses a data synthesis method to realize data enhancement, which has better applicability and diversity compared with the traditional data enhancement method.
2. The data synthesized by the method can be applied to not only a target detection task but also an example segmentation task with higher difficulty.
3. The data synthesis method can be continuously used on the basis of the traditional data enhancement method, and achieves a better data enhancement effect.
4. The invention uses the image processing method, has very little calculation amount and high operation speed, and basically does not increase the time for training the network.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a schematic diagram of an original image.
Fig. 3 is a schematic diagram of an original image and a label.
Fig. 4 is a schematic diagram of a composite image.
Fig. 5 is a schematic diagram of a composite image and a label.
Detailed Description
The present invention will be described in further detail with reference to examples and drawings, but the present invention is not limited thereto.
As shown in fig. 1, the present embodiment provides a balanced example segmentation data synthesis method, including the following steps:
1) the data set uses a Microsoft COCO data set (COCO for short), and an object example library is constructed by using images and labels of the COCO data set.
2) Reading an image and a label in the COCO data set, as shown in fig. 2 and fig. 3, generating a foreground-background mask map for the image according to the label, and uniformly generating 10 × 10 candidate points according to the size of the image, including the following steps:
2.1) Generation of a foreground-background mask map
The method comprises the steps of obtaining an original image and a corresponding label, generating a single-channel image of the original image in length and width dimensions, setting each pixel value to be 0, namely, setting the single-channel image to be a foreground background mask image, obtaining a mask of an object in the label, setting the pixel value of the corresponding position of the foreground background mask image to be 1, setting an area with the final pixel value of 1 to be a foreground, and setting an area with the pixel value of 0 to be a background.
2.2) generating 10 × 10 candidate points
The length and width values of the original image are obtained, the length and the width are respectively and uniformly divided into 11 segments, namely 10 equally divided points are obtained, and the length and the width are respectively combined with the 10 equally divided points to finally obtain 10 multiplied by 10 candidate points.
3) Setting a paste size list, calculating the area taking 10 multiplied by 10 candidate points in the step 2) as the center and the foreground background mask image according to the set paste size list, selecting the area which is not overlapped with the foreground and adding the area into the paste area, and comprising the following steps:
3.1) set a paste size list
For relative equalization of paste sizes, a list of paste sizes is set to 150, 120, 90, 60, 30.
3.2) generating paste area
And sequentially selecting the size from the paste size list, taking the selected size as the side length of the square, taking the candidate point as the center, judging whether the candidate point meets the condition of no foreground, if not, adding the paste area and selecting the next size, and if so, selecting the next candidate point. Until the paste size list and all candidate points are traversed.
4) Sequentially processing the pasting areas in the step 3), randomly selecting a category from the object instance library in the step 1), randomly selecting an object from the category, if the size of the object is smaller than the size of the pasting area by a threshold value, randomly selecting an object again, scaling the size of the object to a size corresponding to the size of the pasting area after the size requirement is met, pasting the object to the pasting area, and adding a new label to the original label, as shown in fig. 4 and 5.
In conclusion, after the scheme is adopted, the invention provides a new method for data enhancement, data synthesis is used as an effective method for data enhancement, the problems of insufficient data quantity and unbalanced data can be effectively solved, the development of a deep learning technology is effectively promoted, and the method has an actual popularization value and is worthy of popularization.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.
Claims (5)
1. A balanced instance partition data synthesis method, comprising the steps of:
1) constructing an object instance library by using the image and the label of the original data set;
2) reading an image and a label in an original data set, generating a foreground background mask image for the image according to the label, and uniformly generating 10 x 10 candidate points according to the size of the image;
3) setting a pasting size list, calculating the area taking 10 multiplied by 10 candidate points in the step 2) as the center and the foreground background mask image according to the set pasting size list, and selecting the area which is not overlapped with the foreground and adding the area into the pasting area;
4) selecting an object from the object instance library in the step 1) through class balance, pasting the object to the pasting area in the step 3) after scaling, and updating the label.
2. The method as claimed in claim 1, wherein in step 1), an object instance library is constructed using the image of the original data set and the label, a mask corresponding to each object in the image is obtained by parsing the label, the object is separated from the original image through the mask, and the object is classified according to the category.
3. A balanced example segmentation data synthesis method according to claim 1, wherein in step 2), an image and a label in the original data set are read, a foreground-background mask map is generated for the image according to the label, and 10 × 10 candidate points are uniformly generated according to the size of the image, and the method comprises the following steps:
2.1) Generation of a foreground-background mask map
Acquiring an original image and a corresponding label, generating a single-channel image of the original image in length and width dimensions, setting each pixel value to be 0, namely, a foreground background mask image, acquiring a mask of an object in the label, setting the pixel value of the corresponding position of the foreground background mask image to be 1, setting an area with a final pixel value of 1 as a foreground, and setting an area with a pixel value of 0 as a background;
2.2) generating 10 × 10 candidate points
The length and width values of the original image are obtained, the length and the width are respectively and uniformly divided into 11 segments, namely 10 equally divided points are obtained, and the length and the width are respectively combined with the 10 equally divided points to finally obtain 10 multiplied by 10 candidate points.
4. The method as claimed in claim 1, wherein in step 3), a paste size list is set, and a region satisfying the requirement is selected from the region centered by 10 × 10 candidate points in step 2) according to the set paste size list and added to the paste region, and the method comprises the following steps:
3.1) set a paste size list
For relative balance of paste sizes, a paste size list is set to 150, 120, 90, 60, 30;
3.2) generating paste area
And sequentially selecting the size from the paste size list, taking the selected size as the side length of a square, taking the candidate point as the center, judging whether the candidate point meets the condition of no foreground, if no foreground exists, adding the paste area and selecting the next size, and if foreground exists, selecting the next candidate point until the paste size list and all candidate points are traversed.
5. The method as claimed in claim 1, wherein in step 4), the paste areas in step 3) are sequentially processed, a category is randomly selected from the object instance library in step 1), an object is randomly selected from the category, if the size of the object is smaller than the size of the paste area by a threshold, an object is randomly selected again, the size of the object is reduced to a size corresponding to the size of the paste area after meeting the size requirement, and then the object is pasted to the paste area, and a new label is added to the original label.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111245120.4A CN114066788B (en) | 2021-10-26 | 2021-10-26 | Balanced instance segmentation data synthesis method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111245120.4A CN114066788B (en) | 2021-10-26 | 2021-10-26 | Balanced instance segmentation data synthesis method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114066788A true CN114066788A (en) | 2022-02-18 |
| CN114066788B CN114066788B (en) | 2024-03-29 |
Family
ID=80235484
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111245120.4A Active CN114066788B (en) | 2021-10-26 | 2021-10-26 | Balanced instance segmentation data synthesis method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114066788B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114418898A (en) * | 2022-03-21 | 2022-04-29 | 南湖实验室 | Data enhancement method based on target overlapping degree calculation and self-adaptive adjustment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120063681A1 (en) * | 2001-05-04 | 2012-03-15 | Barry Sandrew | Minimal artifact image sequence depth enhancement system and method |
| CN111768415A (en) * | 2020-06-15 | 2020-10-13 | 哈尔滨工程大学 | Image instance segmentation method without quantization pooling |
| CN111832745A (en) * | 2020-06-12 | 2020-10-27 | 北京百度网讯科技有限公司 | Data augmentation method and device and electronic equipment |
| CN112132832A (en) * | 2020-08-21 | 2020-12-25 | 苏州浪潮智能科技有限公司 | Method, system, device and medium for enhancing image instance segmentation |
-
2021
- 2021-10-26 CN CN202111245120.4A patent/CN114066788B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120063681A1 (en) * | 2001-05-04 | 2012-03-15 | Barry Sandrew | Minimal artifact image sequence depth enhancement system and method |
| CN111832745A (en) * | 2020-06-12 | 2020-10-27 | 北京百度网讯科技有限公司 | Data augmentation method and device and electronic equipment |
| CN111768415A (en) * | 2020-06-15 | 2020-10-13 | 哈尔滨工程大学 | Image instance segmentation method without quantization pooling |
| CN112132832A (en) * | 2020-08-21 | 2020-12-25 | 苏州浪潮智能科技有限公司 | Method, system, device and medium for enhancing image instance segmentation |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114418898A (en) * | 2022-03-21 | 2022-04-29 | 南湖实验室 | Data enhancement method based on target overlapping degree calculation and self-adaptive adjustment |
| CN114418898B (en) * | 2022-03-21 | 2022-07-26 | 南湖实验室 | Data enhancement method based on target overlapping degree calculation and self-adaptive adjustment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114066788B (en) | 2024-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112801164B (en) | Training method, device, equipment and storage medium of target detection model | |
| CN108647585B (en) | Traffic identifier detection method based on multi-scale circulation attention network | |
| CN113033537B (en) | Method, apparatus, device, medium and program product for training a model | |
| CN110379020B (en) | Laser point cloud coloring method and device based on generation countermeasure network | |
| CN107506792B (en) | Semi-supervised salient object detection method | |
| CN110598698B (en) | Natural scene text detection method and system based on adaptive regional suggestion network | |
| CN112084923A (en) | Semantic segmentation method for remote sensing image, storage medium and computing device | |
| CN114998337B (en) | Scratch detection method, device, equipment and storage medium | |
| WO2020125062A1 (en) | Image fusion method and related device | |
| CN109523558A (en) | A kind of portrait dividing method and system | |
| CN110599453A (en) | Panel defect detection method and device based on image fusion and equipment terminal | |
| CN114332590B (en) | Joint perception model training method, joint perception method, device, equipment and medium | |
| CN110633717A (en) | Training method and device for target detection model | |
| CN114066788B (en) | Balanced instance segmentation data synthesis method | |
| CN111462164A (en) | Foreground segmentation method and data enhancement method based on image synthesis | |
| CN114359269A (en) | Virtual food box defect generation method and system based on neural network | |
| Sun et al. | Cfr-icl: Cascade-forward refinement with iterative click loss for interactive image segmentation | |
| CN111597845A (en) | Two-dimensional code detection method, device and equipment and readable storage medium | |
| CN115620081A (en) | Training method of target detection model, target detection method and device | |
| CN114419070A (en) | Image scene segmentation method, device, equipment and storage medium | |
| CN113139463A (en) | Method, apparatus, device, medium and program product for training a model | |
| CN113610856A (en) | Method and device for training image segmentation model and image segmentation | |
| CN113361535A (en) | Image segmentation model training method, image segmentation method and related device | |
| CN113343979B (en) | Method, apparatus, device, medium and program product for training a model | |
| US11741657B2 (en) | Image processing method, electronic device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |