CN111062862A - Color-based data enhancement method and system, computer device and storage medium - Google Patents
Color-based data enhancement method and system, computer device and storage medium Download PDFInfo
- Publication number
- CN111062862A CN111062862A CN201911318909.0A CN201911318909A CN111062862A CN 111062862 A CN111062862 A CN 111062862A CN 201911318909 A CN201911318909 A CN 201911318909A CN 111062862 A CN111062862 A CN 111062862A
- Authority
- CN
- China
- Prior art keywords
- image
- color
- texture
- module
- input image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000006243 chemical reaction Methods 0.000 claims abstract description 58
- 239000003086 colorant Substances 0.000 claims abstract description 44
- 230000011218 segmentation Effects 0.000 claims abstract description 41
- 238000010606 normalization Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims description 11
- 238000012935 Averaging Methods 0.000 claims description 2
- 230000002708 enhancing effect Effects 0.000 abstract description 3
- 238000013136 deep learning model Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 14
- 238000012549 training Methods 0.000 description 14
- 238000013528 artificial neural network Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000002372 labelling Methods 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000003416 augmentation Effects 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013434 data augmentation Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000012467 final product Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 238000001429 visible spectrum Methods 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Facsimile Image Signal Circuits (AREA)
Abstract
The embodiment of the disclosure provides a method, a system and a device for enhancing data based on color. The method comprises the following steps: acquiring an input image; performing ROI segmentation on the input image, and extracting a target object image; converting the target object image from a source color to a pure color image of one or more different target colors by color conversion based on RGB rules; converting the input image into a gray image with texture maintained; and adding the texture features of the gray level image into the pure color image, and performing texture normalization to generate one or more output images with different target colors. The data enhancement method provided by the embodiment of the disclosure expands the source color of the target object in the image into a plurality of target colors, thereby expanding the number of image samples under the condition of maintaining the quality of the original image, simultaneously reducing the time for generating a new image, and reducing the cost for collecting new data, thereby improving the performance of the deep learning model.
Description
Technical Field
The present disclosure relates to the field of computer vision technologies, and in particular, to a color-based data enhancement method and system, a computer device, and a storage medium.
Background
Currently, deep learning techniques based on neural networks have made tremendous progress in areas such as object classification, image recognition, text processing, facial recognition, autopilot, man-machine conversation, and so on. With the deepening of the neural network structure and the improvement of the algorithm, the demand of data samples for training the neural network is larger and larger. In particular, in the neural network related to the image, poor images or a small number of images may result in an ideal generalization capability of the trained neural network model only in the training domain, but not in practical use, thereby adversely affecting the working performance of the final product (e.g., a monitoring system).
Disclosure of Invention
In a first aspect, an embodiment of the present disclosure provides a color-based data enhancement method, including: acquiring an input image; performing ROI segmentation on the input image, and extracting a target object image; converting the target object image from a source color to a pure color image of one or more different target colors by color conversion based on RGB rules; converting the input image into a gray image with texture maintained; and adding the texture features of the gray level image into the pure color image, and performing texture normalization to generate one or more output images with different target colors.
In some embodiments, the performing ROI segmentation on the input image includes: based on the shape and color values of the input image, ROI segmentation is performed using a semantic segmentation method.
In some embodiments, the RGB rule-based color conversion includes: in the search space of the RGB rule, color conversion is achieved by adjusting the values of the individual color channels.
In some embodiments, the adding the texture feature of the grayscale image to the solid-color image for texture normalization includes: and carrying out superposition average processing on the pure color image and the gray level image.
In a second aspect, embodiments of the present disclosure provide a color-based data enhancement system, comprising: the image input module is used for acquiring an input image; the ROI segmentation module is used for carrying out ROI segmentation on the input image and extracting a target object image; the color conversion module is used for converting the target object image from a source color into one or more pure color images with different target colors through color conversion based on RGB rules; the gray level conversion module is used for converting the input image into a gray level image with texture maintained; the texture normalization module is used for adding the texture features of the gray level image into the pure color image, performing texture normalization and generating one or more output images with different target colors; and the image output module is used for outputting the output image.
In some embodiments, the ROI segmentation module is specifically configured to perform ROI segmentation using a semantic segmentation method based on a shape and a color value of the input image.
In some embodiments, the color conversion module is specifically configured to implement color conversion by adjusting values of each color channel in a search space of the RGB rule.
In some embodiments, the texture normalization module is specifically configured to perform an average process on the pure color image and the grayscale image.
In a third aspect, embodiments of the present disclosure provide a computer device comprising
A memory for non-transitory storage of computer readable instructions; and
a processor for executing the computer-readable instructions, when executed by the processor, performing the method of any of the above embodiments.
In a fourth aspect, embodiments of the present disclosure provide a storage medium that non-transitory stores computer-readable instructions that, when executed by a computer, may perform the method of any of the above embodiments.
In some possible embodiments of the present disclosure, the following benefits are achieved, at least in part:
1. the color of the target object does not need to be identified, and the color of the target object can be changed on the premise of reserving the texture of the original target object through the gray value;
2. through a search space based on RGB rules, the source color of a target object can be changed into a plurality of expected target colors;
3. under the condition of no need of manual interaction or labeling, a plurality of groups of image samples which can be used for training can be provided;
4. the output image provided based on the scheme disclosed by the invention can be close to a real image to the utmost extent, and is obviously suitable for training a deep neural network.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings used in the description of the embodiments will be briefly introduced below.
FIG. 1 is a schematic flow chart diagram of a method for color-based data enhancement provided by an embodiment of the present disclosure;
FIG. 2 is a diagram of ROI segmentation of an input image in an application scenario example of the present disclosure;
FIG. 3 is a schematic diagram of a pure color image obtained by conversion in an example application scenario of the present disclosure;
FIG. 4 is a schematic diagram of a pure color image obtained by conversion in an example application scenario of the present disclosure;
FIG. 5 is a schematic diagram of texture normalization in an application scenario example of the present disclosure;
FIG. 6 is a schematic mechanical diagram of a color-based data enhancement system provided by one embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of a computer device according to an embodiment of the present disclosure.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those skilled in the art, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only some embodiments of the present disclosure, not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
The terms "including" and "having," and any variations thereof, in the description and claims of this disclosure and the above-described drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
In neural networks, the quality of the samples is improved by manually labeling the samples for training, or manually reviewing machine-generated samples, or manually collecting new images that are used to augment the training samples. The manual labeling is to label and review data by people, and the process is tedious and time-consuming. Machine-generated data annotates data using a number of pre-trained models, which are then examined by humans, but pre-trained models tend to produce a large number of false positives making this approach more cumbersome in practice than manually annotating data. Furthermore, it is expensive to hire a annotator to perform manual review and collect new images for augmenting training samples.
In order to solve the above problem, the related art uses a data enhancement method to improve accuracy of use such as image classification. The data enhancement functions include: the training data volume is increased, and the generalization capability of the model is improved; noise data is increased, and robustness of the model is improved. Known data enhancement techniques include image flipping, rotation, scaling, expansion, cropping, translation, adding noise, and the like. The applicant has found that the above data enhancement methods are of limited effectiveness, some of which are time consuming and of low quality, and some of which are unnatural, e.g. the results of the augmentation operation on the image deviate too much from reality.
Referring to fig. 1, an embodiment of the present disclosure provides a method for enhancing data based on color, the method may include:
step 101: acquiring an input image;
step 102: performing ROI segmentation on the input image, and extracting a target object image;
step 103: converting the target object image from a source color to a pure color image of one or more different target colors by color conversion based on RGB rules;
step 104: converting the input image into a grayscale image maintaining texture;
step 105: adding the texture features of the gray level image into the pure color image, and performing texture normalization to generate one or more output images with different target colors;
step 106: and outputting the output image.
As described above, embodiments of the present disclosure provide a simple and effective method of color-based data enhancement (Dataaugmentation) that can be used to augment training images used in machine learning or deep learning. For example, assume that in the input original image (i.e., input image), the umbrella is the object of interest (i.e., object), and its original color (or source color) is gray, for example. The method provided by the present embodiment may change the color of the umbrella to a set of predefined target colors, such as black, white, orange, yellow, green, blue, purple, pink, red, brown, etc. The effect of this is to expand the input image with a gray umbrella into a plurality of new images (i.e. output images) of umbrellas with different target colors. Thus, the number of images is expanded by N times, wherein N is a natural number and depends on the number of preset target colors. If N is equal to 100, the number of images can be expanded by a factor of 100. In short, the technical solution disclosed in the present embodiment can expand the number of image samples while maintaining the original image quality, such as luminance, and at the same time, reduce the time for generating new images and the cost for collecting new data.
The following describes in detail the steps of the method according to an embodiment of the disclosure:
step 101: an input image is acquired.
The method of the disclosed embodiments may be performed by a computer system. The computer system includes an input module for acquiring single or multiple input images. The input image is an abbreviation of an input original image.
In embodiments of the present disclosure, an image may include one or more still images, and may also include a moving image such as a video frame.
Step 102: and carrying out ROI segmentation on the input image and extracting a target object image.
The challenge in generating a realistic augmentation of an original image or video frame is to determine the ROI (region of interest) because the image may contain a large number of different objects (or objects).
Optionally, different objects in the image may be determined by methods such as manual labeling or Semantic Segmentation (Semantic Segmentation), and different regions may be separated at a pixel level by a deep neural network. The present embodiments provide a technical means to accurately segment different targets within an image or video frame, e.g., different ROIs within an image may be determined based on the shape and color values of the image.
The computer system may include a ROI segmentation module that may determine the shape of an object at the pixel level within an image, and thus the ROI, using, for example, a semantic segmentation Convolutional Neural Network (CNN). Semantic segmentation convolutional neural networks may be, for example, by FCN semantic segmentation (e.g., U-net is one implementation of FCN semantic segmentation), SegNet, DeconvNet, Deeplab v3, and the like.
Further, the boundary of the segmentation can be better determined in combination with the superpixel. A small area formed by a series of pixel points with adjacent positions and similar characteristics such as color, brightness, texture and the like in the image forms a super pixel. Neighboring superpixels that are similar (e.g., average color values below a preset threshold) may be merged together based on the average color in each superpixel (e.g., comprising a group of pixels within an image). This process is repeated until no neighboring superpixels (or merged superpixels) have a weighted difference less than the threshold. The remaining superpixels after iterative merging may represent the ROI of the image.
Accurate identification of objects or regions in an image may not guarantee accurate augmented reality. Another challenge faced in this process is the normalization of the region boundaries. Semantic segmentation CNN can produce perfect region boundaries or generate smaller regions or larger regions, but can result in artifacts or noise around boundary pixels. To address the above problem, embodiments of the present disclosure may use poisson image interpolation or any other edge enhancement method to remove artifacts, one of which is to group or average similar pixels in the boundary, normalizing the boundary to its surrounding pixels.
Alternatively, the image may be processed by using an ROI Mask (region Mask, or image Mask, or Mask) to segment and mark the target region of interest or the target object image from the image. The output image obtained after the processing looks more real and natural.
Step 103: converting the target object image from a source color to a pure color image of one or more different target colors by color conversion based on RGB rules.
In general, color conversion requires defining the correspondence of a source color to a target color, and there are various methods for implementing color conversion such as 1 to 1, 1 to N, M to N, where M and N are natural numbers. In a 1 to 1 conversion, a single source color is converted to a single target color, which is less feasible if the source and target colors are time-varying. In the 1 to N conversion, a single source color is converted into N target colors, although the conversion is still limited to a single source color. In the M-to-N conversion, M source colors are converted into N target colors. However, the color conversions such as 1 to 1 and 1 to N, M to N described above all require the source color to be identified in advance, and the color identification of the source color may introduce extra noise in the whole conversion process, so the conversion process is not ideal.
In some embodiments of the present disclosure, an original image (or video frame) having a source color may be enhanced as realistically as possible by using a color model such as RGB, HSV, or the like. RGB (red, green and blue) refers to a color representation system used on computer displays. Red, green and blue may be combined in various proportions to form any color in the visible spectrum. HSV (hue, saturation, value) is an alternative representation of the RGB color model, which is suitable for use in the field of print production to be closer to human visual perception.
In some embodiments of the present disclosure, an accurate conversion method of a source color to a target color is provided. This conversion method is advantageous to overcome the above-mentioned limitations. In this embodiment, the color conversion operation may be implemented by a color conversion module in the computer system. The color conversion module may convert the object into a pure color image having a plurality of object colors through color conversion based on RGB rules without using color recognition.
The RGB rule-based color conversion means that color conversion is performed by using an RGB rule search space (RGB tree-based search space). In the search space of the RGB rule, the color channels are RGB, and color conversion can be realized by adjusting the values of their respective color channels to preset values, such as n1-n 6. For example, if the target color is desired to be red, all values of the target umbrella are set to R255, G0, and B0 to obtain a red umbrella. Likewise, if other target colors are desired, such as blue, the RGB values are modified accordingly. In this way, the source color is converted to any desired target color, e.g., N target colors, without the need to identify the target colors in the original image.
Step 104: converting the input image into a grayscale image that retains texture.
In order to maintain the texture of the final output image, in this step, the input original image is converted into a grayscale image, the conversion method may use common open source conversion software, such as cvtColor in OpenCV, and the converted grayscale image can maintain the texture characteristics of the original image. This step may be implemented by a grayscale conversion module in a computer system.
Step 105: and adding texture features of the gray-scale image into the pure-color image, and performing texture normalization (texture normalization) to generate one or more output images with different target colors.
The texture features of the original image are lost in the pure color image converted in step 103, the texture features of the original image are maintained in the gray image converted in step 104, texture normalization is performed in this step, the texture features of the gray image are added to the pure color image, and one or more output images with different target colors are generated. This step may be implemented by a texture normalization module in the computer system. The texture features of the grayscale image are added to the solid-color image, and for example, the solid-color image and the grayscale image may be subjected to an overlap-and-average process.
It should be noted that, assuming that the original image includes M source colors, the conversion into the grayscale image in step 104 is an M-1 conversion, and the conversion into the N target colors in step 103 is an M-N conversion, and after texture normalization in step 105, an output image is obtained, which is an M-1-N conversion method.
The M-1-N conversion method is beneficial to overcoming the limitations of the existing conversion method that the color identification of the source color is required. In the method, an original image obtained by an image input module may have a plurality of source colors, and a color conversion module may convert an object into a pure color image having a plurality of object colors through color conversion based on RGB rules without using color recognition.
As described above, in the embodiment of the present disclosure, based on the ROI segmentation module, the target image is extracted by a Mask segmentation method; the color conversion module converts an interested target ROI (region of interest), namely a target object image, into a plurality of different colors through a search space based on RGB (red, green and blue) rules; moreover, for the original image with any color input by the image input module, the gray scale conversion module converts the original image into a gray scale image, so that the gray scale image with rich texture still can be obtained; then, a plurality of output images having texture features and different target colors are generated by a texture normalization module.
It is understood that the above aspects of the embodiments of the present disclosure may be embodied in, for example, a computer device.
In order to better understand the technical solutions provided by the embodiments of the present disclosure, the following description is given by taking an implementation mode in a specific scenario as an example.
As shown in fig. 2, for a single or multiple input images acquired in step 101, ROI segmentation is performed in step 102 using a semantic segmentation method. Fig. 2 shows a way of ROI segmentation using UNet neural network, and a ROI Mask (region Mask) is shown as 201, and an object image is extracted through Mask processing.
As shown in fig. 3, the target object image is color-converted by the search space of the RGB rule in step 103. In its input, RGB represents red, green and blue, respectively, and n1-6 represents the different target colors formed after modifying the RGB values, with 301 representing the converted pure color image.
As shown in fig. 4, for the single or multiple input images acquired in step 101, the color image is converted into a grayscale image by a library such as OpenCV, PIL, etc. in step 104, and the converted grayscale image is shown in 401.
As shown in fig. 5, the pure color image 301 and the grayscale image 401 obtained in the above steps are normalized, and the grayscale image having texture features is added to the pure color image to provide the texture to the pure color image. In the texture normalization process, the grayscale image and the solid-color image may be subjected to a superposition averaging process to form a plurality of output images, and the converted grayscale images are output in step 106.
As can be seen from the above, in some possible embodiments of the present disclosure, a color-based data enhancement method is disclosed, which automatically converts a plurality of source colors into target colors and maintains texture features by using a grayscale channel and using an M-1-N conversion method, expands an original image into N output images, can improve the image number and image quality of training samples for machine learning or deep learning, and does not require human intervention or additional manpower to convert a plurality of source colors into target colors.
In order to better implement the above-mentioned aspects of the embodiments of the present disclosure, the following also provides related devices for implementing the above-mentioned aspects in a matching manner.
Referring to fig. 6, an embodiment of the present disclosure provides a color-based data enhancement system, which may include:
an image input module 601, configured to obtain a single or multiple input images;
an ROI segmentation module 602, configured to perform ROI segmentation on the input image and extract an object image;
a color conversion module 603, configured to convert the target object image from a source color to a pure color image of one or more different target colors through RGB rule-based color conversion;
a grayscale conversion module 604, configured to convert the input image into a grayscale image with retained texture;
a texture normalization module 605, configured to add texture features of the grayscale image to the pure color image, perform texture normalization, and generate one or more output images with different target colors;
an image output module 606, configured to output the output image.
Optionally, the ROI segmentation module 602 is specifically configured to perform ROI segmentation by using a semantic segmentation method based on the shape and the color value of the input image.
Optionally, the color conversion module 603 is specifically configured to implement color conversion by adjusting values of each color channel in a search space of an RGB rule.
Optionally, the texture normalization module 605 is specifically configured to perform an average processing on the pure color image and the gray level image.
Those skilled in the art will appreciate that the modules depicted in fig. 6 and in the above-described embodiments are functional entities, and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of a processor executing corresponding functional software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
For example, a processor may be a general-purpose logical operation device having data processing capabilities and/or program execution capabilities, such as a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Microprocessor (MCU), or the like, that execute computer instructions of corresponding functions to implement the corresponding functions. The computer instructions comprise one or more processor operations defined by an instruction set architecture corresponding to the processor, which may be logically embodied and represented by one or more computer programs.
For example, a processor may be a hardware entity, such as a field programmable logic array (FPGA) or an Application Specific Integrated Circuit (ASIC), with programmable functions to perform the respective functions.
For example, the processor may be a hardware circuit specifically designed to perform the corresponding function, such as a Tensor Processor (TPU) or a neural Network Processor (NPU), or the like.
It can be understood that the functions of each functional module of the color-based data enhancement system according to the embodiment of the present disclosure may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the relevant description in the foregoing method embodiment, which is not described herein again.
Embodiments of the present disclosure also provide a computer device, including:
a memory for non-transitory storage of computer readable instructions; and
a processor for executing the computer readable instructions, the computer readable instructions when executed by the processor performing the method of any of the preceding embodiments.
Referring to fig. 7, a specific implementation of the computer apparatus is provided. The computer device 700 of the present disclosure may include: a processor 701, a memory 702, a communication interface 703, a bus 704; the processor 701, the memory 702 and the communication interface 703 are in communication with each other through the bus 704; the communication interface 703 is used for receiving and transmitting data; the memory 702 is used for storing computer executable instructions, the processor 701 is connected to the memory 702 through the bus 704, and when the computer device is running, the processor 701 executes the computer executable instructions stored in the memory 702, so as to enable the computer device to execute the color-based data enhancement method according to the embodiment of fig. 1.
The disclosed embodiments also provide a storage medium, which stores non-transitory computer readable instructions, and when the computer readable instructions are executed by a computer, the method according to any one of the above embodiments can be executed.
In the embodiments of the present disclosure, the memory, the storage medium, and the like may be a local physical storage device, and may also be a virtual storage device connected by remote communication, such as a VPS, a cloud storage, and the like.
It will be readily appreciated that the computer devices may communicate from a server, cloud, or the like to obtain samples of images, videos, etc. for use in performing the present disclosure, and that such data may also be shared across a network of multiple computer devices. The communication connection may be a wireless network, a wired network, and/or any combination of wireless and wired networks. The network may include a local area network, the Internet, a telecommunications network, an Internet of Things (Internet of Things) based on the Internet and/or a telecommunications network, and/or any combination thereof, and/or the like. The wired network may communicate using twisted pair, coaxial cable, or fiber optic transmission, for example, and the wireless network may communicate using a wireless wide area communication network (WWAN), bluetooth, Zigbee, or Wi-Fi, for example.
In summary, the embodiment of the present disclosure discloses a method, a system, and an apparatus for enhancing data based on color, which have the following advantages: 1. the color of the target object does not need to be identified, and the color of the target object can be changed on the premise of reserving the texture of the original target object through a gray value (gray channel); 2. through a search space based on RGB rules, the source color of a target object can be changed into a plurality of expected target colors; 3. under the condition of no need of manual interaction or labeling, a plurality of groups of image samples which can be used for training can be provided; 4. the output image provided based on the scheme disclosed by the invention can be close to a real image to the utmost extent, and is obviously suitable for training a deep neural network.
The technical scheme disclosed by the invention is simple and effective, and can amplify the source color of the target object in the training image used for deep learning into multiple target colors, so that the number of image samples can be expanded under the condition of keeping the quality of the original image, and meanwhile, the time for generating a new image and the cost for collecting new data are reduced, thereby improving the performance of the deep learning model. By adopting the technical scheme, the number of the training images can be simply amplified to be N times, wherein N is the number of the preset target colors.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to the related descriptions of other embodiments.
It is noted that while for simplicity of explanation, the foregoing method embodiments have been described as a series of acts or combination of acts, it will be appreciated by those skilled in the art that the present disclosure is not limited by the order of acts, as some steps may, in accordance with the present disclosure, occur in other orders and concurrently. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required for the disclosure.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The functional units in the embodiments of the present disclosure may be integrated into one processing unit, may exist alone physically, or may be integrated into one unit from two or more units. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. The storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable flash disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The present disclosure has been described in detail, and the principles and embodiments of the present disclosure are explained herein using specific embodiments, which are merely used to help understand the method and the core concept of the present disclosure; meanwhile, for a person skilled in the art, based on the idea of the present disclosure, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present disclosure should not be construed as a limitation to the present disclosure.
Claims (10)
1. A method for color-based data enhancement, comprising:
acquiring an input image;
performing ROI segmentation on the input image, and extracting a target object image;
converting the target object image from a source color to a pure color image of one or more different target colors by color conversion based on RGB rules;
converting the input image into a gray image with texture maintained;
and adding the texture features of the gray level image into the pure color image, and performing texture normalization to generate one or more output images with different target colors.
2. The method of claim 1, wherein the performing ROI segmentation on the input image comprises:
based on the shape and color values of the input image, ROI segmentation is performed using a semantic segmentation method.
3. The method as claimed in claim 1, wherein the RGB rule based color conversion comprises:
in the search space of the RGB rule, color conversion is achieved by adjusting the values of the individual color channels.
4. The method of claim 1, wherein the adding texture features of the grayscale image to the solid-color image for texture normalization comprises:
and carrying out superposition average processing on the pure color image and the gray level image.
5. A color-based data enhancement system, comprising:
the image input module is used for acquiring an input image;
the ROI segmentation module is used for carrying out ROI segmentation on the input image and extracting a target object image;
the color conversion module is used for converting the target object image from a source color into one or more pure color images with different target colors through color conversion based on RGB rules;
the gray level conversion module is used for converting the input image into a gray level image with texture maintained;
the texture normalization module is used for adding the texture features of the gray level image into the pure color image, performing texture normalization and generating one or more output images with different target colors;
and the image output module is used for outputting the output image.
6. The system of claim 5,
the ROI segmentation module is specifically configured to perform ROI segmentation using a semantic segmentation method based on a shape and a color value of the input image.
7. The system of claim 5,
the color conversion module is specifically configured to implement color conversion by adjusting values of each color channel in a search space of an RGB rule.
8. The system of claim 5,
the texture normalization module is specifically configured to perform superposition averaging processing on the pure color image and the gray level image.
9. A computer device, comprising
A memory for non-transitory storage of computer readable instructions; and
a processor for executing the computer-readable instructions, which when executed by the processor perform the method of any of claims 1-4.
10. A storage medium non-transitory storing computer-readable instructions that, when executed by a computer, may perform the method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911318909.0A CN111062862A (en) | 2019-12-19 | 2019-12-19 | Color-based data enhancement method and system, computer device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911318909.0A CN111062862A (en) | 2019-12-19 | 2019-12-19 | Color-based data enhancement method and system, computer device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111062862A true CN111062862A (en) | 2020-04-24 |
Family
ID=70300810
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911318909.0A Pending CN111062862A (en) | 2019-12-19 | 2019-12-19 | Color-based data enhancement method and system, computer device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111062862A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113642372A (en) * | 2020-04-27 | 2021-11-12 | 百度（美国）有限责任公司 | Method and system for recognizing object based on gray-scale image in operation of autonomous driving vehicle |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130236091A1 (en) * | 2012-03-06 | 2013-09-12 | Apple Inc. | Method and interface for converting images to grayscale |
| CN106898035A (en) * | 2017-01-19 | 2017-06-27 | 博康智能信息技术有限公司 | A kind of dress ornament sample set creation method and device |
| CN108492343A (en) * | 2018-03-28 | 2018-09-04 | 东北大学 | A kind of image combining method for the training data expanding target identification |
| CN109544529A (en) * | 2018-11-19 | 2019-03-29 | 南京信息工程大学 | Pathological image data enhancement methods towards deep learning model training and study |
| CN110176054A (en) * | 2018-02-14 | 2019-08-27 | 辉达公司 | For training the generation of the composograph of neural network model |
| CN110322396A (en) * | 2019-06-19 | 2019-10-11 | 怀光智能科技（武汉）有限公司 | A kind of pathological section color method for normalizing and system |
| CN110458173A (en) * | 2019-08-16 | 2019-11-15 | 京东数字科技控股有限公司 | Method and apparatus for generating article color value |
-
2019
- 2019-12-19 CN CN201911318909.0A patent/CN111062862A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130236091A1 (en) * | 2012-03-06 | 2013-09-12 | Apple Inc. | Method and interface for converting images to grayscale |
| CN106898035A (en) * | 2017-01-19 | 2017-06-27 | 博康智能信息技术有限公司 | A kind of dress ornament sample set creation method and device |
| CN110176054A (en) * | 2018-02-14 | 2019-08-27 | 辉达公司 | For training the generation of the composograph of neural network model |
| CN108492343A (en) * | 2018-03-28 | 2018-09-04 | 东北大学 | A kind of image combining method for the training data expanding target identification |
| CN109544529A (en) * | 2018-11-19 | 2019-03-29 | 南京信息工程大学 | Pathological image data enhancement methods towards deep learning model training and study |
| CN110322396A (en) * | 2019-06-19 | 2019-10-11 | 怀光智能科技（武汉）有限公司 | A kind of pathological section color method for normalizing and system |
| CN110458173A (en) * | 2019-08-16 | 2019-11-15 | 京东数字科技控股有限公司 | Method and apparatus for generating article color value |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113642372A (en) * | 2020-04-27 | 2021-11-12 | 百度（美国）有限责任公司 | Method and system for recognizing object based on gray-scale image in operation of autonomous driving vehicle |
| CN113642372B (en) * | 2020-04-27 | 2024-02-20 | 百度（美国）有限责任公司 | Method and system for recognizing object based on gray image in operation of autonomous driving vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10614574B2 (en) | Generating image segmentation data using a multi-branch neural network | |
| CN112614060A (en) | Method and device for rendering human face image hair, electronic equipment and medium | |
| CN108280426B (en) | Dark light source expression identification method and device based on transfer learning | |
| CN107292307B (en) | Automatic identification method and system for inverted Chinese character verification code | |
| CN111275034B (en) | Method, device, equipment and storage medium for extracting text region from image | |
| CN107169508B (en) | Cheong-sam image emotion semantic recognition method based on fusion features | |
| CN109359527B (en) | Hair region extraction method and system based on neural network | |
| Li et al. | Globally and locally semantic colorization via exemplar-based broad-GAN | |
| Feng et al. | URNet: A U-Net based residual network for image dehazing | |
| CN113411550B (en) | Video coloring method, device, equipment and storage medium | |
| CN108985201A (en) | Image processing method, medium, device and calculating equipment | |
| CN108647696B (en) | Picture color value determining method and device, electronic equipment and storage medium | |
| WO2018151043A1 (en) | Image processing method and computer program | |
| CN108711160A (en) | A kind of Target Segmentation method based on HSI enhancement models | |
| CN116681636A (en) | Light infrared and visible light image fusion method based on convolutional neural network | |
| Cai et al. | Perception preserving decolorization | |
| CN113052783A (en) | Face image fusion method based on face key points | |
| CN112839167B (en) | Image processing method, device, electronic equipment and computer readable medium | |
| CN115967823A (en) | Video cover generation method and device, electronic equipment and readable medium | |
| CN114359030B (en) | Synthesis method of face backlight picture | |
| CN113610720A (en) | Video denoising method and device, computer readable medium and electronic device | |
| CN111062862A (en) | Color-based data enhancement method and system, computer device and storage medium | |
| CN115063800B (en) | Text recognition method and electronic equipment | |
| Wu et al. | Color transfer with salient features mapping via attention maps between images | |
| Jimenez-Arredondo et al. | Multilevel color transfer on images for providing an artistic sight of the world |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200424 |
|
| WD01 | Invention patent application deemed withdrawn after publication |