CN108021881B

CN108021881B - Skin color segmentation method, device and storage medium

Info

Publication number: CN108021881B
Application number: CN201711249162.9A
Authority: CN
Inventors: 金晓东
Original assignee: Tencent Cyber Tianjin Co Ltd
Current assignee: Tencent Cyber Tianjin Co Ltd
Priority date: 2017-12-01
Filing date: 2017-12-01
Publication date: 2023-09-01
Anticipated expiration: 2037-12-01
Also published as: CN108021881A

Abstract

The embodiment of the invention discloses a skin color segmentation method, a skin color segmentation device and a storage medium; according to the embodiment of the invention, the scatter diagrams can be drawn for the image to be segmented under the preset multiple color spaces respectively, then the scatter diagrams under each color space are analyzed, and at least one skin color judgment algorithm is selected from a preset algorithm set based on the scatter diagram characteristics obtained by analysis so as to segment the skin color of the image to be segmented; the scheme can improve the accuracy of identification and the skin color segmentation effect.

Description

Skin color segmentation method, device and storage medium

Technical Field

The present invention relates to the field of communications technologies, and in particular, to a skin color segmentation method, apparatus, and storage medium.

Background

In face recognition, as well as in some other image processing applications, skin tone segmentation is a paramount part of this. The existing skin color segmentation scheme is generally judged by traversing all pixels of an image and then analyzing the similarity of the color and the skin color of the pixels one by one. For example, taking a certain pixel point as an example, three color components of the pixel point in a three primary color (RGB, red, green and Blue) color space can be obtained, a euclidean distance (i.e. a sum of squares of differences between three colors and skin color values) is calculated according to the three color components, if the euclidean distance is lower than a preset threshold value, the pixel is determined to be skin, if the euclidean distance is higher than the preset threshold value, the pixel is determined not to be skin, and the corresponding skin segmentation map can be output after all pixels of the image are traversed by analogy.

In the research and practice process of the prior art, the inventor of the present invention finds that, because objects similar to skin colors are likely to appear in the background of the image, if the preset threshold is set to be not high enough, points on the objects are very easy to be determined as skin, and if the preset threshold is set to be high, the skin in some figures cannot be identified, so that the identification accuracy of the prior art is low and the segmentation effect is not good.

Disclosure of Invention

The embodiment of the invention provides a skin color segmentation method, a skin color segmentation device and a storage medium, which can improve the accuracy of identification and improve the skin color segmentation effect.

The embodiment of the invention provides a skin color segmentation method, which is characterized by comprising the following steps:

acquiring an image to be segmented;

respectively drawing a scatter diagram for the image to be segmented under the preset multiple color spaces;

analyzing the scatter diagrams in each color space to obtain scatter diagram features;

selecting at least one skin color judgment algorithm from a preset algorithm set based on the scatter diagram features;

and performing skin color segmentation on the image to be segmented according to the selected skin color judgment algorithm.

The embodiment of the invention also provides a skin color segmentation device, which comprises:

An acquisition unit for acquiring an image to be segmented;

the drawing unit is used for drawing a scatter diagram for the image to be segmented under the preset multiple color spaces respectively;

the analysis unit is used for analyzing the scatter diagrams in each color space to obtain the scatter diagram characteristics;

a selecting unit, configured to select at least one skin color judging algorithm from a preset algorithm set based on the scatter diagram feature;

and the segmentation unit is used for carrying out skin color segmentation on the image to be segmented according to the selected skin color judgment algorithm.

The embodiment of the invention also provides a storage medium, which stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor to execute the steps in any skin color segmentation method provided by the embodiment of the invention.

According to the embodiment of the invention, the scatter diagrams can be drawn for the image to be segmented under the preset multiple color spaces respectively, then the scatter diagrams under each color space are analyzed, and at least one skin color judgment algorithm is selected from a preset algorithm set based on the scatter diagram characteristics obtained by analysis so as to segment the skin color of the image to be segmented; because the scheme can refer to the scatter diagrams of a plurality of color spaces to comprehensively analyze the skin color similarity relation in the image to be segmented, and flexibly select the skin color judgment algorithm according to the skin color similarity relation, compared with the scheme based on the specific color space and the single skin color judgment algorithm in the prior art, the scheme has more accurate screening and resolving power, greatly improves the identification accuracy and improves the skin color segmentation effect.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1a is a schematic view of a skin color segmentation method according to an embodiment of the present invention;

fig. 1b is a flowchart of a skin color segmentation method according to an embodiment of the present invention;

fig. 2a is another flow chart of a skin color segmentation method according to an embodiment of the present invention;

FIG. 2b is a schematic diagram of skin color segmentation effect according to an embodiment of the present invention;

fig. 3a is a schematic structural diagram of a skin color segmentation apparatus according to an embodiment of the present invention;

fig. 3b is another schematic structural diagram of a skin color segmentation apparatus according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a network device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.

The embodiment of the invention provides a skin color segmentation method, a skin color segmentation device and a storage medium.

The skin color segmentation device can be integrated in network equipment, such as a terminal or a server.

For example, taking the skin color segmentation device integrated in a network device as an example, referring to fig. 1a, after the network device obtains an image to be segmented, the network device may draw a scatter diagram for the image to be segmented under a preset multiple color spaces, for example, under color spaces of RGB, YCbCr, lab, HSV, CMYK and the like, respectively, analyze the scatter diagrams under the respective color spaces (for example, RGB scatter diagrams, YCbCr scatter diagrams, lab scatter diagrams, HSV scatter diagrams and CMYK scatter diagrams) to obtain a scatter diagram feature, then select at least one skin color determination algorithm from the preset algorithm set based on the scatter diagram feature, and perform skin color segmentation on the image to be segmented according to the selected skin color determination algorithm. For example, if the scatter plot feature is "circular", a skin tone determination algorithm corresponding to the "circular" is selected to calculate the similarity between each pixel point in the image to be segmented and the skin tone, if the scatter plot feature is "parallelogram", a skin tone determination algorithm corresponding to the "parallelogram" is selected to calculate the similarity between each pixel point in the image to be segmented and the skin tone, and so on, then a gray value is calculated according to the similarity, and a skin tone segmentation gray map corresponding to the segmented image is generated according to the gray value.

The following will describe in detail. The order of the following examples is not to be construed as a preferred limitation of the examples.

Embodiment 1,

Embodiments of the present invention will be described in terms of skin tone segmentation apparatus that may be integrated in a network device, such as a terminal or server, which may include devices such as a cell phone, tablet, notebook, and/or personal computer (PC, personal Computer).

A skin tone segmentation method comprising: obtaining an image to be segmented, drawing a scatter diagram for the image to be segmented under a plurality of preset color spaces, analyzing the scatter diagram under each color space to obtain scatter diagram features, selecting at least one skin color judgment algorithm from a preset algorithm set based on the scatter diagram features, and performing skin color segmentation on the image to be segmented according to the selected skin color judgment algorithm.

As shown in fig. 1b, the specific flow of the skin color segmentation method may be as follows:

101. and acquiring an image to be segmented.

For example, the image to be segmented may be read from the local (i.e. skin color segmentation device) or the image to be segmented sent by another device may be received.

The formats of the image to be segmented may include BitMaP (BMP, bitMaP), joint photographic experts group (JPEG, joint Photographic Experts Group), tag image file format (TIFF, tag Image File Format), RAW (a lossless compression format), and personal computer exchange (PCX, personal Computer exchange), among others.

102. And drawing a scatter diagram for the image to be segmented under the preset multiple color spaces respectively. For example, the following may be specifically mentioned:

(1) And classifying the pixel points in the image to be segmented.

For example, the specific view may perform face recognition and human body recognition on the image to be segmented based on the deep learning model, so as to obtain a recognition result, and divide the pixels in the image to be segmented into a plurality of preset categories according to the recognition result.

The deep learning model may include, among other things, convolutional neural networks (CNN, convolutional Neural Networks), deep belief networks (DBN, deep Belief Network), recurrent neural networks (RNN, recurrent Neural Network), recurrent neural tensor networks (RNTN, recursive Neural Tensor Network), and generate antagonism networks (GAN, generative Adversarial Networks), among others.

The preset categories may be set according to the requirements of practical applications, for example, may be divided into pixels of the skin on the face (excluding eyes and mouth) obtained by face detection, pixels of the human body (excluding the face) obtained by human body detection, and pixels of the image to be segmented other than the human body and the face, that is, the background, and so on. For convenience of description, in the embodiment of the present invention, the pixels (excluding eyes and mouth) of the skin on the face obtained by face detection are called as face pixels, and are classified into face pixel categories; the pixels of the human body (without human faces) obtained from human body detection are called human body pixels and are classified into human body pixels; and (3) the pixel points except the human body and the human face in the image to be segmented are called background pixel points and are classified into background pixel point categories.

If the classification is to be a face pixel class, a human body pixel class and a background pixel class, the step of dividing the pixels in the image to be segmented into a plurality of preset classes according to the recognition result is specifically as follows:

and dividing the pixel points in the image to be segmented into a human face pixel point category, a human body pixel point category, a background pixel point category and the like according to the identification result.

It should be noted that the above three categories are merely examples, and it should be understood that other dividing methods may be used, which are not described herein.

(2) And respectively drawing a scatter diagram under a plurality of preset color spaces according to the classification result.

For example, specifically, all pixel points in the image to be segmented can be mapped to a preset plurality of color spaces according to the classification result, and drawing is performed according to the color distribution in the plurality of color spaces according to the mapping result, so as to obtain a scatter diagram in each color space.

For example, the preset multiple color spaces include a trichromatic color space (RGB, red, green, blue), YCbCr (one color space commonly used in digital photographing systems, Y is a so-called lumen, which represents the light density and is nonlinear, a gamma correction (gamma correction) encoding process is used, CB and CR are the density offset components of Blue and Red), a color model (Lab, L represents the brightness), a represents the range from Magenta to Green, b represents the range from Yellow to Blue), a hexagonal pyramid model (HSV, hue, saturation, value), and a printing color mode (CMYK, wherein C represents Cyan, M represents Magenta, Y represents Yellow, K represents black), and then the step "map all pixels in the image to be segmented into the preset multiple color spaces according to the result of the gamma correction, respectively, and" draw a specific color space according to the following color distribution according to the multiple color map "according to the following classification result:

All pixel points in the image to be segmented can be mapped into a RGB, YCbCr, lab, HSV color space, a CMYK color space and the like according to the classification result, and then the image is drawn according to the RGB mapping result and the color distribution to obtain a scatter diagram under RGB; drawing according to the mapping result in YCbCr and the color distribution to obtain a scatter diagram under YCbCr; drawing according to the mapping result at the Lab and the color distribution to obtain a scatter diagram under the Lab; drawing according to the mapping result of the HSV and the color distribution to obtain a scatter diagram under the HSV; and drawing according to the mapping result in CMYK and the color distribution to obtain a scatter diagram in CMYK.

Various mapping modes can be adopted, for example, each component in each color space can be combined in pairs, quantization mapping can be performed respectively, then color values obtained by the quantization mapping are mapped onto a preset matrix, for example, a 256×256 matrix, all pixel points in the image to be segmented are counted on the matrix according to the classification result, and a mapping result of the corresponding color space can be obtained.

103. And analyzing the scatter diagram under each color space to obtain the scatter diagram characteristics.

For example, a preset computer vision algorithm may be used to perform basic geometric fitting on the scatter plots in each color space, and the scatter plot features of each scatter plot are determined according to the fitting result.

The computer vision algorithm may be set according to the requirements of practical applications, for example, openCV (a cross-platform computer vision algorithm) or the like. While the basic geometry may include circular, parallelogram, and triangle geometries.

Taking a basic geometric shape specifically as a circle, a parallelogram and a triangle as an example, if the fitting result is approximately a "circle", the scatter diagram is characterized as a "circle"; if the fitting result is approximate to a parallelogram, the scatter diagram is characterized as the parallelogram; if the fitting result is approximately "triangle", the scatter plot features "triangle", and so on.

Optionally, because the scatter plots in some color spaces are more scattered in most cases and are difficult to fit, in order to reduce unnecessary calculation overhead and improve processing efficiency, before performing the basic geometric fitting, the scatter plot may be screened, that is, before the step of performing the basic geometric fitting on the scatter plots in each color space by using a preset computer vision algorithm, the skin color segmentation method may further include:

And screening the scatter diagrams in each color space according to preset screening conditions to obtain screened scatter diagrams in each color space.

Then, the step of performing the basic geometric fitting on the scatter diagram under each color space by using a preset computer vision algorithm may specifically be: and adopting a preset computer vision algorithm to perform basic geometric shape fitting on the screened scatter diagram in each color space.

The screening condition may be set according to the requirements of practical applications, for example, in YCbCr color space, each scatter plot related to the Y component may not be analyzed, for example, in Lab color space, each scatter plot related to the L analysis may not be analyzed, in HSV color space, only the scatter plot related to the H component may be analyzed, in CMYK color space, only the scatter plot related to the C component may be analyzed, and so on.

104. At least one skin tone decision algorithm is selected from a set of preset algorithms based on the scatter plot features.

For example, specific preset configuration information may be obtained, where the configuration information stores a one-to-one correspondence between the scatter plot features and the skin tone determination algorithms, and according to the correspondence, a corresponding skin tone determination algorithm is selected for each scatter plot feature from the preset algorithm set.

The algorithm set may be set according to the actual application requirement, for example, the algorithm set may include a skin color determination algorithm corresponding to "circle", a skin color determination algorithm corresponding to "parallelogram", a skin color determination algorithm corresponding to "triangle", and so on.

105. The skin color segmentation is performed on the image to be segmented according to the selected skin color judgment algorithm, for example, the following specific steps are adopted:

(1) And calculating the similarity between each pixel point in the image to be segmented and the skin color according to the selected skin color judging algorithm.

For example, a threshold value in the selected skin color determination algorithm may be specifically set according to the scatter diagram feature, the selected skin color determination algorithm and the set threshold value are used as variable parameters to be transmitted into a primitive shader of a preset graphics program interface, the texture mapping processing of the graphics program interface is performed on the image to be segmented, the texture sampling is performed on the image to be segmented after the texture mapping processing through the primitive shader, and the similarity between each pixel point in the image to be segmented and the skin color is calculated according to the sampling result.

The graphical program interface may be set according to the requirements of the practical application, for example, may be specifically an open graphics library (OpenGL, open Graphics Library).

(2) And calculating a gray value according to the similarity, and generating a skin color segmentation gray map corresponding to the segmentation image according to the gray value.

Since there may be multiple skin tone determination algorithms, there may be multiple corresponding similarities for a portion of the pixels in the image to be segmented (a corresponding similarity calculation result may be obtained by using one skin tone determination algorithm), so a functional relationship between the multiple similarities and the gray value may be established in advance, and then the gray value may be calculated according to the similarities based on the functional relationship.

For example, when the plurality of similarities are all higher than a preset value, the gray value may be set to 0% (black), otherwise, when any one of the plurality of similarities is lower than the preset value, the gray value may be set to 100% (black), or the like, or alternatively, may be set to other values, which are not described herein.

As can be seen from the foregoing, in this embodiment, a scatter diagram may be drawn for an image to be segmented under a preset plurality of color spaces, then the scatter diagram under each color space is analyzed, and at least one skin color judgment algorithm is selected from a preset algorithm set based on the scatter diagram feature obtained by the analysis, so as to perform skin color segmentation on the image to be segmented; because the scheme can refer to the scatter diagrams of a plurality of color spaces to comprehensively analyze the skin color similarity relation in the image to be segmented, and flexibly select the skin color judgment algorithm according to the skin color similarity relation, compared with the scheme based on the specific color space and the single skin color judgment algorithm in the prior art, the scheme has more accurate screening and resolving power, greatly improves the identification accuracy and improves the skin color segmentation effect.

Embodiment II,

The method described in the previous examples is described in further detail below by way of example.

In this embodiment, the specific integration of the skin color segmentation apparatus in a network device, which may be a terminal or a server, will be described as an example.

As shown in fig. 2a, a specific procedure of the skin color segmentation method may be as follows:

201. the network device acquires an image to be segmented.

For example, the image to be segmented may be read from the local (i.e. the network device) or the image to be segmented sent by another device may be received.

The format of the image to be segmented may include BMP, JPEG, TIFF, RAW, PCX, and the like.

202. And the network equipment performs face recognition and human body recognition on the image to be segmented based on the deep learning model to obtain a recognition result.

For example, the network device may perform face recognition on the image to be segmented based on the deep learning model to obtain pixels (including glasses and mouth) on the face, perform human body recognition on the image to be segmented based on the deep learning model to obtain pixels (without face) of the human body, and so on.

The deep learning model may include CNN, DBN, RNN, RNTN, GAN, and the like, among others.

203. The network device divides the pixels in the image to be segmented into a plurality of preset categories according to the identification result, and then step 204 is executed.

The preset categories may be set according to requirements of practical applications, for example, if the preset categories are divided into three categories of face pixel points, human body pixel points and background pixel points, at this time, the network device may divide the pixel points in the image to be segmented into the face pixel point category, the human body pixel point category and the background pixel point category according to the recognition result, where definitions of the three categories may be as follows:

(1) A face pixel point category;

the face pixel point class refers to the pixel points of the skin on the face obtained by face detection, that is, the pixel points on the face excluding eyes and mouth are excluded, and in the embodiment of the present invention, the class a point is also called.

(2) Human body pixel point category;

the human body pixel point category refers to a human body pixel point (without a human face) obtained by human body detection, and is also called a B-type point in the embodiment of the present invention.

(3) Background pixel point category;

the background pixel point category refers to the pixels in the image to be segmented except for the human body pixel point and the human face pixel point, and is also called as a class-C point in the embodiment of the invention.

204. The network equipment maps all pixel points in the image to be segmented into a plurality of preset color spaces according to the classification result, and draws the pixel points according to the color distribution in the plurality of color spaces according to the mapping result to obtain a scatter diagram in each color space.

For example, the network device may specifically perform quantization mapping on each component in each color space after combining each component in each color space, map color values obtained by quantization mapping to a preset matrix, then respectively perform statistics on all pixels in the image to be segmented according to the classification result (different types of pixels correspond to different color values) on the matrix, and draw a color distribution diagram according to the statistics result (i.e., mapping result), so as to obtain a scatter diagram under each color space.

The following will take the preset color spaces including RGB, YCbCr, lab, HSV and CMYK as examples, which may be specifically as follows:

(1)RGB；

since the color space RGB mainly includes three basic colors of red (R), green (G), and blue (B), and other colors such as cyan, yellow, magenta, and the like can be obtained by superimposing R, G and B of different intensities, R, G and B in the color space RGB can be combined two by two as components, for example, r× G, R ×b, and g×b, and the like, respectively.

(2)YCbCr；

In the color space YCbCr, Y refers to the luminance component, cb refers to the blue chrominance component, and Cr refers to the red chrominance component. YCgCr is an improvement of YCbCr, which further proposes a component "Cg", which is the difference between the green component G and the luminance Y, and it should be noted that, in the embodiment of the present invention, the color space YCbCr includes YCgCr, and therefore, the component "Cg" may also be considered when performing component combination. That is, Y, cb, cr, and Cg in the color space YCbCr may be two by two, for example, y×cb, y×cr, y×cg, cb×cr, cb×cg, and cr×cg.

(3)Lab；

The color space Lab is the human perception of color, and the values in Lab describe all colors that a normal-vision human can see. In Lab, L refers to luminance (luminance), which ranges from 0 to 100; a represents a range from magenta to green, and b represents a range from yellow to blue. Therefore, L, a and b in Lab can be taken as components, respectively, and combined two by two, for example, l×a, l×b, and a×b.

(4)HSV；

In the color space HSV, three parameters, hue (H), saturation (S) and brightness (V), are mainly included, and thus, the three parameters H, S and V can be combined two by two, for example, h× S, H ×v and s×v, respectively.

(5)CMYK；

CMYK is a color pattern used in color printing, which uses the principle of three primary colors of coloring material mixed with black ink to form a so-called "full-color printing" by mixing and superimposing four colors in total. Wherein, four standard colors are: c is Cyan (Cyan), M is Magenta (Magenta), also known as "Magenta", Y is Yellow (Yellow), and K is a keyplate (black), which is primarily black. Thus, C, M, Y and K can be combined two by two as components to give: c× M, C × Y, C × K, M × Y, M ×k, and y×k.

Experiments show that in the component combinations, the scatter diagrams of some component combinations are more dispersed, and the reference value is not high, for example, in the Lab color space, the L component has little effect on distinguishing 'whether skin is or not', so that the L component can be omitted, in the YCbCr color space, the Y component can be omitted, in the CMYK color space, the K component can be omitted, and the like. Thus, alternatively, to increase the mapping effect, and to reduce the consumption of computing resources, only some of the component combinations may be employed, e.g., see table one.

Table one:

color space	Combination relation 1	Combination relation 2	Combination relation 3
				RGB	R×G	R×B	G×B
YCbCr	Cb×Cr	Cb×Cg	Cr×Cg
				Lab	L×a	L×b	a×b
HSV	H×S	H×V	S×V
				CMYK	C×M	C×Y	M×Y

The quantization mapping can be performed on each color space according to the component combinations of each color space, such as the component combinations of each color space shown in table one, the color values obtained by the quantization mapping are mapped to a preset matrix, such as a 256×256 matrix, then statistics is performed on all pixel points in the image to be segmented according to the classification result on the matrix, and a color distribution diagram is drawn according to the statistics result (i.e., the mapping result), so as to obtain a scatter diagram under each color space. For example, taking a matrix mapped to 256×256 as an example, the following may be specifically mentioned:

for the color space RGB, quantization mapping can be specifically performed on the component combinations r× G, R ×b and g×b, color values obtained by the quantization mapping are mapped into a matrix of 256×256, then statistics is performed on all pixel points in the image to be segmented on the matrix according to the classification result, and a color distribution diagram is drawn according to the statistics result, so that a scatter diagram under RGB can be obtained.

For the color space YCbCr, specifically, quantization mapping may be performed on the component combinations cb×cr, cb×cg and cr×cg, and color values obtained by the quantization mapping are mapped into a matrix of 256×256, then, on the matrix, statistics is performed on all pixel points in the image to be segmented according to the classification result, and a color distribution diagram is drawn according to the statistics result, so as to obtain a scatter diagram under YCbCr.

For the color space Lab, quantization mapping can be specifically performed on the component combinations L×a, L×b and a×b, color values obtained by the quantization mapping are mapped into a matrix of 256×256, then statistics is performed on all pixel points in the image to be segmented on the matrix according to the classification result, and a color distribution diagram is drawn according to the statistics result, so that a scatter diagram under Lab can be obtained.

For the color space HSV, quantization mapping can be specifically performed on the component combinations h× S, H ×v and s×v, color values obtained by the quantization mapping are mapped into a matrix of 256×256, then statistics is performed on all pixel points in the image to be segmented on the matrix according to the classification result, and a color distribution diagram is drawn according to the statistics result, so that a scatter diagram under HSV can be obtained.

For the color space CMYK, specifically, quantization mapping may be performed on the component combinations c× M, C ×y and m×y, color values obtained by the quantization mapping are mapped into a matrix of 256×256, then statistics is performed on all pixel points in the image to be segmented according to the classification result on the matrix, and a color distribution diagram is drawn according to the statistics result, so as to obtain a scatter diagram under CMYK.

It should be noted that, the pixel points of different types correspond to different color values, for example, red 255 may be used to represent a class a point, green 128 may be used to represent a class B point, blue 128 may be used to represent a class C point, and so on; therefore, the color distribution map can be drawn with the color component of the pixel point as coordinates and the class of the pixel point as a value. Alternatively, other chromaticity may be used to represent the class a point, the class B point, and the class C point, respectively, which will not be described herein.

205. The network equipment screens the scatter diagrams in each color space according to preset screening conditions to obtain screened scatter diagrams in each color space.

The screening condition may be set according to the requirements of practical applications, for example, in YCbCr color space, each scatter diagram related to the Y component may not be analyzed, for example, in Lab color space, each scatter diagram related to the L analysis may not be analyzed, in HSV color space, only the scatter diagram related to the H component may be analyzed, and similarly, in CMYK color space, only the scatter diagram related to the C component may be analyzed (the C component has a good filtering capability), and so on.

Note that if in step 204, only the scatter diagram of the partial component combinations is drawn, for example, if the scatter diagrams related to the Y component are not drawn in the YCbCr color space, then the scatter diagrams in the YCbCr color space may not be screened, and other color spaces are similar to the scatter diagrams and are not described herein.

206. The network equipment adopts a preset computer vision algorithm to perform basic geometric fitting on the filtered scatter diagrams in each color space, and determines the scatter diagram characteristics of each scatter diagram according to fitting results.

The computer vision algorithm may be set according to the actual application requirement, for example, openCV. While the basic geometry may include circular, parallelogram, and triangle geometries.

It should be noted that, the cvPointPolygonTest function in the OpenCV algorithm may be used to determine whether a pixel is located in the basic geometry fitted in step 206, such as whether it is located in a "circle", whether it is located in a "parallelogram", etc., and if the pixel is located in the fitted basic geometry in all the filtered scatter diagrams, it may be considered that the pixel is skin, so that a corresponding skin color determination algorithm may be set for different scatter diagram features (i.e. basic geometry) to determine whether the pixel is skin. Referring specifically to steps 207-209, the following is provided:

207. The network device selects at least one skin tone decision algorithm from a set of preset algorithms based on the scatter plot features.

For example, the network device may specifically obtain preset configuration information, where the configuration information stores a one-to-one correspondence between the scatter plot features and the skin tone determination algorithms, and then select, according to the correspondence, a corresponding skin tone determination algorithm for each scatter plot feature from the preset algorithm set.

The algorithm set may be set according to the actual application requirement, for example, the algorithm set may include a skin color determination algorithm corresponding to a "circle", a skin color determination algorithm corresponding to a "parallelogram", a skin color determination algorithm corresponding to a "triangle", and so on; the skin color determination algorithms can be specifically set according to the actual application requirements, and are not described herein.

208. And the network equipment calculates the similarity between each pixel point in the image to be segmented and the skin color according to the selected skin color judging algorithm. For example, the following may be specifically mentioned:

(1) The network device may specifically set a threshold in the selected skin tone decision algorithm based on the scatter plot feature.

For example, if the scatter plot feature is "circular", the threshold value in the skin tone determination algorithm corresponding to "circular" may be set according to the threshold value setting rule of "circular"; if the scatter diagram is characterized as a "parallelogram", the threshold value in the skin tone determination algorithm corresponding to the "parallelogram" may be set according to the threshold value setting rule of the "parallelogram", and so on.

The setting rule of the threshold may be determined according to the requirement of the actual application, which is not described herein.

(2) The network device transmits the selected skin color judging algorithm and the set threshold value as variable parameters to a fragment shader of a preset graphic program interface.

The graphical program interface may be set according to the actual application requirement, for example, may be OpenGL specifically.

(3) The network device performs texture mapping processing of the graphical program interface on the image to be segmented.

For example, if OpenGL is adopted in step (2), at this time, the image to be segmented may be specifically input into OpenGL for performing texture mapping processing.

(4) The network device performs texture sampling on the image to be segmented after the texture mapping processing through the fragment shader.

(5) And the network equipment calculates the similarity between each pixel point in the image to be segmented and the skin color according to the sampling result.

209. And the network equipment calculates a gray value according to the similarity and generates a skin color segmentation gray map corresponding to the segmentation image according to the gray value.

For example, see fig. 2b, where the left image is an image to be segmented, the right image is a skin tone segmentation gray scale image corresponding to the segmented image, in which the white portion (gray scale value 0% (black)) is skin and the black portion (gray scale value 100% (black)) is a non-skin portion.

Third embodiment,

In order to better implement the above method, the embodiment of the present invention further provides a skin color segmentation apparatus, where the skin color segmentation apparatus may be integrated in a network device, such as a terminal or a server, where the terminal may include a device such as a mobile phone, a tablet computer, a notebook computer, and/or a PC.

For example, as shown in fig. 3a, the skin tone segmentation apparatus may include an acquisition unit 301, a rendering unit 302, an analysis unit 303, a selection unit 304, and a segmentation unit 305, as follows:

(1) An acquisition unit 301;

an acquisition unit 301, configured to acquire an image to be segmented.

For example, the acquiring unit 301 may be specifically configured to read an image to be segmented from a local (i.e. the network device), or receive an image to be segmented sent by another device.

(2) A drawing unit 302;

a drawing unit 302, configured to draw a scatter diagram for the image to be segmented under a preset plurality of color spaces, respectively.

For example, the drawing unit 302 may include a classification subunit and a drawing subunit as follows:

and the classifying subunit can be used for classifying the pixel points in the image to be segmented.

For example, the classifying subunit may be specifically configured to perform face recognition and human body recognition on the image to be segmented based on a deep learning model to obtain a recognition result, and divide the pixels in the image to be segmented into preset categories, such as a face pixel category, a human body pixel category and a background pixel category, according to the recognition result.

The deep learning model may include CNN, DBN, RNN, RNTN, GAN, etc., and the preset categories may be set according to the actual application requirements, which are detailed in the foregoing method embodiments and are not described herein.

And the drawing subunit can be used for drawing the scatter diagram under a plurality of preset color spaces respectively according to the classification result.

For example, the drawing subunit may be specifically configured to map all the pixel points in the image to be segmented to a preset multiple color space according to the classification result, and draw according to the color distribution in the multiple color space according to the mapping result, so as to obtain a scatter diagram in each color space.

The preset color spaces may include RGB, YCbCr, lab, HSV, CMYK, and other color spaces, and the method embodiments described above may be referred to herein, and are not described herein.

(3) An analysis unit 303;

and an analysis unit 303, configured to analyze the scatter diagrams in the respective color spaces, and obtain scatter diagram features.

For example, the analysis unit 303 may specifically be configured to perform basic geometric fitting on the scatter plots in each color space by using a preset computer vision algorithm, and determine the scatter plot features of each scatter plot according to the fitting result.

Optionally, since the scatter plots in some color spaces are more scattered in most cases and are difficult to fit, in order to reduce unnecessary calculation overhead and improve processing efficiency, before performing the basic geometry fitting, the scatter plot may be screened, that is, as shown in fig. 3b, the skin color segmentation apparatus may further include a screening unit 306 as follows:

the filtering unit 306 may be configured to filter the scatter diagrams in each color space according to a preset filtering condition, so as to obtain filtered scatter diagrams in each color space.

At this time, the analysis unit 303 may specifically be configured to perform basic geometric shape fitting on the filtered scatter plots in each color space by using a preset computer vision algorithm.

(4) A selection unit 304;

a selecting unit 304, configured to select at least one skin color decision algorithm from a preset algorithm set based on the scatter plot feature;

for example, the selecting unit 304 may be specifically configured to obtain preset configuration information, where the configuration information stores a one-to-one correspondence between the scatter plot features and the skin tone determining algorithms, and select, according to the correspondence, a corresponding skin tone determining algorithm from the preset algorithm set for each scatter plot feature.

(5) A dividing unit 305;

the segmentation unit 305 is configured to segment the skin color of the image to be segmented according to the selected skin color determination algorithm.

For example, the segmentation unit 305 may include a calculation subunit and a generation subunit, as follows:

the calculating subunit is used for calculating the similarity between each pixel point in the image to be segmented and the skin color according to the selected skin color judging algorithm, and calculating the gray value according to the similarity.

For example, the calculating subunit may be specifically configured to set a threshold in the selected skin tone determination algorithm according to the scatter plot feature; transmitting the selected skin color judgment algorithm and the set threshold value as variable parameters into a fragment shader of a preset graphic program interface; performing texture mapping processing of the graphical program interface on the image to be segmented; texture sampling is carried out on the image to be segmented after texture mapping processing through the fragment shader; and calculating the similarity between each pixel point in the image to be segmented and the skin color according to the sampling result.

And the generation subunit is used for generating a skin color segmentation gray level image corresponding to the segmentation image according to the gray level value.

In the implementation, each unit may be implemented as an independent entity, or may be implemented as the same entity or several entities in any combination, and the implementation of each unit may be referred to the foregoing method embodiment, which is not described herein again.

As can be seen from the foregoing, the drawing unit 302 in the skin color segmentation apparatus of the present embodiment may draw a scatter diagram for an image to be segmented under a preset plurality of color spaces, then the analyzing unit 303 analyzes the scatter diagram under each color space, and the selecting unit 304 selects at least one skin color determination algorithm from a preset algorithm set based on the scatter diagram feature obtained by the analysis, so that the segmentation unit 305 performs skin color segmentation on the image to be segmented; because the scheme can refer to the scatter diagrams of a plurality of color spaces to comprehensively analyze the skin color similarity relation in the image to be segmented, and flexibly select the skin color judgment algorithm according to the skin color similarity relation, compared with the scheme based on the specific color space and the single skin color judgment algorithm in the prior art, the scheme has more accurate screening and resolving power, greatly improves the identification accuracy and improves the skin color segmentation effect.

Fourth embodiment,

The embodiment of the invention also provides a network device which can comprise a server, a terminal and other devices. As shown in fig. 4, a schematic structural diagram of a network device according to an embodiment of the present invention is shown, specifically:

the network device may include one or more processors 401 of a processing core, memory 402 of one or more computer readable storage media, power supply 403, and input unit 404, among other components. Those skilled in the art will appreciate that the network device structure shown in fig. 4 is not limiting of the network device and may include more or fewer components than shown, or may combine certain components, or may be a different arrangement of components. Wherein:

the processor 401 is a control center of the network device, connects various parts of the entire network device using various interfaces and lines, and performs various functions of the network device and processes data by running or executing software programs and/or modules stored in the memory 402, and calling data stored in the memory 402, thereby performing overall detection of the network device. Optionally, processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, an application program, etc., and the modem processor mainly processes wireless communication. It will be appreciated that the modem processor described above may not be integrated into the processor 401.

The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the network device, etc. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

The network device further comprises a power supply 403 for supplying power to the various components, and preferably the power supply 403 may be logically connected to the processor 401 by a power management system, so that functions of charge, discharge, and power consumption management are performed by the power management system. The power supply 403 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

The network device may also include an input unit 404, which input unit 404 may be used to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

Although not shown, the network device may further include a display unit or the like, which is not described herein. In this embodiment, the processor 401 in the network device loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the application programs stored in the memory 402, so as to implement various functions as follows:

obtaining an image to be segmented, drawing a scatter diagram for the image to be segmented under a plurality of preset color spaces, analyzing the scatter diagram under each color space to obtain scatter diagram features, selecting at least one skin color judgment algorithm from a preset algorithm set based on the scatter diagram features, and performing skin color segmentation on the image to be segmented according to the selected skin color judgment algorithm.

For example, face recognition and human body recognition can be specifically performed on the image to be segmented based on a deep learning model to obtain a recognition result, then, pixel points in the image to be segmented are divided into a face pixel point type, a human body pixel point type and a background pixel point type according to the recognition result, scatter diagrams are respectively drawn under preset multiple color spaces according to the classification result, then, basic geometric shape fitting is performed on the scatter diagrams under each color space by adopting a preset computer vision algorithm, and scatter diagram features of each scatter diagram are determined according to the fitting result.

The deep learning model may include CNN, DBN, RNN, RNTN, GAN, and the like, among others. The basic geometry may then include circular, parallelogram, and triangle geometries. The computer vision algorithm may be set according to the requirements of practical applications, for example, openCV.

Optionally, since the scatter diagrams in some color spaces are in most cases more scattered, and it is difficult to fit, in order to reduce unnecessary computation overhead and improve processing efficiency, before performing the basic geometry fitting, the scatter diagrams may be screened, that is, the processor 401 may further run an application program stored in the memory 402, so as to implement the following functions:

If the scatter plot has been filtered, then a preset computer vision algorithm may be used to perform basic geometry fitting on the filtered scatter plot in each color space when performing basic geometry fitting later, as described in the previous embodiments.

The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.

As can be seen from the above, the network device of the present embodiment may draw a scatter diagram for an image to be segmented under a preset plurality of color spaces, then analyze the scatter diagram under each color space, and select at least one skin color decision algorithm from a preset algorithm set based on the scatter diagram feature obtained by the analysis, so as to segment the skin color of the image to be segmented; because the scheme can refer to the scatter diagrams of a plurality of color spaces to comprehensively analyze the skin color similarity relation in the image to be segmented, and flexibly select the skin color judgment algorithm according to the skin color similarity relation, compared with the scheme based on the specific color space and the single skin color judgment algorithm in the prior art, the scheme has more accurate screening and resolving power, greatly improves the identification accuracy and improves the skin color segmentation effect.

Fifth embodiment (V),

Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present invention provides a storage medium having stored therein a plurality of instructions capable of being loaded by a processor to perform steps in any of the virtual resource transfer methods provided by the embodiment of the present invention. For example, the instructions may perform the steps of:

Optionally, because the scatter diagrams in some color spaces are more scattered in most cases and are difficult to fit, in order to reduce unnecessary calculation overhead and improve processing efficiency, before the basic geometric fitting is performed, the scatter diagrams may be screened, that is, the instruction may further perform the following steps:

If the scatter diagram is screened, then when the basic geometric shape is fitted later, a preset computer vision algorithm can be adopted to fit the basic geometric shape to the screened scatter diagram in each color space.

Wherein the storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.

The instructions stored in the storage medium may perform steps in any skin color segmentation method provided by the embodiments of the present invention, so that the beneficial effects that any skin color segmentation method provided by the embodiments of the present invention can be achieved are detailed in the previous embodiments, and are not repeated herein.

The foregoing describes in detail a skin color segmentation method, apparatus and storage medium provided by the embodiments of the present invention, and specific examples are applied herein to illustrate the principles and embodiments of the present invention, where the foregoing examples are provided to assist in understanding the method and core idea of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present invention, the present description should not be construed as limiting the present invention.

Claims

1. A skin tone segmentation method, comprising:

acquiring an image to be segmented;

respectively drawing a scatter diagram under each color space of the image to be segmented according to color distribution under the preset multiple color spaces, wherein the scatter diagram comprises the following components: classifying the pixel points in the image to be segmented; respectively drawing the scatter diagram under the preset multiple color spaces according to the classification result;

Analyzing the scatter diagram related to partial components in each color space to obtain the scatter diagram features, wherein the scatter diagram features comprise: performing basic geometric shape fitting on the scatter diagrams in each color space by adopting a preset computer vision algorithm; determining the scatter diagram features of each scatter diagram according to the fitting result;

based on the scatter plot features, selecting at least one skin tone determination algorithm from a preset algorithm set, comprising: acquiring preset configuration information, wherein the configuration information stores a one-to-one correspondence between the scatter diagram features and the skin color judgment algorithm; selecting a corresponding skin color judgment algorithm for each scatter diagram feature from the preset algorithm set according to the corresponding relation;

2. The method of claim 1, wherein classifying pixels in the image to be segmented comprises:

performing face recognition and human body recognition on the image to be segmented based on a deep learning model to obtain a recognition result;

and dividing the pixel points in the image to be segmented into a human face pixel point category, a human body pixel point category and a background pixel point category according to the identification result.

3. The method according to claim 1, wherein the drawing the scatter diagram under the preset multiple color spaces according to the classification result includes:

mapping all pixel points in the image to be segmented into a plurality of preset color spaces according to the classification result;

and drawing according to the mapping result under the multiple color spaces and the color distribution to obtain a scatter diagram under each color space.

4. The method of claim 1, wherein prior to performing the basic geometry fitting on the scatter plot in each color space using a predetermined computer vision algorithm, further comprising:

screening the scatter diagrams in each color space according to preset screening conditions to obtain screened scatter diagrams in each color space;

the method adopts a preset computer vision algorithm to perform basic geometric shape fitting on the scatter diagram under each color space, and specifically comprises the following steps: and adopting a preset computer vision algorithm to perform basic geometric shape fitting on the screened scatter diagram in each color space.

5. A method according to any one of claims 1 to 3, wherein said skin tone segmentation of the image to be segmented according to a selected skin tone decision algorithm comprises:

Calculating the similarity between each pixel point in the image to be segmented and the skin color according to the selected skin color judging algorithm;

calculating a gray value according to the similarity;

and generating a skin color segmentation gray level image corresponding to the segmentation image according to the gray level value.

6. The method of claim 5, wherein the calculating the similarity of each pixel in the image to be segmented to skin tone according to the selected skin tone determination algorithm comprises:

setting a threshold value in a selected skin color judging algorithm according to the scatter diagram features;

transmitting the selected skin color judgment algorithm and the set threshold value as variable parameters into a fragment shader of a preset graphic program interface;

performing texture mapping processing of the graphical program interface on the image to be segmented;

texture sampling is carried out on the image to be segmented after texture mapping processing through the fragment shader;

and calculating the similarity between each pixel point in the image to be segmented and the skin color according to the sampling result.

7. A skin tone segmentation apparatus, comprising:

an acquisition unit for acquiring an image to be segmented;

the drawing unit is used for drawing a scatter diagram under each color space for the image to be segmented according to color distribution under the preset multiple color spaces respectively;

The analysis unit is used for analyzing the scatter diagram related to the partial components in each color space to obtain scatter diagram features, and is particularly used for performing basic geometric fitting on the scatter diagram in each color space by adopting a preset computer vision algorithm, and determining the scatter diagram features of each scatter diagram according to fitting results;

the selecting unit is used for selecting at least one skin color judging algorithm from a preset algorithm set based on the scatter diagram features, and particularly used for acquiring preset configuration information, wherein the configuration information stores one-to-one correspondence between the scatter diagram features and the skin color judging algorithms, and the corresponding skin color judging algorithm is selected for each scatter diagram feature from the preset algorithm set according to the correspondence;

the segmentation unit is used for carrying out skin color segmentation on the image to be segmented according to the selected skin color judgment algorithm;

the drawing unit comprises a classification subunit and a drawing subunit;

the classifying subunit is used for classifying the pixel points in the image to be segmented;

and the drawing subunit is used for drawing the scatter diagram under the preset multiple color spaces respectively according to the classification result.

8. The apparatus of claim 7, wherein the device comprises a plurality of sensors,

the classifying subunit is specifically configured to perform face recognition and human body recognition on the image to be segmented based on a deep learning model, obtain a recognition result, and divide the pixels in the image to be segmented into a face pixel class, a human body pixel class and a background pixel class according to the recognition result.

9. The apparatus of claim 7, wherein the device comprises a plurality of sensors,

the drawing subunit is specifically configured to map all the pixel points in the image to be segmented into a preset plurality of color spaces according to the classification result, and draw the pixel points according to the color distribution in the plurality of color spaces according to the mapping result, so as to obtain a scatter diagram in each color space.

10. The apparatus of claim 7, further comprising a screening unit;

the screening unit is used for screening the scatter diagrams in each color space according to preset screening conditions to obtain screened scatter diagrams in each color space;

the analysis unit is specifically configured to perform basic geometric shape fitting on the filtered scatter diagrams in each color space by using a preset computer vision algorithm.

11. The apparatus according to any one of claims 7 to 9, wherein the segmentation unit comprises a calculation subunit and a generation subunit;

the calculating subunit is used for calculating the similarity between each pixel point in the image to be segmented and the skin color according to the selected skin color judging algorithm and calculating a gray value according to the similarity;

12. The apparatus according to claim 11, wherein the computing subunit is specifically configured to:

setting a threshold value in the selected skin color judging algorithm according to the scatter diagram features;

13. A storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps of the skin tone segmentation method of any one of claims 1 to 6.