WO2022135574A1 - Procédé et appareil de détection de couleur de peau, ainsi que terminal mobile et support de stockage - Google Patents

Procédé et appareil de détection de couleur de peau, ainsi que terminal mobile et support de stockage Download PDF

Info

Publication number
WO2022135574A1
WO2022135574A1 PCT/CN2021/141248 CN2021141248W WO2022135574A1 WO 2022135574 A1 WO2022135574 A1 WO 2022135574A1 CN 2021141248 W CN2021141248 W CN 2021141248W WO 2022135574 A1 WO2022135574 A1 WO 2022135574A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
color
skin
image data
face
Prior art date
Application number
PCT/CN2021/141248
Other languages
English (en)
Chinese (zh)
Inventor
杨敏
Original Assignee
百果园技术(新加坡)有限公司
杨敏
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 百果园技术(新加坡)有限公司, 杨敏 filed Critical 百果园技术(新加坡)有限公司
Publication of WO2022135574A1 publication Critical patent/WO2022135574A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the embodiments of the present application relate to the technical field of computer vision, for example, to a skin color detection method, device, mobile terminal, and storage medium.
  • the mainstream skin color detection can be divided into skin color detection based on color space, skin color detection based on machine learning classification, and skin color detection based on deep learning image segmentation.
  • skin color detection based on color space has a wide range of applications in the field of real-time stream processing due to its fast and efficient characteristics, but the accuracy is low.
  • Skin color detection based on machine learning classification and skin color detection based on deep learning image segmentation have higher
  • performance often becomes a bottleneck, which is difficult to apply to devices such as restricted mobile terminals.
  • the embodiments of the present application provide a method for detecting skin color, including:
  • the video data includes multiple frames of image data
  • the first target image data is image data collected at a first time point;
  • the embodiments of the present application also provide a skin color detection device, including:
  • a video data collection module configured to collect video data, the video data including multiple frames of image data
  • a face data detection module configured to detect face data in multiple frames of the image data respectively
  • the mapping function correction module is configured to modify the mapping function with the color value of the face data in the first target image data as a priori knowledge in response to detecting the face data in the first target image data, and the mapping function For identifying skin data based on color values, the first target image data is image data collected at a first time point;
  • a mapping detection module configured to substitute the color value of the second target image data into the mapping function to detect skin data in the second target image data, the second target image data being collected at a second time point image data.
  • an embodiment of the present application further provides a mobile terminal, where the mobile terminal includes:
  • memory arranged to store at least one program
  • the at least one processor When the at least one program is executed by the at least one processor, the at least one processor implements the skin color detection method according to the first aspect.
  • an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the skin color detection according to the first party is implemented method.
  • FIG. 1 is a flowchart of a skin color detection method provided in Embodiment 1 of the present application.
  • FIG. 2 is an example diagram of a time axis of video data according to Embodiment 1 of the present application.
  • FIG. 3 is an example diagram of a mapping function provided in Embodiment 1 of the present application.
  • FIG. 4 is an example diagram of a modified mapping function provided by Embodiment 1 of the present application.
  • FIG. 5 is a grayscale distribution diagram corresponding to a probability distribution of skin color detection provided in Embodiment 1 of the present application;
  • FIG. 6 is a schematic structural diagram of a skin color detection device provided in Embodiment 2 of the present application.
  • FIG. 7 is a schematic structural diagram of a mobile terminal according to Embodiment 3 of the present application.
  • Embodiment 1 is a flowchart of a skin color detection method provided in Embodiment 1 of the present application.
  • color values of face data can be used as prior knowledge to perform skin color detection, and the method can be performed by a skin color detection device.
  • the apparatus can be implemented by software and/or hardware, and can be configured in a mobile terminal, for example, a mobile phone, a tablet computer, a smart wearable device (such as a smart watch, smart glasses, etc.), etc., and includes the following steps:
  • Step 101 Collect video data.
  • the video data waiting for skin color detection generally refers to video data generated, transmitted or played in a real-time service scenario.
  • skin color detection can be performed on the video data in the mobile terminal that generates the video data.
  • the camera of the mobile terminal can be turned on, and the camera can collect the video data.
  • skin color detection may also be performed on the video data on the mobile terminal that plays the video data, which is not limited in this embodiment.
  • the video data waiting for skin color detection may refer to the video data used to carry the live broadcast content
  • the mobile terminal logged in by the anchor user generates the video data
  • the video data is distributed to at least one The device logged in by the viewer user plays the video.
  • the video data is usually detected by skin color on the mobile terminal logged in by the host user.
  • the video data waiting for skin color detection may refer to the video data used to carry the content of the call, and the mobile terminal logged in by the user who initiates the call generates video data, and sends the video data to at least A device logged in by a user who is invited to a call is played.
  • skin color detection is usually performed on the video data on the mobile terminal logged in by the user who initiated the call.
  • the video data waiting for skin color detection may refer to video data used to carry conference content
  • the mobile terminal logged in by the user who is speaking generates video data
  • the video data is sent to at least A device logged in by a user participating in the conference plays the video.
  • the mobile terminal logged in by the user who is usually speaking performs skin color detection on the video data.
  • the video data waiting for skin color detection may also refer to video data generated in a service scenario with low real-time requirements, such as a short video, which is not limited in this embodiment.
  • Step 102 Detect face data in multiple frames of image data respectively.
  • the multi-frame image data of the video data usually contains the user's character image.
  • the so-called character image can refer to the pixels in the image data used to represent the character.
  • the character image includes at least face data (pixels) for representing a human face, and may also include hair data (pixels) for representing hair and body data (pixels) for representing a body , limb data (pixel points) used to represent limbs, etc., which are not limited in this embodiment of the present application.
  • skin data for representing skin may be included.
  • face detection can be performed on multiple frames of image data respectively, and face data included in the image data can be identified. Using the face data as a high-quality anchor point, high-quality skin data can be captured.
  • face data is represented by face key points, that is, given face data, key regions of the face are located, including eyebrows, eyes, nose, mouth, facial contours, and so on.
  • a range of a specified shape can be generated based on the key points of the face, and the range is used to represent the face data, wherein the shape includes a rectangle, an ellipse, etc., and the range of the rectangle can also be called a face frame.
  • the following methods can be used to perform face detection on multi-frame image data:
  • Use artificial extraction features such as haar features, use features to train classifiers, and use classifiers for face detection.
  • Convolutional neural network using cascade structure for example, Cascade CNN (Cascade Convolutional Neural Network), MTCNN (Multi-task Cascaded Convolutional Networks, multi-task convolutional neural network).
  • Cascade CNN Cascade Convolutional Neural Network
  • MTCNN Multi-task Cascaded Convolutional Networks, multi-task convolutional neural network.
  • These methods for implementing face detection can be integrated in the modules of the application, and the modules of the application are directly called to perform face detection on image data. These methods for implementing face detection can also be integrated in the SDK (Software Development Kit, software development kit). ), the SDK is used as the assembly data of the application, the application can request the SDK to perform face detection on multi-frame image data, the SDK detects the face data in the image data, and returns the face data to the application.
  • the SDK Software Development Kit, software development kit
  • the system application usually provides API (Application Program Interface, Application Program Interface) for these methods of implementing face detection, as a face detection interface, for example, in the Android (Android) system , provides two face detection interfaces android.media.FaceDetector and android.media.FaceDetector.Face, in iOS system, provides two face detection interfaces AVCaptureMetadataOutput and CIDetector.
  • API Application Program Interface
  • Application Program Interface Application Program Interface
  • the face frame is a rectangular frame , which can be used to frame the face data, that is, the face data is located in the face frame.
  • the face detection interface provided by the system application, there is hardware support, and the face frame is calibrated based on a few key points of the face (2 key points of the face), the speed is fast, the performance consumption is very low, and the accuracy can meet the requirements of the first knowledge requirements.
  • the video data includes multiple frames of image data, which are denoted as P 1 , P 2 , ..., P t-1 , P t , P t+1 , ... , P n in the order of generation, where t and n are positive Integer, t+1 ⁇ n, since the video data is generated in real time, n increases continuously with the generation of the video data until the generation of the video data is completed.
  • each frame of image data in the video data is sequentially traversed to perform skin color detection.
  • the image data collected at the first time point is called the first target image data
  • the image data collected at the second time point will be called the first target image data.
  • the image data is called the second target image data.
  • the second time point is located after the first time point.
  • the first time point is the closest time point to the second time point when face data is detected.
  • the second time point may be the same as the second time point.
  • the first time point is adjacent, that is, there is no other time point between the second time point and the first time point, and the second time point may not be adjacent to the first time point, that is, the second time point and the first time point other time points in between.
  • T represents the time axis
  • t 0 to t 11 represent time points.
  • time point t 2 is the first time point
  • time point t 3 -t 6 are the second time points
  • the time point t 7 is the first time point
  • the time points t 8 -t 11 are the second time points.
  • Step 103 in response to detecting the face data in the first target image data, correct the mapping function by using the color value of the face data as prior knowledge.
  • a mapping function can be set by counting the color values of pixels in different samples such as skin data and non-skin data, and the mapping function can be used to identify skin data based on the color value, that is, the input of the mapping function is The color value is output as the probability that the current data belongs to skin data. At this time, the probability that the pixels in different samples are skin data under a certain color value can be counted, so that these color values and their probabilities can be fitted as a mapping function.
  • the mapping function can refer to the skin data of users of different races, different ages, and different skin tones, and can also refer to the skin data under different lighting conditions. Therefore, the mapping function is relatively broad and the accuracy is relatively high, but , in the case of skin color detection for the current user, the accuracy is lacking.
  • the probability that the current data in the mapping function belongs to skin data is a continuous value, such as [0-1]. In some cases, the probability that the current data in the mapping function belongs to skin data is a discrete value, such as 0, 1, this The embodiment does not limit this.
  • the abscissa (horizontal axis) is the color value (X), and the ordinate (vertical axis) is the probability (P) that the current data belongs to skin data .
  • the mapping function includes a first color map segment (the abscissa is [x 0 , x 1 ]), a second color map segment (the abscissa is [x 1 , x 2 ]), and a third color map segment (the abscissa is [x 1 , x 2 ]), which are connected in sequence
  • the coordinates are [x 2 , x 3 ]), wherein the probability that the current data in the first color mapping segment belongs to skin data increases from 0 to 1, the probability that the current data in the second color mapping segment belongs to skin data is 1, and the third The probability that the current data in the colormap segment belongs to skin data decreases from 1 to 0.
  • the second color map segment belongs to a line segment
  • the first color map segment and the third color map segment belong to a curve, which can be fitted by using a polynomial function or the like.
  • mapping function includes a first color mapping segment and a second color mapping segment connected in sequence, wherein the first color The probability that the current data in the mapping segment belongs to skin data increases from 0 to 1, and the probability that the current data in the second color mapping segment belongs to skin data decreases from 1 to 0.
  • mapping function is expressed by functions such as a quadratic equation, etc. etc., which are not limited in the embodiments of the present application.
  • those skilled in the art may also adopt other mapping functions according to actual needs, which are not limited in the embodiments of the present application.
  • mapping functions can be set for the color components.
  • red chrominance components R
  • green chrominance components G
  • the mapping function may be set for the red chrominance component (R)
  • the mapping function may be set for the green chrominance component (G)
  • the mapping function may be set for the blue chrominance component (B)
  • the color value can be counted within the range of the face data, and the counted color value
  • the confidence of the color value of the skin data is high, which can be used as a priori knowledge to correct the mapping function for the current user and improve the accuracy of the mapping function for the current user.
  • step 103 may include the following steps:
  • Step 1031 Determine the mapping function.
  • the abscissa of the mapping function is the color value
  • the ordinate is the probability that the current data belongs to skin data
  • the mapping function includes a first color mapping segment, a second color mapping segment, and a third color mapping segment connected in sequence
  • the probability that the current data in the first color map segment belongs to skin data increases from 0 to 1
  • the probability that the current data in the second color map segment belongs to skin data is 1
  • the probability that the current data in the third color map segment belongs to skin data increases from 1 down to 0.
  • Step 1032 Detect skin data in the face data based on the color space.
  • the area of the multiple face data can be counted separately. If the face data is framed by a face frame, the width of the face frame is w and the height is w. is h, then the area of the face data
  • n is a positive integer, such as 3
  • the skin data to be detected the skin data to be detected
  • the color value of the statistical skin data due to the imaging characteristics of the camera
  • the data method can accurately describe the color of the face data with a larger area. Selecting the n face data with the largest area can ensure the accuracy of the color value of the subsequent statistical skin data under the condition of reducing the amount of calculation.
  • the n face data with the largest area may be the face data sorted according to the area from large to small, and the n face data may be the top n face data in the sorting.
  • skin data may also be detected in all the face data based on the color space, which is not limited in this embodiment.
  • the color space-based method can be used to detect the skin color in the face data, and the pixels representing the skin data can be detected.
  • the color space-based method is simple to calculate, so the calculation speed is fast and time-consuming. Counting the color values of multiple skin data can maintain a high accuracy as a whole, meeting the requirements of prior knowledge.
  • the method corresponding to the color space detects skin data in the face data, which is not limited in this embodiment.
  • the pixel is skin data (that is, the probability that the pixel belongs to skin data is 1), and the color value of the pixel does not meet the following conditions
  • the pixel is not skin data (that is, the probability that the pixel belongs to skin data is 0):
  • Max represents the maximum value
  • Min represents the minimum value
  • Abs represents the absolute value
  • R represents the red chrominance component
  • G represents the green chrominance component
  • B represents the blue chrominance component.
  • the pixel is skin data (that is, the probability that the pixel belongs to skin data is 1), and the color value of the pixel does not meet the following conditions. Under the following conditions, it can be considered that the pixel is not skin data (that is, the probability that the pixel belongs to skin data is 0):
  • Cb represents the blue chrominance component
  • Cr represents the red chrominance component
  • the face data may include non-skin data such as hair data and background data.
  • non-skin data such as hair data and background data.
  • Reduce the area of face data on the edge reduce the amount of non-skin data, and increase the proportion of skin data, so as to detect skin data in candidate area images based on color space, and improve the accuracy of skin data detection.
  • the center point P(x 0 , y 0 ) of the face frame can be determined, and on the condition that the center point P(x 0 , y 0 ) is maintained , reduce the width and height of the face frame as a candidate area image.
  • the candidate area image can be expressed as [x 0 ⁇ (a*w), y 0 ⁇ (b*h)[, where a is a coefficient less than 1, such as 0.4, b is a coefficient less than 1, such as 0.1.
  • Step 1033 count the color values of the skin data.
  • the average value of the color values among multiple pixel points can be counted as the color value of the entire skin data.
  • Step 1034 Mark the color value of the skin data on the horizontal axis of the coordinate system where the mapping function is located.
  • the color value C of the skin data can be marked on the horizontal axis of the coordinate system where the mapping function is located.
  • the color value C of the skin data is generally located in the second color mapping segment of the mapping function (the abscissa is [x 1 , x 2 ]).
  • Step 1035 under the condition that the first color map segment and the third color map segment are maintained, reduce the second color map segment with reference to the color value of the skin data.
  • the mapping function can be converged to the color value C of the skin data as a whole, the range of the mapping function can be reduced, and some regions with low probability can be excluded.
  • the shape of the first color mapping segment and the third color mapping segment are kept unchanged as a precondition, and the color value of the user's skin data in the current scene is used as a reference to reduce the size of the second color mapping segment. range, and adjust the range of the first color map segment and the third color map segment accordingly.
  • step 1035 includes the following steps:
  • Step 10351 Converging the second color map segment towards the color value of the skin data.
  • either end of the second color mapping segment may be converged, or both ends of the second color mapping segment may be converged at the same time, which is not limited in this embodiment.
  • the color value C of the skin data is marked on the horizontal axis, and the two ends of the second color mapping segment are converged toward the color value C of the skin data at the same time, and the abscissa of the second color mapping segment before convergence is [x 1 , x 2 ], the abscissa of the second color mapping segment after convergence is [x 1 ', x 2 '], where x 1 '>x 1 , x 2 ' ⁇ x 2 .
  • the target length L t can be determined. Assuming that the length of the second color map segment is L, the target length L t is less than the length L of the second color map segment, that is, L t ⁇ L. The color value C is used as the center point to reduce the second color map segment until the length L of the second color map segment is equal to the target length L t , thereby reducing the range of the second color map segment.
  • the target length L t x 2 ′-x 1 ′
  • the color value of the skin data is
  • the target length L t is a statistical value, which is a constant.
  • the target length L t is an adaptively adjusted value, which is a variable.
  • the reference length T may be determined, and the reference length T is a statistical value, which is a constant, and is smaller than the length L of the second color map segment, that is, T ⁇ L.
  • the time difference ⁇ t between the second time point and the first time point is mapped to the correction coefficient w t by the preset mapping method f(), and the correction coefficient w t is positively correlated with the time difference, that is, the larger the time difference, the greater the correction coefficient w t On the contrary, the smaller the time difference, the smaller the correction coefficient w t .
  • the second time point is closer to the first time point.
  • the farther away, the higher the probability of changes in lighting and other conditions in the scene, and the lower the reference value of the color value of the skin data counted at the first time point. point) is farther and farther away from the current time (second time point), then gradually relax the target length L t , so that the target length L t is getting closer and closer to the length L of the second color mapping segment, so as to ensure that the second Accuracy of colormap segments.
  • Step 10352 Shift the first color mapping segment in a direction close to the second color mapping segment until the first color mapping segment is connected to the second color mapping segment.
  • the first color mapping segment can be shifted to the right on the horizontal axis, so that the first color mapping segment moves toward the second color mapping segment until the first color mapping segment is reached.
  • the second color map segment is connected, that is, the first color map segment is connected to the second color map at the endpoint x 1 ', and the abscissa of the first color map segment before translation is [x 0 , x 1 ], after translation
  • Step 10353 Shift the third color mapping segment in a direction close to the second color mapping segment until the third color mapping segment is connected to the second color mapping segment.
  • the third color mapping segment can be shifted to the left on the horizontal axis, so that the third color mapping segment moves toward the second color mapping segment until the third color mapping segment is reached.
  • the second color map segment is connected, that is, the third color map segment is connected to the second color map at the endpoint x 2 '.
  • the abscissa of the third color map segment is [x 2 , x 3 ].
  • Step 104 Substitute the color value of the second target image data into the mapping function to detect skin data in the second target image data.
  • the face data is detected at the first target image data corresponding to the first time point, and the color value of the face data in the face data is used as the prior knowledge to correct the mapping function.
  • the mapping function can be located after the first time point.
  • skin color detection is performed on the second target image data (that is, the second target image data is the image data collected at the second time point), so as to detect the degree to which the pixel points represent the skin data in the second target image data .
  • the color value of each pixel of the second target image data can be marked on the horizontal axis in the coordinate system where the mapping function is located. If the color value of the pixel in the second target image data is outside the mapping function, Then the probability that the pixel belongs to the skin data is 0. If the color value of the pixel in the second target image data is within the mapping function, the probability corresponding to the color value on the vertical axis can be found through the mapping function, as the pixel belongs to. Probability of skin data.
  • the abscissa of the mapping function is the color value
  • the ordinate is the probability that the pixel belongs to the skin data.
  • the color value has different chromaticity components, and each color component has a corresponding mapping function, For the same pixel, different probabilities can be calculated.
  • the color value of each pixel in the second target image data can be queried, and the color value can be substituted into the corresponding mapping function to map out the candidate probability that each pixel belongs to the skin data under the color value, based on
  • These candidate probabilities can be calculated by averaging, summing, multiplying, linear fusion (that is, summing after configuring weights) to calculate the target probability that the pixel belongs to the skin data.
  • These target probabilities reflect the ROI (region of interest, region of interest) area.
  • the second target image data has a blue chrominance component Cb, a red chrominance component Cr.
  • the color value of the blue chrominance component Cb is substituted into the mapping function corresponding to the blue chrominance component Cb, so as to map out the blue probability that the pixel belongs to the skin data under the blue chrominance component Cb, as the candidate probability
  • the color value of the red chrominance component Cr is substituted into the mapping function corresponding to the red chrominance component Cr, so as to map the red probability that the pixel belongs to the skin data under the red chrominance component Cr, as the candidate probability
  • the blue probability can be calculated with red probability
  • the product between as the target probability that the pixel belongs to the skin data
  • the skin data can be detected in the image data based on the color space.
  • operations such as beautifying processing (such as skin resurfacing) can be performed on the video data according to the needs of the user.
  • the left side is the grayscale distribution corresponding to the probability distribution obtained by using the color space-based skin color detection
  • the right side is the probability distribution corresponding to the skin color detection obtained by applying this embodiment.
  • Grayscale distribution in which, the higher the grayscale of a pixel (more white), the higher the probability that the pixel belongs to skin data, and the lower the grayscale (more black), the lower the probability that the pixel belongs to skin data. .
  • the skin color detection based on the color space considers clothes, hair, and backgrounds (such as electric lights, etc.) to belong to skin data, and this embodiment can well exclude clothes, hair, and backgrounds (such as electric lights, etc.). ), which greatly improves the accuracy of skin color detection, and can well protect clothes, hair, and backgrounds (such as lights, etc.) in subsequent beauty treatments (such as microdermabrasion).
  • video data is collected, and the video data includes multiple frames of image data, and face data is detected in the multiple frames of image data respectively. If face data is detected in the first target image data, the face data
  • the color value is used as a priori knowledge to correct the mapping function, the mapping function is used to identify skin data based on the color value, the first target image data is the image data collected at the first time point, and the color value of the second target image data is substituted into the mapping function , in order to detect skin data in the second target image data, the second target image data is the image data collected at the second time point, the video data has continuity, therefore, the content between multiple frames of image data has correlation, so that The previous image data can be used as the prior knowledge of the subsequent image data, and the face data can be used as the anchor to capture high-quality skin data.
  • This adaptive modification of the mapping function can improve the accuracy of skin color detection performed by the mapping function in the current business scenario, and operations such as face detection, statistics of the color value of the skin data, correction of the mapping function and its application are relatively simple. , the calculation amount is small, the speed is fast, and the time-consuming is short. In the case of limited performance of mobile terminals and other equipment, the skin color detection of video data can be realized in real time.
  • some data similar to skin data such as hair, clothes, background, etc.
  • FIG. 6 is a structural block diagram of a skin color detection device provided in Embodiment 2 of the present application, which may include the following modules:
  • the video data collection module 601 is configured to collect video data, and the video data includes multiple frames of image data;
  • a face data detection module 602 configured to detect face data in the multiple frames of the image data respectively;
  • the mapping function correction module 603 is configured to modify the mapping function with the color value of the face data in the first target image data as prior knowledge in response to detecting the face data in the first target image data, and the mapping function The function is used to identify skin data based on the color value, and the first target image data is the image data collected at the first time point;
  • the mapping detection module 604 is configured to substitute the color value of the second target image data into the mapping function, so as to detect skin data in the second target image data, the second target image data is at the second time point acquired image data.
  • the face data detection module 602 includes:
  • a face detection interface calling module configured to call the face detection interface provided by the camera, to request to detect the face data in the multiple frames of the image data collected by the camera;
  • the face frame receiving module is configured to receive at least one face frame returned by the face detection interface, where the face frame is used to frame face data.
  • mapping function correction module 603 includes:
  • the mapping function determination module is set to determine the mapping function, the abscissa of the mapping function is the color value, and the ordinate is the probability that the current data belongs to the skin data, and the mapping function includes the first color mapping segment and the second color that are connected in turn.
  • mapping section and third color mapping section the probability that the current data in the first color mapping section belongs to skin data increases from 0 to 1, the probability that the current data in the second color mapping section belongs to skin data is 1, and the The probability that the current data in the third color map segment belongs to skin data decreases from 1 to 0;
  • a face skin detection module configured to detect skin data in the face data in the first target image data based on the color space
  • a color value statistics module configured to count color values for the skin data
  • a color value marking module configured to mark the color value based on the statistics of the skin data on the horizontal axis of the coordinate system where the mapping function is located;
  • the color map segment correction module is configured to reduce the second color map segment with reference to the color value based on the statistics of the skin data under the condition of maintaining the first color map segment and the third color map segment.
  • mapping function correction module 603 further includes:
  • an area statistics module configured to count the areas of a plurality of the face data respectively in response to the existence of a plurality of face data in the first target image data
  • a face extraction module configured to extract the n pieces of face data with the largest area.
  • the face skin detection module includes:
  • a face data convergence module configured to converge the face data in the first target image data to obtain a candidate area image
  • a candidate detection module configured to detect skin data in the candidate region image based on the color space.
  • the face data convergence module includes:
  • a center point determination module configured to frame the face frame in response to the face data in the first target image data, and determine the center point of the face frame
  • the face frame reduction module is configured to set the center point of the face frame as the center point of the candidate area image, and reduce the width and height of the face frame as the width and height of the candidate area image.
  • the color map segment correction module includes:
  • a second color map segment convergence module configured to converge the second color map segment toward the color value based on the skin data statistics
  • a first color map segment translation module configured to translate the first color map segment in a direction close to the second color map segment, until the first color map segment is connected to the second color map segment;
  • a third color mapping segment translation module is configured to translate the third color mapping segment in a direction close to the second color mapping segment until the third color mapping segment is connected to the second color mapping segment.
  • the second color map segment convergence module includes:
  • a target length determination module configured to determine a target length, the target length being less than the length of the second color map segment
  • a color map segment reduction module configured to reduce the second color map segment with the color value based on the skin data statistics as a center point until the length of the second color map segment is equal to the target length.
  • the target length determination module includes:
  • a reference length determination module configured to determine a reference length, the reference length being less than the length of the second color mapping segment
  • An adjustable length calculation module configured to calculate the difference between the length of the second color map segment and the reference length as an adjustable length
  • a correction coefficient calculation module configured to map the time difference between the second time point and the first time point as a correction coefficient, and the correction coefficient is positively correlated with the time difference;
  • a correction length calculation module configured to calculate the product between the adjustable length and the correction coefficient as the correction length
  • the target length calculation module is configured to calculate the sum value between the reference length and the corrected length as the target length.
  • mapping detection module 604 includes:
  • a color value query module configured to query the color value of a pixel in the second target image data
  • a candidate probability mapping module configured to substitute the color value of the pixel into the mapping function to map the pixel as a candidate probability that the pixel belongs to skin data under the color value;
  • the target probability calculation module is configured to calculate the target probability that the pixel point belongs to the skin data based on the candidate probability.
  • the candidate probability mapping module includes:
  • the blue probability mapping module is configured to substitute the color value of the blue chrominance component into the mapping function corresponding to the blue chrominance component, so as to map the pixel points to the skin data under the blue chrominance component.
  • the blue probability as the candidate probability
  • a red probability mapping module configured to substitute the color value of the red chromaticity component into the mapping function corresponding to the red chromaticity component, so as to map the pixel to the red color of the skin data under the red chromaticity component probability, as a candidate probability;
  • the target probability calculation module includes:
  • the probability product calculation module is configured to calculate the product between the blue probability and the red probability as the target probability that the pixel belongs to the skin data.
  • An image detection module configured to detect skin data in the image data based on the color space in response to the previously undetected face data.
  • the skin color detection apparatus provided by the embodiment of the present application can execute the skin color detection method provided by any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method.
  • FIG. 7 is a schematic structural diagram of a mobile terminal according to Embodiment 3 of the present application.
  • FIG. 7 shows a block diagram of an exemplary mobile terminal 12 suitable for use in implementing embodiments of the present application.
  • the mobile terminal 12 shown in FIG. 7 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present application.
  • the mobile terminal 12 takes the form of a general-purpose computing device.
  • Components of the mobile terminal 12 may include, but are not limited to, one or more processors or processing units 16, a system memory (also referred to as memory) 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16). .
  • Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures.
  • these architectures include, but are not limited to, Industry Standard Architecture (ISA, Industry Standard Architecture) bus, Micro Channel Architecture (MCA, Micro Channel Architecture) bus, enhanced ISA bus, Video Electronics Standards Association (VESA, Video Electronics Standards Association) local bus and Peripheral Component Interconnect (PCI, Peripheral Component Interconnect) bus.
  • ISA Industry Standard Architecture
  • MCA Micro Channel Architecture
  • VESA Video Electronics Standards Association
  • PCI Peripheral Component Interconnect
  • Mobile terminal 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the mobile terminal 12, including volatile and non-volatile media, removable and non-removable media.
  • System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and/or cache memory 32 .
  • the mobile terminal 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media.
  • storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (not shown in FIG. 7, commonly referred to as a "hard drive”).
  • a disk drive for reading and writing to removable non-volatile magnetic disks eg "floppy disks”
  • removable non-volatile optical disks eg CD-ROM, DVD-ROM
  • each drive may be connected to bus 18 through one or more data media interfaces.
  • Memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present application.
  • a program/utility 40 having a set (at least one) of program modules 42, which may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data , each or some combination of these examples may include an implementation of a network environment.
  • Program modules 42 generally perform the functions and/or methods of the embodiments described herein.
  • the mobile terminal 12 may also communicate with one or more external devices 14 (eg, a keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with the mobile terminal 12, and/or communicate with Any device (eg, network card, modem, etc.) that enables the mobile terminal 12 to communicate with one or more other computing devices. Such communication may take place through input/output (I/O) interface 22 .
  • the mobile terminal 12 can also communicate with one or more networks (such as a local area network (LAN, Local Area Network), a wide area network (WAN, Wide Area Network) and/or a public network, such as the Internet, through the network adapter 20. As shown in FIG.
  • the network adapter 20 communicates with other modules of the mobile terminal 12 via the bus 18 .
  • other hardware and/or software modules may be used in conjunction with the mobile terminal 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID (Redundant Arrays) of Independent Disks, disk array) systems, tape drives, and data backup storage systems.
  • the processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing the skin color detection method provided by the embodiments of the present application.
  • the fourth embodiment of the present application also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, each process of the above-mentioned skin color detection method can be achieved, and the same effect can be achieved, In order to avoid repetition, details are not repeated here.
  • the computer-readable storage medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. More specific examples (non-exhaustive list) of computer readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM, Read-Only Memory), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable of the above The combination.
  • a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • the embodiments of the present application propose a skin color detection method, device, mobile terminal, and storage medium, so as to solve the problem of how to take into account the real-time performance and accuracy of skin color detection under the condition of limited performance.
  • video data is collected, and the video data includes multiple frames of image data, and face data is detected in the multiple frames of image data respectively. If face data is detected in the first target image data, the face data
  • the color value is used as a priori knowledge to correct the mapping function, the mapping function is used to identify skin data based on the color value, the first target image data is the image data collected at the first time point, and the color value of the second target image data is substituted into the mapping function , in order to detect skin data in the second target image data, the second target image data is the image data collected at the second time point, and the video data has continuity, therefore, the content between multiple frames of image data has correlation, so that The previous image data can be used as the prior knowledge of the subsequent image data, and the face data can be used as an anchor to capture high-quality skin data.
  • This adaptive modification of the mapping function can improve the accuracy of skin color detection performed by the mapping function in the current business scenario, and operations such as face detection, statistics of the color value of the skin data, correction of the mapping function and its application are relatively simple. , the calculation amount is small, the speed is fast, and the time-consuming is short. In the case of limited performance of mobile terminals and other equipment, the skin color detection of video data can be realized in real time.
  • some data similar to skin data such as hair, clothes, background, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

La présente demande porte, dans les modes de réalisation, sur un procédé et sur un appareil de détection de couleur de peau, ainsi que sur un terminal mobile et sur un support de stockage. Le procédé consiste : à acquérir des données vidéo, les données vidéo comprenant de multiples trames de données d'image ; à détecter respectivement des données faciales dans les multiples trames de données d'image ; à la suite de la détection de données faciales dans des premières données d'image cible, à utiliser une valeur de couleur des données faciales comme connaissance a priori pour corriger une fonction de mappage, la fonction de mappage étant utilisée pour identifier des données de peau sur la base de la valeur de couleur ; et à substituer une valeur de couleur de secondes données d'image cible dans la fonction de mappage de sorte à détecter des données de peau dans les secondes données d'image cible.
PCT/CN2021/141248 2020-12-25 2021-12-24 Procédé et appareil de détection de couleur de peau, ainsi que terminal mobile et support de stockage WO2022135574A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011563923.X 2020-12-25
CN202011563923.XA CN112712569B (zh) 2020-12-25 2020-12-25 一种肤色检测方法、装置、移动终端和存储介质

Publications (1)

Publication Number Publication Date
WO2022135574A1 true WO2022135574A1 (fr) 2022-06-30

Family

ID=75546610

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/141248 WO2022135574A1 (fr) 2020-12-25 2021-12-24 Procédé et appareil de détection de couleur de peau, ainsi que terminal mobile et support de stockage

Country Status (2)

Country Link
CN (1) CN112712569B (fr)
WO (1) WO2022135574A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112712569B (zh) * 2020-12-25 2023-12-12 百果园技术(新加坡)有限公司 一种肤色检测方法、装置、移动终端和存储介质
CN113888543B (zh) * 2021-08-20 2024-03-19 北京达佳互联信息技术有限公司 肤色分割方法、装置、电子设备及存储介质
CN113554007B (zh) * 2021-09-18 2022-01-11 上海齐感电子信息科技有限公司 人脸框的计算方法及计算系统

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060088209A1 (en) * 2004-10-21 2006-04-27 Microsoft Corporation Video image quality
CN102968623A (zh) * 2012-12-07 2013-03-13 上海电机学院 肤色检测系统及方法
CN103455790A (zh) * 2013-06-24 2013-12-18 厦门美图网科技有限公司 一种基于肤色模型的皮肤识别方法
CN104392211A (zh) * 2014-11-12 2015-03-04 厦门美图网科技有限公司 一种基于显著性检测的皮肤识别方法
CN106650583A (zh) * 2016-09-28 2017-05-10 广东欧珀移动通信有限公司 人脸检测方法、装置及终端设备
CN108388885A (zh) * 2018-03-16 2018-08-10 南京邮电大学 面向大型直播场景的多人特写实时识别与自动截图方法
CN112712569A (zh) * 2020-12-25 2021-04-27 百果园技术(新加坡)有限公司 一种肤色检测方法、装置、移动终端和存储介质

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10491895B2 (en) * 2016-05-23 2019-11-26 Intel Corporation Fast and robust human skin tone region detection for improved video coding
CN107292228A (zh) * 2017-05-05 2017-10-24 珠海数字动力科技股份有限公司 一种加快人脸识别搜索速度的方法
CN107172354B (zh) * 2017-06-21 2020-04-03 深圳市万普拉斯科技有限公司 视频处理方法、装置、电子设备及存储介质
CN107392166A (zh) * 2017-07-31 2017-11-24 北京小米移动软件有限公司 肤色检测方法、装置及计算机可读存储介质
CN109063580A (zh) * 2018-07-09 2018-12-21 北京达佳互联信息技术有限公司 人脸识别方法、装置、电子设备及存储介质
CN111047517B (zh) * 2018-10-11 2022-09-30 展讯通信（上海）有限公司 肤色调节方法及装置、可读存储介质
CN110706295A (zh) * 2019-09-10 2020-01-17 中国平安人寿保险股份有限公司 人脸检测方法、装置及计算机可读存储介质
CN111291638A (zh) * 2020-01-19 2020-06-16 上海云从汇临人工智能科技有限公司 一种对象比对方法、系统、设备及介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060088209A1 (en) * 2004-10-21 2006-04-27 Microsoft Corporation Video image quality
CN102968623A (zh) * 2012-12-07 2013-03-13 上海电机学院 肤色检测系统及方法
CN103455790A (zh) * 2013-06-24 2013-12-18 厦门美图网科技有限公司 一种基于肤色模型的皮肤识别方法
CN104392211A (zh) * 2014-11-12 2015-03-04 厦门美图网科技有限公司 一种基于显著性检测的皮肤识别方法
CN106650583A (zh) * 2016-09-28 2017-05-10 广东欧珀移动通信有限公司 人脸检测方法、装置及终端设备
CN108388885A (zh) * 2018-03-16 2018-08-10 南京邮电大学 面向大型直播场景的多人特写实时识别与自动截图方法
CN112712569A (zh) * 2020-12-25 2021-04-27 百果园技术(新加坡)有限公司 一种肤色检测方法、装置、移动终端和存储介质

Also Published As

Publication number Publication date
CN112712569B (zh) 2023-12-12
CN112712569A (zh) 2021-04-27

Similar Documents

Publication Publication Date Title
JP7110502B2 (ja) 深度を利用した映像背景減算法
WO2022135574A1 (fr) Procédé et appareil de détection de couleur de peau, ainsi que terminal mobile et support de stockage
CN109952594B (zh) 图像处理方法、装置、终端及存储介质
WO2020063744A1 (fr) Procédé et dispositif de détection de visage, procédé de traitement de service, dispositif terminal et support d'informations
US9864901B2 (en) Feature detection and masking in images based on color distributions
US8983152B2 (en) Image masks for face-related selection and processing in images
WO2022135579A1 (fr) Procédé et dispositif de détection de couleur de peau, terminal mobile et support de stockage
WO2022156640A1 (fr) Procédé et appareil de correction du regard pour image, dispositif électronique, support d'enregistrement lisible par ordinateur et produit programme d'ordinateur
WO2022199583A1 (fr) Procédé et appareil de traitement d'image, dispositif informatique et support de stockage
US9053573B2 (en) Systems and methods for generating a virtual camera viewpoint for an image
US10430694B2 (en) Fast and accurate skin detection using online discriminative modeling
WO2022161009A1 (fr) Procédé et appareil de traitement d'image, support de stockage et terminal
CN105243371A (zh) 一种人脸美颜程度的检测方法、系统及拍摄终端
WO2021115242A1 (fr) Procédé de traitement d'image à très haute résolution et appareil associé
CN112614060A (zh) 人脸图像头发渲染方法、装置、电子设备和介质
CN111242074B (zh) 一种基于图像处理的证件照背景替换方法
CN109033935B (zh) 抬头纹检测方法及装置
WO2023045183A1 (fr) Traitement d'images
WO2023273247A1 (fr) Procédé et dispositif de traitement d'image de visage, support de stockage lisible par ordinateur, terminal
CN107153806B (zh) 一种人脸检测方法及装置
CN114187166A (zh) 图像处理方法、智能终端及存储介质
CN112597911A (zh) 一种磨皮处理方法、装置、移动终端和存储介质
WO2023010796A1 (fr) Procédé de traitement d'image et appareil associé
CN113743243A (zh) 一种基于深度学习的人脸美颜方法
CN111915529A (zh) 一种视频的暗光增强方法、装置、移动终端和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21909569

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21909569

Country of ref document: EP

Kind code of ref document: A1