CN109033935B - Head-up line detection method and device - Google Patents
Head-up line detection method and device Download PDFInfo
- Publication number
- CN109033935B CN109033935B CN201810554520.5A CN201810554520A CN109033935B CN 109033935 B CN109033935 B CN 109033935B CN 201810554520 A CN201810554520 A CN 201810554520A CN 109033935 B CN109033935 B CN 109033935B
- Authority
- CN
- China
- Prior art keywords
- key point
- image
- head
- area
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
- G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/162—Detection; Localisation; Normalisation using pixel segmentation or colour matching
Abstract
The application discloses a method and a device for detecting raised lines. Wherein, the method comprises the following steps: after a target image is acquired, determining key points of a human face in the target image; the target image is an image of the to-be-detected raised line; intercepting an image of a first area from the target image according to the key point of the face in the target image, wherein the first area is an area corresponding to the forehead; determining the proportion of non-skin areas in the first area through a skin color clustering method, inputting the image of the first area into a target network model under the condition that the proportion of the non-skin areas is smaller than a preset proportion, and outputting the detection result of the new line. Corresponding apparatus is also provided. By the aid of the method and the device, the accuracy of raised line detection can be improved.
Description
Technical Field
The application relates to the technical field of computers, in particular to a method and a device for detecting raised lines.
Background
One of the signs of facial skin aging is wrinkles, so the head-up lines on the forehead of a human face can be regarded as an important skin feature in the human face. Meanwhile, the severity of the raised lines is different due to different genders, ages, geographical attributes and the like.
In the field of beauty and make-up, the severity of the face raised line can be automatically detected through a mobile phone self-shot picture, and other skin characteristics of the face are combined, so that skin care opinions are provided for users. And the method also has great application value in the aspect identification and age identification by detecting the severity of the raised line in the field of image identification.
Therefore, how to detect the raised line is a problem which is being researched by the person skilled in the art.
Disclosure of Invention
The application provides a method and a device for detecting a head-up line, which can improve the accuracy of detecting the head-up line.
In a first aspect, an embodiment of the present application provides a method for detecting a raised line, including:
after a target image is acquired, determining key points of a human face in the target image; the target image is an image of the to-be-detected raised line;
intercepting an image of a first area from the target image according to the key point of the face in the target image, wherein the first area is an area corresponding to the forehead;
determining the proportion of non-skin areas in the first area through a skin color clustering method, inputting the image of the first area into a target network model under the condition that the proportion of the non-skin areas is smaller than a preset proportion, and outputting the detection result of the new line.
In the embodiment of the application, after the image of the first area is obtained, the proportion of the non-skin area is determined by a skin color clustering method, and when the proportion of the non-skin area is smaller than a preset proportion, the image of the first area is input into a target network model; the situation that the target network model cannot be effectively identified due to the fact that the image of the first area is shielded by other objects such as hair, a headband or a hat can be effectively avoided; by implementing the embodiment of the application, the occupation ratio of the non-skin area can be preferentially determined, so that whether the new line is detected by inputting the new line into the target network model is determined according to the occupation ratio of the non-skin area, the accuracy of outputting the target network model can be improved, and the output efficiency is improved.
In a possible implementation manner, the inputting the image of the first area to a target network model and outputting the detection result of the new line includes:
cutting the first area to obtain a first number of sub-areas, and respectively inputting images containing the first number of sub-areas into the target network model;
and outputting the detection results of the first number of the head-up lines, and determining the severity of the head-up lines according to the ratio of the images with the head-up lines.
In the embodiment of the application, the first region is divided, so that the divided sub-regions are input into the target network model, the severity of the raised line is determined according to the detection result of each sub-region, the accuracy of raised line detection can be effectively improved, and the accuracy of the target network model in detecting the raised line is improved.
In one possible implementation manner, the key points of the face in the target image include a first key point and a second key point; the first key point is the highest point of the left eyebrow in the key points of the face, and the second key point is the highest point of the right eyebrow in the key points of the face;
the image of the first area is intercepted from the target image according to the key points of the face in the target image, and the method comprises the following steps:
determining the first side length of the first area according to the abscissa of the first key point and the abscissa of the second key point;
determining a second side length of the first area according to the translation amount of the ordinate of the first key point or the second key point;
and intercepting the image of the first area according to the first side length and the second side length.
In the embodiment of the application, the first region is determined by the first key point and the second key point, so that the method is simple and feasible, and the interception efficiency of the first region is improved.
In one possible implementation, the method further includes:
determining that the target image has no new line if the proportion of the non-skin area is not less than the predetermined proportion.
In one possible implementation, before the inputting the image of the first area to the target network model, the method further includes:
after a face image sample is collected, determining face key points of the face image sample;
intercepting an image of a second area according to the face key points of the face image sample;
cutting the second area to obtain a second number of sub-areas;
and respectively inputting the sub-regions of the second number into a network model, and training the network model to obtain the target network model.
After a face image sample is collected, determining face key points of the face image sample, which can be understood as determining the face key points in each image in the face image sample; intercepting the image of the second area according to the face key point of the face image sample, which can be understood as intercepting the image of the second area according to the face key point in each image in the face image sample; that is, the number of the face image samples is at least two, and when the face key point is determined and the image of the second region is cut, each image in the face image samples can be operated.
In the embodiment of the application, the network model is trained by the method, so that the efficiency of the trained network model can be effectively improved, namely, the accuracy of the output detection result of the network model is improved.
In one possible implementation, before the inputting the image of the first area to the target network model, the method further includes:
receiving the target network model from a training device; the training device is used for training a network model to obtain the target network model.
In a second aspect, an embodiment of the present application provides a raised line detection apparatus, including:
the first determining unit is used for determining the key points of the human face in the target image after the target image is obtained; the target image is an image of the to-be-detected raised line;
the intercepting unit is used for intercepting an image of a first area from the target image according to the key points of the face in the target image, wherein the first area is an area corresponding to the forehead;
a second determining unit, configured to determine a proportion of a non-skin region in the first region by a skin color clustering method;
an input unit, configured to input the image of the first area to a target network model when the proportion of the non-skin area is smaller than a predetermined proportion;
and the output unit is used for outputting the detection result of the head raising line.
In one possible implementation, the input unit includes:
a cutting subunit, configured to cut the first region, to obtain a first number of sub-regions;
an input subunit, configured to input images including the first number of sub-regions to the target network model, respectively;
the output unit includes:
the output subunit is used for outputting the detection results of the first number of the head raising lines;
the first determining subunit is used for determining the severity of the head-up line according to the proportion of the image with the head-up line.
In one possible implementation, the first keypoint and the second keypoint; the first key point is the highest point of the left eyebrow in the key points of the face, and the second key point is the highest point of the right eyebrow in the key points of the face; the intercepting unit includes:
the second determining subunit is configured to determine a first side length of the first area according to the abscissa of the first keypoint and the abscissa of the second keypoint;
a third determining subunit, configured to determine a second side length of the first area according to the translation amount of the ordinate of the first keypoint or the second keypoint;
and the intercepting subunit is used for intercepting the image of the first area according to the first side length and the second side length.
In one possible implementation, the apparatus further includes:
a third determining unit, configured to determine that the target image has no new line if the proportion of the non-skin area is not less than the predetermined proportion.
In a possible implementation manner, the first determining unit is further configured to determine a face key point of a face image sample after the face image sample is acquired;
the intercepting unit is also used for intercepting an image of a second area according to the face key points of the face image sample;
the device further comprises:
the training unit is used for cutting the second area to obtain a second number of sub-areas; and respectively inputting the second number of sub-regions into a network model, and training the network model to obtain the target network model.
In one possible implementation, the apparatus further includes:
a receiving unit, configured to receive the target network model from a training apparatus; the training device is used for training a network model to obtain the target network model.
In a third aspect, an embodiment of the present application further provides a head-up line detection device, including: the system comprises a processor, a memory and an input/output interface, wherein the processor, the memory and the input/output interface are interconnected through lines; wherein the memory stores program instructions; the program instructions, when executed by the processor, cause the processor to perform the respective method according to the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored, the computer program including program instructions, which, when executed by a processor of a raised-head-print detection apparatus, cause the processor to perform the method of the first aspect.
In a fifth aspect, embodiments of the present application provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of the first aspect.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments or the background art of the present application, the drawings required to be used in the embodiments or the background art of the present application will be described below.
Fig. 1 is a schematic flowchart of a method for detecting a raised line according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of a face key point provided in an embodiment of the present application;
FIG. 3 is a schematic diagram of determining a first region according to an embodiment of the present disclosure;
FIG. 4 is a schematic view of an original forehead provided by an embodiment of the present application;
fig. 5 is a schematic view of a forehead after skin color clustering according to an embodiment of the present application;
FIG. 6 is a schematic view of a cutting sub-region provided in an embodiment of the present application;
FIG. 7 is a flowchart illustrating a method for training a target network model according to an embodiment of the present disclosure;
fig. 8 is a schematic structural diagram of a head-up line detection apparatus according to an embodiment of the present disclosure;
fig. 9a is a schematic structural diagram of an input unit according to an embodiment of the present application;
fig. 9b is a schematic structural diagram of an output unit according to an embodiment of the present application;
FIG. 9c is a schematic structural diagram of an intercepting unit provided in an embodiment of the present application;
fig. 10 is a schematic structural diagram of another head-up line detection apparatus according to an embodiment of the present application;
fig. 11 is a schematic structural diagram of another head-up line detection apparatus according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clear, the present application will be further described in detail with reference to the accompanying drawings.
The terms "first," "second," and the like in the description and claims of the present application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, or apparatus.
Referring to fig. 1, fig. 1 is a schematic flow chart of a raised line detection method provided in an embodiment of the present application, where the raised line detection method may be applied to a raised line detection apparatus, the raised line detection apparatus may include a server or a terminal device, and the terminal device may include a mobile phone, a desktop computer, a laptop computer, other devices, and the like.
As shown in fig. 1, the method for detecting the raised line includes:
101. after a target image is acquired, determining key points of a human face in the target image; the target image is an image of the raised line to be detected.
In the embodiment of the application, acquiring the target image can be understood as acquiring the target image by the raised line detection device so as to acquire the target image; it can also be understood that the raised line detection device acquires the target image from other devices, and the embodiment of the present application is not limited to how the raised line detection device acquires the target image. It can be understood that the target image is an image of the raised line to be detected, that is, the target image is an image for which the severity of the raised line needs to be determined, or the target image is an image for which a target network model is needed to predict the severity of the raised line.
In the embodiment of the application, the method for determining the key points of the face in the target image comprises the following steps: can be determined by algorithms such as edge detection robert algorithm, sobel algorithm, etc.; but also by correlation models such as active contour snake models and the like.
Although the above algorithms or models can be used to determine the key points of the face in the target image, the above methods are relatively complex on one hand and relatively poor on the other hand. Therefore, the embodiment of the present application provides a simple method, which is not only simple to implement, but also can effectively determine key points of a face, as follows:
the determining the key points of the face in the target image includes:
and determining the key points of the human face in the target image through a third-party application.
In the embodiment of the application, the third-party application may be a third-party tool package dlib, where dlib is a tool package with a good positioning effect on the open-source face key points and is a C + + open-source tool package including a machine learning algorithm. Tool packages dlib are currently widely used in areas including robotics, embedded devices, mobile phones and large high-performance computing environments. Therefore, the tool kit can be effectively used for positioning the key points of the face to obtain the key points of the face. Specifically, the face key points may be 68 face key points, and so on. As shown in fig. 2, fig. 2 is a schematic diagram of a face key point provided in the embodiment of the present application. It can be seen that the face key points may include key point 0, key point 1 … …, key point 67, i.e. 68 key points.
102. And intercepting an image of a first area from the target image according to the key point of the face in the target image, wherein the first area is an area corresponding to the forehead.
In the embodiment of the application, the forehead area can be directly intercepted from the target image, for example, the forehead area is intercepted based on the fixed length and width. However, when the cutting is performed with a fixed length and width, the size of the forehead of each person is different, so that the first area to be cut is different. Therefore, an embodiment of the present application further provides a method for capturing an image of a first area, which is as follows:
the above-mentioned image of intercepting the first area from the above-mentioned target image according to the key point of human face in the above-mentioned target image, including:
determining a first side length of the first area according to the abscissa of the first key point and the abscissa of the second key point;
determining a second side length of the first area according to the translation amount of the ordinate of the first key point or the second key point;
and cutting the image of the first area according to the first side length and the second side length.
In an embodiment of the present invention, the first key point is a highest point of a left eyebrow in the key points of the face, and the second key point is a highest point of a right eyebrow in the key points of the face. That is, the first keypoint and the second keypoint may be the highest point of the eyebrow, respectively. Specifically, as shown in fig. 2, the key points of the face include key point 19 and key point 24, and are located in the forehead area and are the highest points of the eyebrows, so that the key points 19 and 24 are used as reference key points. It can be understood that when the face key points are located through the face key points, each key point has coordinates, namely pixel point coordinates.
Thus, the abscissa of the keypoints 19 and 24 is taken as the first side length of the first region, and the translation of the ordinate of the keypoint 19 or 24 is taken as the second side length of the first region. The embodiment of the application also provides a method for determining the second side length, namely, the difference value of the abscissa pixels of two key points in the key points of the human face is used as the translation amount. The second side length is determined by the amount of translation, for example, using the amount of difference in the horizontal coordinates between the key point 31 and the key point 32 as the amount of translation. Or, the difference between the horizontal coordinates of the key points 31 and 32 is used as the basic unit, so that the vertical coordinate of the key point 19 or 24 is translated by 1 basic unit upwards, and then the vertical coordinate of the key point 19 or 24 is translated by 4 basic units upwards, and the difference of the two translations of the key point 19 or 24, that is, 3 basic units, is the second side length.
For example, the key points 19 and 24 with the highest two sides of the eyebrow are selected as the reference points, the x1 coordinate is the abscissa of the key point 19, the x2 coordinate is the abscissa of the key point 24, the pixel unit is the difference between the abscissas of the key point 32 and the key point 31 (the difference here represents the difference of absolute values, and the following is similar), the y1 coordinate is the coordinate after the ordinate of the key point 19 is translated upwards by 1 unit, and the y2 coordinate is the coordinate after 4 units of units are averaged in the ordinate of the key point 19, the forehead area is determined by the coordinates (x1, y1, x2, y2), and the image of the forehead area is intercepted as the image of the first area. As shown in fig. 3. Fig. 3 is a schematic diagram of determining a first area according to an embodiment of the present application. In fig. 3, the length of the first region is the difference between the abscissas of the key points 19 and 24, and the width of the first region is the difference between the key point 19 shifted upward by 1 unit and 4 units, i.e. the width of the first region is 3 units.
It can be understood that in the embodiment of the present application, the coordinate system of the abscissa and the ordinate of the first keypoint is in standard agreement, and is in agreement with the coordinate system of the abscissa and the ordinate of the second keypoint. For example, the abscissa of the first keypoint and the abscissa of the second keypoint may be normalized by coordinates in a pixel coordinate system, and the ordinate of the first keypoint or the second keypoint may also be normalized by the pixel coordinate system.
103. And determining the proportion of the non-skin area in the first area by a skin color clustering method, inputting the image of the first area into a target network model under the condition that the proportion of the non-skin area is less than a preset proportion, and outputting the detection result of the new line.
Wherein, the method further comprises: and determining that the target image has no head-up line when the occupation ratio of the non-skin area is not less than the predetermined ratio.
That is, in step 103, the ratio of the non-skin area in the first area is determined by a skin color clustering method, whether the ratio of the non-skin area is smaller than a predetermined ratio is determined, and if the ratio is smaller than the predetermined ratio, the image of the first area is input to a target network model, and the detection result of the new line is output; and under the condition that the ratio is not less than the preset ratio, determining that the target image has no head raising line.
The effect of the proportion of non-skin areas was analyzed specifically as follows:
because the forehead is often shielded by hair, a headband, a hat and the like, and misjudgment is easily caused on the detection of the head-up line, the proportion of non-skin areas in the forehead can be judged by a method based on skin color clustering before the head-up line is detected. The embodiment of the application is based on Red Green Blue (RGB) color space to perform skin color clustering, pixel points in a well-defined skin color range are found out according to an RGB color model, and the pixel points outside the range are set to be black. The range of skin color under the RGB model satisfies the following constraints: the following discriminant is satisfied under uniform illumination: r >95, G >40, B >20, max (R, G, B) -min (R, G, B) >15, abs (R-G) >15, R > G and R > B; in a side-light shooting environment: r >220, G >210, B >170, abs (R-G) < ═ 15, R > B, and G > B. It will be appreciated that the constraints shown above may need to be satisfied simultaneously in a particular implementation, such as the above-shown criteria for uniform illumination, and the above criteria for side-light photography.
If the ratio of the non-skin color area to the total forehead area is not less than 90%, the wrinkle-free condition is directly judged, and therefore, the detection time can be saved. It is understood that the original forehead diagram shown in fig. 4 is only an example, and the original forehead diagram may be in other colors, which are not shown here.
It can be understood that, in the embodiment of the present application, in the case where the proportion of the non-skin area is smaller than the predetermined example, the image of the first area may be directly input to the target network model to detect the severity of the raised line. However, the embodiment of the application also provides a method, which can better detect the raised lines and improve the detection efficiency and accuracy. As follows:
the inputting the image of the first area into a target network model and outputting the detection result of the new line comprises:
cutting the first area to obtain a first number of sub-areas, and respectively inputting images containing the first number of sub-areas into the target network model;
and outputting the detection results of the first number of the head-up lines, and determining the severity of the head-up lines according to the occupation ratio of the images with the head-up lines.
Specifically, the first region is divided into a first number of sub-regions, the first number is at least 2, and the pixel size of each sub-region is the same. As shown in fig. 6, fig. 6 is a schematic diagram of a sub-region provided in the embodiment of the present application. In fig. 6, the first region is cut into 30 sub-regions, 30 being the first number. In order to ensure that the sub-regions are the same, some regions that may occur during the cutting process cannot keep the same pixel size as other regions, and therefore, the embodiments of the present application may discard these regions, such as the regions represented by the bottom row in fig. 6.
After the first region is switched to a first number of sub-regions, each sub-region may be scaled down or up to 227 x 227 pixels for input to the target network model. After the first number of sub-regions are respectively input into the target network model, a first number of detection results are input, and the detection results can indicate whether the sub-regions have the new line or not. After the first number of detection results is obtained, the severity of the head-up streak can be determined according to the proportion of the head-up streak. Wherein, the severity of the raised lines can be classified into severe, moderate, mild and none. Specifically, if the ratio of the head-up line is greater than or equal to 0.5, the head-up line is severe; if the ratio of the head-up line is more than 0.2 and less than 0.5, the head-up line is moderate; if the ratio of the head-up lines is more than 0.05 and less than or equal to 0.2, the head-up lines are light, and if the ratio of the head-up lines is less than or equal to 0.05, the head-up lines are not present. It is understood that the above is only an example and should not be interpreted as a limitation of the embodiments of the present application.
It is understood that the target network model in the embodiment of the present application may be trained by the raised-head-print detection apparatus itself, or the target network model may also be a network model that is sent to the raised-head-print detection apparatus after being trained by other apparatuses, such as a training apparatus. The implementation manner of the apparatus for detecting the new line can refer to the method shown in fig. 7. In a case where the target network model is sent to the new line detection device by the training device, before the image of the first area is input to the target network model, the method further includes:
receiving the target network model from a training device; the training device is used for training a network model to obtain the target network model.
In this embodiment of the present application, the training apparatus may be any device, such as a server, a terminal device, and the like. And the embodiment of the present application does not limit how the training apparatus trains the target network model.
In the embodiment of the application, after the image of the first area is obtained, the proportion of the non-skin area is determined by a skin color clustering method, and when the proportion of the non-skin area is smaller than a preset proportion, the image of the first area is input into a target network model; the situation that the target network model cannot be effectively identified due to the fact that the image of the first area is shielded by other objects such as hair, a headband or a hat can be effectively avoided; by implementing the embodiment of the application, the occupation ratio of the non-skin area can be preferentially determined, so that whether the new line is detected by inputting the new line into the target network model is determined according to the occupation ratio of the non-skin area, the accuracy of outputting the target network model can be improved, and the output efficiency is improved.
For the raised line detection method shown in fig. 1, the target network model is a trained network model, that is, the target network model is obtained by training the network model. Therefore, an embodiment of the present application further provides a method for training a network model, referring to fig. 7, fig. 7 is a flowchart illustrating a method for training a target network model provided in an embodiment of the present application, and as shown in fig. 7, the method for training includes:
701. after the face image sample is collected, determining the face key points of the face image sample.
The head raising line detection device can collect the face image sample and can also collect the face image sample through other devices. The following describes how to collect face image samples by taking the head-up line detection apparatus as an example. The above-mentioned face image sample of gathering includes: and acquiring a third number of face images as the face image samples.
Wherein the third number is greater than or equal to 300 as described above. In the embodiment of the application, the collecting of the face image sample is specifically collecting a face image, the number of the face image is at least 300, and the face image is a face image with a forehead image. In order to train the network model better, the face image sample may include an image with a head-up line on the forehead, and may also include an image without a head-up line on the forehead. The embodiment of the application does not limit what kind of device is adopted to collect the face image sample. Such as a mobile phone, a camera, etc.
The number of the collected face image samples is at least 300, and the training effect of the face image samples less than 300 is not good than or equal to 300 in the training process. On the other hand, when the number of the collected face image samples is larger than or equal to 300, the generalization capability of the trained network model is better.
In an embodiment of the present application, the determining the face key points of the face image sample includes:
and determining the face key points of the face image sample through a third-party application.
The third-party application may be a third-party toolkit dlib, and the implementation of the method for determining the face key point may refer to the specific implementation shown in fig. 1, which is not described in detail here.
702. And intercepting the image of the second area according to the face key points of the face image sample.
In the embodiment of the application, the second region is a region corresponding to the forehead, for example, the first side length of the second region can be determined according to the abscissa of the key points 19 and 24 in the key points of the face; the second side length of the second region is determined from the translation of the ordinate of the keypoint 19 or 24. The method for intercepting the second area may specifically refer to the implementation manner shown in fig. 1, and details are not repeated here.
703. And cutting the second area to obtain a second number of sub-areas.
Because the number of face images in the face image sample is at least 300, after each image is cut, if each image is cut into images of 40 sub-regions, after the images of 40 sub-regions are obtained, whether each sub-region has a new line or not can be marked to prepare for a subsequent training network model. In this embodiment of the application, when each image in the face image sample is cut, the number of sub-regions cut between the images may be the same or different, and this embodiment of the application is not limited. If the face image sample includes sample 1 and sample 2, the number of the sub-regions obtained by cutting sample 1 is 40, and the number of the sub-regions obtained by cutting sample 2 may be 40 or 30.
704. And inputting the second number of sub-regions into a network model respectively, and training the network model to obtain the target network model.
The network model may be trained by a (VGG) network proposed by a visual geometry group, namely VGGNet, or may be trained by GoogleNet or ResNet, and so on. And the network model in the embodiment of the application can also be trained through an AlexNet network.
It can be understood that, because the face image samples shown in the embodiment of the present application are sufficient, in the training process, all layers of the AlexNet network can be trained, that is, both convolutional layers and fully-connected layers can be trained. Specifically, the AlexNet network is trained according to a loss function of the detection result (e.g., whether wrinkles exist) output by the AlexNet network and the detection result of the mark.
It can be understood that the AlexNet network in the embodiment of the present application is 2012, and the Hinton task group, in order to prove the potential of deep learning, first participates in the ImageNet image recognition game, and captures champions at a time through the constructed CNN network, AlexNet, and rolls on the classification performance of the second name. However, the embodiment of the application is better suitable for the raised head line detection through training.
In the embodiment of the application, the network model is trained by the method, so that the efficiency of the trained network model can be effectively improved, namely, the accuracy of the output detection result of the network model is improved.
It will be appreciated that the method embodiments shown in fig. 1 and 7 are of particular importance, and that implementations not described in detail in one embodiment may also refer to other embodiments.
The method of the embodiments of the present application is set forth above in detail and the apparatus of the embodiments of the present application is provided below.
Referring to fig. 8, fig. 8 is a schematic structural diagram of a head-up line detection apparatus according to an embodiment of the present application, and as shown in fig. 8, the head-up line detection apparatus includes:
a first determining unit 801, configured to determine a face key point in a target image after the target image is acquired; wherein, the target image is an image of the raised line to be detected;
an intercepting unit 802, configured to intercept an image of a first region from the target image according to a key point of a face in the target image, where the first region is a region corresponding to a forehead;
a second determining unit 803, configured to determine the proportion of the non-skin area in the first area by a skin color clustering method;
an input unit 804, configured to input the image of the first area to a target network model when the proportion of the non-skin area is smaller than a predetermined proportion;
an output unit 805, configured to output a detection result of the above-mentioned raised line.
In the embodiment of the application, after the image of the first area is obtained, the proportion of the non-skin area is determined by a skin color clustering method, and when the proportion of the non-skin area is smaller than a preset proportion, the image of the first area is input into a target network model; the situation that the target network model cannot be effectively identified due to the fact that the image of the first area is shielded by other objects such as hair, a headband or a hat can be effectively avoided; by implementing the embodiment of the application, the occupation ratio of the non-skin area can be preferentially determined, so that whether the new line is detected by inputting the new line into the target network model is determined according to the occupation ratio of the non-skin area, the accuracy of outputting the target network model can be improved, and the output efficiency is improved.
Specifically, as shown in fig. 9a, the input unit 804 includes:
a cutting subunit 8041, configured to cut the first region to obtain a first number of sub-regions;
an input subunit 8042, configured to input the images including the first number of sub-areas to the target network model respectively;
as shown in fig. 9b, the output unit 805 includes:
an output subunit 8051, configured to output the detection result of the first number of raised lines;
the first determining subunit 8052 is configured to determine the severity of the above-mentioned raised line according to the proportion of the image with the raised line.
As shown in fig. 9c, the intercepting unit 802 includes:
a second determining subunit 8021, configured to determine a first side length of the first area according to an abscissa of the first key point and an abscissa of the second key point;
a third determining subunit 8022, configured to determine a second side length of the first area according to the translation amount of the ordinate of the first key point or the second key point;
a clipping sub-unit 8023, configured to clip the image of the first area according to the first side length and the second side length.
Optionally, as shown in fig. 10, the apparatus further includes:
a third determining unit 806, configured to determine that the target image does not have the new line if the proportion of the non-skin area is not less than the predetermined proportion.
Specifically, the first determining unit 801 is further configured to determine a face key point of the face image sample after the face image sample is acquired;
the intercepting unit 802 is further configured to intercept an image of a second region according to the face key point of the face image sample;
as shown in fig. 10, the above apparatus further includes:
a training unit 807 for cutting said second region to obtain a second number of sub-regions; and inputting the second number of sub-regions into a network model respectively, and training the network model to obtain the target network model.
Optionally, as shown in fig. 10, the apparatus further includes:
a receiving unit 808, configured to receive the target network model from a training apparatus; the training device is used for training a network model to obtain the target network model.
The implementation of each unit in the embodiment of the present application may also correspond to the corresponding description of the method embodiments shown in fig. 1 and fig. 7, and is not described in detail here.
Referring to fig. 11, fig. 11 is a schematic structural diagram of a head-up streak detection apparatus provided in an embodiment of the present application, where the head-up streak detection apparatus includes a processor 1101, a memory 1102, and an input/output interface 1103, and the processor 1101, the memory 1102, and the input/output interface 1103 are connected to each other through a bus.
The memory 1102 includes, but is not limited to, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CD-ROM), and the memory 1102 is used for related instructions and data.
The input/output interface 1103 can communicate with another device through the input/output interface.
The processor 1101 may be one or more Central Processing Units (CPUs), and in the case where the processor 1101 is one CPU, the CPU may be a single-core CPU or a multi-core CPU.
Specifically, the implementation of each operation may also correspond to the corresponding description of the method embodiments shown in fig. 1 and fig. 7. And the implementation of the respective operations may also correspond to the respective description of the apparatus embodiments shown in fig. 8, 9a to 9c and 10.
As in one embodiment, the processor 1101 may be configured to perform the method shown in step 101, and as such, the processor 1101 may also be configured to perform the method performed by the first determining unit 801, the intercepting unit 802, the second determining unit 803, and the like.
As another example, in an embodiment, the processor 1101 may be configured to acquire a facial image sample or acquire a target image, or acquire the facial image sample or the target image through the input/output interface 1103, and how to acquire the facial image sample or the target image is not limited in this embodiment of the application.
Also for example, in one embodiment, the input output interface 1103 can also be used to perform a method performed by the receiving unit 808.
One of ordinary skill in the art will appreciate that all or part of the processes in the methods of the above embodiments may be implemented by hardware related to instructions of a computer program, which may be stored in a computer-readable storage medium, and when executed, may include the processes of the above method embodiments. And the aforementioned storage medium includes: various media capable of storing program codes, such as ROM or RAM, magnetic or optical disks, etc.
Claims (5)
1. A head-up line detection method is characterized by comprising the following steps:
after a target image is obtained, determining a face key point in the target image through a third-party application; the target image is an image of the to-be-detected raised line;
determining the first side length of the first area according to the abscissa of the first key point and the abscissa of the second key point; determining a second side length of the first area according to the translation amount of the ordinate of the first key point or the second key point; cutting an image of the first area according to the first side length and the second side length, wherein the first area is an area corresponding to the forehead, the first key point is the highest point of the left eyebrow in the key points of the human face, the first key point comprises a key point 19, the second key point is the highest point of the right eyebrow in the key points of the human face, the second key point comprises a key point 24, the translation amount is the difference of the horizontal coordinates between the key point 31 and the key point 32, or, taking the difference of the horizontal coordinates between the key point 31 and the key point 32 as the basic unit, translating the vertical coordinate of the key point 19 or the key point 24 upward by 1 basic unit, and then translating the vertical coordinate of the key point 19 or the key point 24 upward by 4 basic units, the difference between the two translations of the key point 19 or the key point 24, that is, 3 basic units, is the second side length;
determining the proportion of a non-skin area in the first area by a skin color clustering method, wherein the non-skin area is determined according to pixel points outside a skin color range, the pixel points outside the skin color range are pixel points outside the pixel points in the skin color range, and the pixel points in the skin color range meet the following constraints: under uniform illumination, the following conditions are satisfied: r >95, G >40, B >20, max (R, G, B) -min (R, G, B) >15, abs (R-G) >15, R > G and R > B; satisfying the following conditions under side light shooting: r >220, G >210, B >170, abs (R-G) < ═ 15, R > B, and G > B;
under the condition that the proportion of the non-skin area is smaller than a preset proportion, inputting the image of the first area into a target network model, and outputting the detection result of the raised line;
the inputting the image of the first area into a target network model and outputting the detection result of the new line comprises:
cutting the first area to obtain a first number of sub-areas, respectively reducing or amplifying the images containing the first number of sub-areas to 227 × 227 pixels, and inputting the images into the target network model;
outputting the detection result of the first number of head-up lines, and determining the severity of the head-up lines according to the ratio of the images with the head-up lines, wherein the ratio of the head-up lines is greater than or equal to 0.5, which indicates that the head-up lines are severe, the ratio of the head-up lines is greater than 0.2 and less than 0.5, which indicates that the head-up lines are moderate, the ratio of the head-up lines is greater than 0.05 and less than or equal to 0.2, which indicates that the head-up lines are mild, and the ratio of the head-up lines is less than or equal to 0.05, which indicates that no head-up lines exist;
before the inputting the image of the first area into the target network model, the method further comprises:
after a third number of face images are collected to serve as face image samples, determining face key points of the face image samples through the third-party application, wherein the third number is more than or equal to 300;
intercepting an image of a second area according to the face key points of the face image sample;
cutting the second area to obtain a second number of sub-areas;
and respectively inputting the sub-regions of the second number into a network model, and training the network model to obtain the target network model.
2. The method of claim 1, further comprising:
determining that the target image has no new line if the proportion of the non-skin area is not less than the predetermined proportion.
3. A raised line detection device, comprising:
the first determining unit is used for determining the key points of the human face in the target image through a third-party application after the target image is obtained; the target image is an image of the to-be-detected raised line;
the intercepting unit is used for determining the first side length of the first area according to the abscissa of the first key point and the abscissa of the second key point; determining a second side length of the first area according to the translation amount of the ordinate of the first key point or the second key point; cutting an image of the first area according to the first side length and the second side length, wherein the first area is an area corresponding to the forehead, the first key point is the highest point of the left eyebrow in the key points of the human face, the first key point comprises a key point 19, the second key point is the highest point of the right eyebrow in the key points of the human face, the second key point comprises a key point 24, the translation amount is the difference of the horizontal coordinates between the key point 31 and the key point 32, or, taking the difference of the horizontal coordinates between the key point 31 and the key point 32 as the basic unit, translating the vertical coordinate of the key point 19 or the key point 24 upward by 1 basic unit, and then translating the vertical coordinate of the key point 19 or the key point 24 upward by 4 basic units, the difference between the two translations of the key point 19 or the key point 24, that is, 3 basic units, is the second side length;
a second determining unit, configured to determine a proportion of a non-skin region in the first region by a skin color clustering method based on a red, green, blue, RGB color space, where the non-skin region is determined according to pixel points outside a skin color range, the pixel points outside the skin color range are pixel points outside the pixel points within the skin color range, and the pixel points within the skin color range satisfy the following constraints: r >95, G >40, B >20, max (R, G, B) -min (R, G, B) >15, abs (R-G) >15, R > G and R > B; satisfying the following conditions under side light shooting: r >220, G >210, B >170, abs (R-G) < ═ 15, R > B, and G > B;
an input unit, configured to input the image of the first area to a target network model when the proportion of the non-skin area is smaller than a predetermined proportion;
the output unit is used for outputting the detection result of the head raising line;
the input unit includes:
a cutting subunit, configured to cut the first region, to obtain a first number of sub-regions;
an input subunit, configured to reduce or enlarge the image including the first number of sub-regions to 227 × 227 pixels, and input the image to the target network model;
the output unit includes:
the output subunit is used for outputting the detection results of the first number of the head raising lines;
a first determining subunit, configured to determine a severity of the head-up streak according to a ratio of an image with the head-up streak, where the ratio of the head-up streak is greater than or equal to 0.5, which indicates that the head-up streak is severe, the ratio of the head-up streak is greater than 0.2 and less than 0.5, which indicates that the head-up streak is moderate, the ratio of the head-up streak is greater than 0.05 and less than or equal to 0.2, which indicates that the head-up streak is mild, and the ratio of the head-up streak is less than or equal to 0.05, which indicates that the head-up streak is not present;
the first determining unit is further configured to determine, by the third party application, a face key point of the face image sample after acquiring a third number of face images as the face image sample, where the third number is greater than or equal to 300;
the intercepting unit is also used for intercepting an image of a second area according to the face key points of the face image sample;
the device further comprises:
the training unit is used for cutting the second area to obtain a second number of sub-areas; and respectively inputting the second number of sub-regions into a network model, and training the network model to obtain the target network model.
4. The head raising line detection device is characterized by comprising a processor, a memory and an input/output interface, wherein the processor, the memory and the input/output interface are interconnected through a line; wherein the memory stores program instructions that, when executed by the processor, cause the processor to perform the method of any of claims 1 to 2.
5. A computer-readable storage medium, in which a computer program is stored, the computer program comprising program instructions which, when executed by a processor of a raised-head-print detection apparatus, cause the processor to carry out the method of any one of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810554520.5A CN109033935B (en) | 2018-05-31 | 2018-05-31 | Head-up line detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810554520.5A CN109033935B (en) | 2018-05-31 | 2018-05-31 | Head-up line detection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109033935A CN109033935A (en) | 2018-12-18 |
| CN109033935B true CN109033935B (en) | 2021-09-28 |
Family
ID=64611935
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810554520.5A Active CN109033935B (en) | 2018-05-31 | 2018-05-31 | Head-up line detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109033935B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110188713A (en) * | 2019-06-03 | 2019-08-30 | 北京字节跳动网络技术有限公司 | Method and apparatus for output information |
| CN111160169B (en) * | 2019-12-18 | 2024-03-15 | 中国平安人寿保险股份有限公司 | Face detection method, device, equipment and computer readable storage medium |
| CN111126344B (en) * | 2019-12-31 | 2023-08-01 | 杭州趣维科技有限公司 | Method and system for generating key points of forehead of human face |
| CN111275610B (en) * | 2020-01-08 | 2023-08-18 | 杭州小影创新科技股份有限公司 | Face aging image processing method and system |
| CN113298593A (en) * | 2020-07-16 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Commodity recommendation and image detection method, commodity recommendation and image detection device, commodity recommendation and image detection equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5835616A (en) * | 1994-02-18 | 1998-11-10 | University Of Central Florida | Face detection using templates |
| CN101877054A (en) * | 2009-11-23 | 2010-11-03 | 北京中星微电子有限公司 | Method and device for determining age of face image |
| CN102479322A (en) * | 2010-11-30 | 2012-05-30 | 财团法人资讯工业策进会 | System, apparatus and method for analyzing facial defect by facial image with angle |
| CN103425985A (en) * | 2013-08-28 | 2013-12-04 | 山东大学 | Method for detecting forehead wrinkles on face |
| CN103632132A (en) * | 2012-12-11 | 2014-03-12 | 广西工学院 | Face detection and recognition method based on skin color segmentation and template matching |
| CN106529429A (en) * | 2016-10-27 | 2017-03-22 | 中国计量大学 | Image recognition-based facial skin analysis system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107808136B (en) * | 2017-10-31 | 2020-06-12 | Oppo广东移动通信有限公司 | Image processing method, image processing device, readable storage medium and computer equipment |
-
2018
- 2018-05-31 CN CN201810554520.5A patent/CN109033935B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5835616A (en) * | 1994-02-18 | 1998-11-10 | University Of Central Florida | Face detection using templates |
| CN101877054A (en) * | 2009-11-23 | 2010-11-03 | 北京中星微电子有限公司 | Method and device for determining age of face image |
| CN102479322A (en) * | 2010-11-30 | 2012-05-30 | 财团法人资讯工业策进会 | System, apparatus and method for analyzing facial defect by facial image with angle |
| CN103632132A (en) * | 2012-12-11 | 2014-03-12 | 广西工学院 | Face detection and recognition method based on skin color segmentation and template matching |
| CN103425985A (en) * | 2013-08-28 | 2013-12-04 | 山东大学 | Method for detecting forehead wrinkles on face |
| CN106529429A (en) * | 2016-10-27 | 2017-03-22 | 中国计量大学 | Image recognition-based facial skin analysis system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109033935A (en) | 2018-12-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109033935B (en) | Head-up line detection method and device | |
| WO2020063744A1 (en) | Face detection method and device, service processing method, terminal device, and storage medium | |
| US9978003B2 (en) | Utilizing deep learning for automatic digital image segmentation and stylization | |
| WO2017190646A1 (en) | Facial image processing method and apparatus and storage medium | |
| US9349076B1 (en) | Template-based target object detection in an image | |
| WO2019169772A1 (en) | Picture processing method, electronic apparatus, and storage medium | |
| WO2019100282A1 (en) | Face skin color recognition method, device and intelligent terminal | |
| WO2019033572A1 (en) | Method for detecting whether face is blocked, device and storage medium | |
| WO2022199583A1 (en) | Image processing method and apparatus, computer device, and storage medium | |
| US20200387748A1 (en) | Facial image data collection method, apparatus, terminal device and storage medium | |
| KR101141643B1 (en) | Apparatus and Method for caricature function in mobile terminal using basis of detection feature-point | |
| WO2019033571A1 (en) | Facial feature point detection method, apparatus and storage medium | |
| CN109448007B (en) | Image processing method, image processing apparatus, and storage medium | |
| US11521334B2 (en) | Augmented reality experiences of color palettes in a messaging system | |
| WO2021139475A1 (en) | Facial expression recognition method and apparatus, device, computer-readable storage medium and computer program product | |
| US20210150691A1 (en) | Method and apparatus for performing damage segmentation on vehicle damage image | |
| US20210312533A1 (en) | Identification of physical products for augmented reality experiences in a messaging system | |
| CN111429338B (en) | Method, apparatus, device and computer readable storage medium for processing video | |
| US11663463B2 (en) | Center-biased machine learning techniques to determine saliency in digital images | |
| WO2019033568A1 (en) | Lip movement capturing method, apparatus and storage medium | |
| US20210312678A1 (en) | Generating augmented reality experiences with physical products using profile information | |
| WO2019033567A1 (en) | Method for capturing eyeball movement, device and storage medium | |
| WO2024001095A1 (en) | Facial expression recognition method, terminal device and storage medium | |
| WO2022135574A1 (en) | Skin color detection method and apparatus, and mobile terminal and storage medium | |
| EP4128026A1 (en) | Identification of physical products for augmented reality experiences in a messaging system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 518000 Guangdong science and technology innovation and Research Institute, Shenzhen, Shenzhen, Nanshan District No. 6, science and technology innovation and Research Institute, Shenzhen, D 10, 1004, 10 Patentee after: Shenzhen Hetai intelligent home appliance controller Co.,Ltd. Address before: 518000 Guangdong science and technology innovation and Research Institute, Shenzhen, Shenzhen, Nanshan District No. 6, science and technology innovation and Research Institute, Shenzhen, D 10, 1004, 10 Patentee before: SHENZHEN H&T DATA RESOURCES AND CLOUD TECHNOLOGY Ltd. |
|
| CP01 | Change in the name or title of a patent holder |