CN110688962B

CN110688962B - Face image processing method, user equipment, storage medium and device

Info

Publication number: CN110688962B
Application number: CN201910939068.9A
Authority: CN
Inventors: 孙碧亮; 刘凯能
Original assignee: Wuhan Ar Show Software Co ltd
Current assignee: Wuhan Ar Show Software Co ltd
Priority date: 2019-09-29
Filing date: 2019-09-29
Publication date: 2022-05-20
Anticipated expiration: 2039-09-29
Also published as: CN110688962A

Abstract

The invention discloses a face image processing method, user equipment, a storage medium and a device. The method comprises the steps of obtaining a target image to be processed, identifying the position of a face in the target image, segmenting the identified face according to the position to generate a sectional image, and mapping the sectional image to the corresponding building block particle color to generate a face modular image. The identified human face is segmented and then mapped to the color of the building block particles to generate a human face building block image, so that the technical problem that the image is seriously distorted easily in the human face building block processing process is solved.

Description

Face image processing method, user equipment, storage medium and device

Technical Field

The present invention relates to the field of image processing, and in particular, to a method, a user equipment, a storage medium, and an apparatus for processing a face image.

Background

Face recognition is a biometric technology for identifying an identity based on facial feature information of a person. A series of related technologies, also called face recognition and face recognition, are used to capture an image or video stream containing a human face by a camera or a video camera, automatically detect and track the human face in the image, and further perform face recognition on the detected human face.

The face building block is a special image formed by identifying the face in the image and processing the color of the image and the like and is used for making portrait pictures, souvenirs, gifts and the like

In the existing face building block processing process, the face building block is manufactured by processing such as image binaryzation, image edge detection and the like, and the situation of serious image distortion is easy to occur.

The above is only for the purpose of assisting understanding of the technical solution of the present invention, and does not represent an admission that the above is the prior art.

Disclosure of Invention

The invention mainly aims to provide a face image processing method, user equipment, a storage medium and a device, and aims to solve the technical problem that images are easy to be seriously distorted in a face building block processing process in the prior art.

In order to achieve the above object, the present invention provides a face image processing method, comprising the following steps:

acquiring a target image to be processed;

identifying the position of the face in the target image;

segmenting the recognized face according to the position to generate a sectional image;

and mapping the sectional image to the corresponding building block particle color to generate the face building block image.

Preferably, the recognizing the position of the face in the target image includes:

and identifying the face in the target image based on a preset convolutional neural network model, and acquiring the position of the face.

Preferably, the segmenting the identified face according to the position, and generating the sectional image includes:

identifying a communication area where the face is located according to the position where the face is located;

and segmenting the face based on the identified connected region to generate a sectional image.

Preferably, the segmenting the human face based on the identified connected regions, and the generating the sectional image includes:

acquiring the maximum height and the maximum width of a communication area;

determining a compression ratio based on the maximum height, the maximum width and the height and width of the building block floor grain pixels;

determining the height and width of the compressed facial features in the target image according to the compression ratio;

when the compressed height is smaller than the height of the building block bottom plate particle pixel or/and the compressed width is smaller than the width of the building block bottom plate particle pixel, the maximum height and the maximum width of the communication area are adjusted, and the adjusted communication area is used as a cutout image.

Preferably, the adjusting the maximum height and the maximum width of the connected region, and taking the adjusted connected region as a matting image includes:

reducing the maximum height and the maximum width of the communication area according to a preset rule;

when the height after compression is greater than the height of building block bottom plate particle pixel and the width after compression is greater than the width of building block bottom plate particle pixel, the current connected region is used as a cutout image.

Preferably, the mapping the scratch image to the corresponding building block particle color to generate the face building block image includes:

dividing the sectional image into a plurality of characteristic subregions;

determining the color characteristic value of each characteristic subregion according to a preset formula;

matching the color characteristic value with the color of the building blocks according to the color characteristic value and the color value interval of the building blocks to be spliced;

and performing threshold segmentation on each characteristic subregion according to the color characteristic value to generate a face cordwood image.

Preferably, the characteristic sub-regions include an eye characteristic sub-region, an eyebrow characteristic sub-region, a nose characteristic sub-region, a mouth characteristic sub-region, and a cheek characteristic sub-region.

In order to achieve the above object, the present invention further provides a user equipment, where the user equipment includes: the face image processing method comprises a memory, a processor and a face image processing program stored on the memory and capable of running on the processor, wherein the steps of the face image processing method are realized when the face image processing program is executed by the processor.

In order to achieve the above object, the present invention further provides a storage medium, in which a face image processing program is stored, and the face image processing program, when executed by a processor, implements the steps of the face image processing method as described above.

In order to achieve the above object, the present invention further provides a face image processing apparatus, including:

the acquisition module is used for acquiring a target image to be processed;

the recognition module is used for recognizing the position of the face in the target image;

the segmentation module is used for segmenting the identified human face according to the position to generate a sectional image;

and the mapping module is used for mapping the sectional image to the corresponding building block particle color so as to generate the face cordwood image.

According to the technical scheme, the position of the face in the target image is identified by acquiring the target image to be processed, the identified face is segmented according to the position to generate a scratch image, and the scratch image is mapped to the corresponding building block particle color to generate a face building block image. According to the technical scheme, the face building block image is generated by mapping the segmented face to the color of the building block particles, so that the technical problem that the image is seriously distorted easily in the face building block processing process is solved.

Drawings

FIG. 1 is a schematic diagram of a user equipment architecture of a hardware operating environment according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of a first embodiment of a face image processing method according to the present invention;

FIG. 3 is a detailed flowchart of step S30 in FIG. 2;

FIG. 4 is a detailed flowchart of step S31 in FIG. 3;

FIG. 5 is a detailed flowchart of step S324 in FIG. 4;

FIG. 6 is a detailed flowchart of step S40 in FIG. 2;

FIG. 7 is a functional block diagram of a face image processing apparatus according to a first embodiment of the present invention;

FIG. 8 is a diagram of a target image in the first embodiment of the face image processing method of the present invention;

FIG. 9 is a sectional image of a first embodiment of the face image processing method of the present invention;

FIG. 10 is a block diagram of a human face according to the first embodiment of the facial image processing method of the present invention.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Fig. 1 is a schematic structural diagram of a user equipment in a hardware operating environment according to an embodiment of the present invention. The user equipment comprises a processor 1001, a network interface 1004, a memory 1005 and a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001. The network interface 1004 may include a standard wired interface, a wireless interface (e.g., WI-FI interface). Optionally, the user device 104 may also include a user interface. The user Interface may include an I/O (Input/Output) Interface such as a USB Interface, and a Video Interface such as an HDMI (High Definition Multimedia Interface), an SDI (Digital component serial) Interface, a VGA (Video Graphics Array) Interface, and a DVI (Digital Visual Interface) Interface, etc. The I/O device and the display device can be in connected communication with the user device through the I/O interface and the video interface, respectively. The I/O device may be an input device such as a keyboard and a mouse. Those skilled in the art will appreciate that the user equipment configuration shown in fig. 1 does not constitute a limitation of the user equipment and may be more or less components than those shown.

The memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, and an access display program. The user equipment can be electronic equipment such as a personal computer or a smart phone.

In the user equipment shown in fig. 1, the network interface 1004 is mainly used for connecting a terminal and communicating data with the terminal; and the processor 1001 may be configured to call the access display program stored in the memory 1005 and perform the following operations:

acquiring a target image to be processed;

identifying the position of the face in the target image;

acquiring the maximum height and the maximum width of a communication area;

dividing the sectional image into a plurality of characteristic subregions;

Based on the hardware structure, the embodiment of the face image processing method is provided.

Referring to fig. 2, fig. 2 is a flowchart illustrating a first embodiment of a face image processing method according to the present invention.

In a first embodiment, the face image processing method includes the steps of:

step S10: acquiring a target image to be processed;

it is worth mentioning that the image may be uploaded locally from the user device or may be obtained by instant shooting. In this embodiment, referring to the face image shown in fig. 8, the target image includes a face, so as to facilitate subsequent processing of the face.

Step S20: identifying the position of the face in the target image;

in order to extract a face image from a target image, the position of the face needs to be recognized first. The position in this embodiment refers to a region where the face is located in the target image. The face usually includes hair, forehead, eyebrow, cheek, nose, ear, mouth, and even ornaments, cap, neck, etc.

Step S30: segmenting the recognized face according to the position to generate a sectional image;

in this embodiment, 1500 images including only a human face and having a large background difference are used to obtain a database of human face images, so as to train a preset convolutional neural network model to recognize the human face. Because the final face is mapped to the building block bottom plate with limited area, too large non-face areas, such as too long hat, hair, too many neck, shoulders and the like, can reduce the features of the face, and finally the effect of building blocks is poor. Therefore, the human face needs to be segmented.

The principle of segmentation is that the proportion of five sense organs of the face is as large as possible, the proportion of features above the head is larger than that below the head, and the features of the five sense organs can still be kept from losing after being reduced to the size of pixels corresponding to the building block bottom plate. In general, a human face is characterized by a height from the chin to the forehead that is greater than the width of the cheeks. Therefore, the height from the chin to the forehead is used as the height of the divided seeds for searching the connected domain.

It is worth mentioning that, in order to meet diversified requirements of customers and improve user experience, the obtained cutout image can be grayed, and in the graying process, the RGB channel is converted according to the standard. Refer to the matte image shown in fig. 9.

Step S40: and mapping the sectional image to the corresponding building block particle color to generate the face building block image.

Because the colors of the building block particles are limited, and the cutout image is colored, the cutout image needs to be mapped to the corresponding building block particle colors, and the cutout image is subjected to face building block by adopting a mapping method in the embodiment. Reference is made to the face-cordwood image shown in figure 10.

According to the technical scheme, the face building block image is generated by mapping the segmented face to the color of the building block particles, so that the technical problem that the image is seriously distorted easily in the face building block processing process is solved.

In a first embodiment, the identifying a position of a face in the target image includes:

The convolutional neural network model is a feedforward neural network including convolutional calculation and having a deep structure, is one of typical algorithms for deep learning, and is widely applied to visual recognition, image processing and the like.

Referring to fig. 3, in the first embodiment, the segmenting the recognized face according to the position and generating the matte image includes:

step S31: identifying a communication area where the face is located according to the position where the face is located;

it should be noted that the Connected Component generally refers to an image area (Blob) formed by foreground pixels having the same pixel value and adjacent positions in the image.

Step S31: and segmenting the face based on the identified connected region to generate a sectional image.

After the face position is identified through the preset convolutional neural network model, in order to further improve the effect of the block batch image, the face is segmented through the communication area so as to remove too many non-face areas.

Referring to fig. 4, the segmenting the human face based on the identified connected regions and generating the sectional image includes:

step S311: acquiring the maximum height and the maximum width of a communication area;

after the connected region is found, the height Hmax and the width Wmax at that time are recorded.

Step S312: determining a compression ratio based on the maximum height, the maximum width and the height and width of the building block floor grain pixels;

the method comprises the steps of obtaining height Hbdst of building block bottom plate particle pixels, calculating a first ratio of the maximum height of a communication area to the height of the building block bottom plate particle pixels, calculating a second ratio of the maximum width of the communication area to the width of the building block bottom plate particle pixels, and obtaining a compression ratio R value according to the first ratio and the second ratio.

Step S313: determining the height and width of the compressed facial features in the target image according to the compression ratio;

and obtaining the height and width of the facial features, and calculating the height and width of the compressed facial features according to the R value. For example, the compressed eye has a height and a width of Heye and Weye, respectively.

Step S314: when the compressed height is smaller than the height of the building block bottom plate particle pixel or/and the compressed width is smaller than the width of the building block bottom plate particle pixel, the maximum height and the maximum width of the communication area are adjusted, and the adjusted communication area is used as a cutout image.

If Heye is less than the height of the building block bottom particle pixel and Weye is less than the width of the building block bottom particle pixel, the scaling ratio at this time is considered to be too large, and details are lost. Hmax and Wmax need to be reduced. If Heye and Weye are larger than one pixel, the feature of five sense organs can be considered to be reserved, and the current segmentation value is acceptable.

Referring to fig. 5, the adjusting the maximum height and the maximum width of the connected region, and the using the adjusted connected region as a matting image includes:

step S3141: reducing the maximum height and the maximum width of the communication area according to a preset rule;

when Heye and Weye are less than one pixel value, the values of Hmax and Wmax need to be reduced, and the above-mentioned processes of solving for the ratio R, Heye and Weye are repeated.

Step S3142: and when the compressed height is greater than the height of the building block bottom plate particle pixel and the compressed width is greater than the width of the building block bottom plate particle pixel, taking the current connected region as a sectional image.

The equivalent of Heye and Weye is obtained, and the values are all more than one pixel. I.e., the features may be preserved, Hmax and Wmax are deemed appropriate.

In some special cases, if the aspect ratio of the face is too different from that of the building block bottom plate (usually, the aspect ratio of the face is 3:4 or 4:5), the face may be deformed after being zoomed. Thus, the face region can be extracted by adopting a direct cutting and little outline extension method, and the image extraction result obtained in the above step is ignored.

Referring to fig. 6, in a second embodiment, the mapping of the scratch images to the corresponding block particle colors to generate a face-building image includes:

step S41: dividing the sectional image into a plurality of characteristic subregions;

it should be noted that the face feature region can be roughly divided into 5 regions of an eye area, an eyebrow area, a nose area, a mouth area, and a cheek area, so that the cutout image is divided into 5 feature sub-regions in this embodiment.

Step S42: determining the color characteristic value of each characteristic subarea according to a preset formula;

it should be noted that the color values of each of the characteristic sub-regions are different, and the color characteristic values are calculated by using a mature calculation formula in this embodiment, which is a mature technique in the art and is not described again.

It should be noted that the preset formula for calculating the color feature value is as follows:

ΔR＝C_1，R-C_2，R

ΔG＝C_1，G-C_2，G

ΔB＝C_1，B-C_2，B

where C1, C2 represent color 1 and color 2, C1, R represents the R channel for color 1, C2, R represents the R channel for color 2. Similarly, C1, G denotes the G channel for color 1, and C2, G denotes the R channel for color 2. C1, B for color 1B channel, C2, B for color 2B channel. Δ C is the color feature value of the region where color 1 is located.

Step S43: matching the color characteristic value with the color of the building blocks according to the color characteristic value and the color value interval of the building blocks to be spliced;

in this embodiment, when the color characteristic value is within the color value range of the building block to be spliced, the color characteristic value is matched with the color value range of the building block to be spliced.

Step S44: and performing threshold segmentation on each characteristic subregion according to the color characteristic value to generate a face cordwood image.

Since the threshold segmentation maps the original multi-color information to several color intervals, the color information is inevitably lost. If the loss of color results in loss of facial features. This is unacceptable. It is therefore necessary to adjust the color values of the multi-thresholding described above. And judging whether the features of the five sense organs are reduced or disappear according to the color values after threshold segmentation. The judgment of disappearance is based on the fact that the color value of the block in the area is not available or the color of the area is modified by other colors. The reduction is determined by reducing the number of pixels of the corresponding color of the region compared to that before thresholding. If the percentage of reduction is greater than the empirical value for the characteristic sub-region (e.g., 50% reduction in the mouth, not recognized at all, 60% reduction in the nose, only the tip or nostril of the nose left, discernable), the threshold value is considered to be unreasonable. A threshold adjustment is required. The direction of adjustment is the reduced pixel value within the sub-region. And repeating the threshold adjusting process, and finally, if all the regions meet the characteristic empirical values of the sub-regions, determining that the adjustment is finished. If not, different priority adjustments are made in order of eyes, mouth, eyebrows, nose, cheeks. And when the eyebrow can be met, considering that the eyebrow is partially met, and finishing threshold adjustment. If all the parameters cannot be met, the face building block is not successful, and the user is prompted to regenerate the picture.

Referring to fig. 7, based on the above-mentioned face image processing method, the present invention further provides a face image processing apparatus, which includes

An obtaining module 100, configured to obtain a target image to be processed;

it is worth mentioning that the image may be uploaded locally from the user device or may be obtained by instant shooting. In this embodiment, the target image includes a face, so as to facilitate subsequent processing of the face.

The recognition module 200 is configured to recognize a position of a face in the target image;

A segmentation module 300, configured to segment the identified face according to the position to generate a matting image;

It is worth mentioning that, in order to meet diversified requirements of customers and improve user experience, the obtained cutout image can be grayed, and in the graying process, the RGB channel is converted according to the standard.

And the mapping module 400 is used for mapping the sectional image to the corresponding building block particle color so as to generate the face cordwood image.

Because the colors of the building block particles are limited, and the cutout image is colored, the cutout image needs to be mapped to the corresponding building block particle colors, and the cutout image is subjected to face building block by adopting a mapping method in the embodiment.

Preferably, the identification module 200 is further configured to:

Preferably, the segmentation module 300 is also used for

it should be noted that Connected Component generally refers to an image area (Region, Blob) composed of foreground pixels with the same pixel value and adjacent positions in the image

Preferably, the segmentation module 300 is also used for

Acquiring the maximum height and the maximum width of a communication area;

after the connected region is found, the height Hmax and the width Wmax are recorded

and obtaining the height and width of the facial features, and calculating the height and width of the compressed facial features according to the R value. For example, the height and width of the compressed eye are Heye and Weye, respectively.

Preferably, the segmentation module 300 is further configured to:

Preferably, the mapping module 400 is further configured to:

dividing the sectional image into a plurality of characteristic subregions;

in this embodiment, when the color characteristic value is within the color value interval of the building block to be spliced, the color characteristic value is matched with the color value interval of the building block to be spliced.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments. The use of the words first, second, third, etc. do not denote any order, but rather the words first, second, etc. are to be interpreted as names.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a user equipment, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent structures or equivalent processes performed by the present invention or directly or indirectly applied to other related technical fields are also included in the scope of the present invention.

Claims

1. A face image processing method is characterized by comprising the following steps:

acquiring a target image to be processed;

identifying the position of the face in the target image;

mapping the sectional image to the corresponding color of the building block particles to generate a face building block image;

the recognizing the position of the face in the target image comprises:

recognizing a face in a target image based on a preset convolutional neural network model, and acquiring the position of the face;

the segmenting the identified face according to the position and generating a matting image comprises:

segmenting the face based on the identified connected region to generate a sectional image;

the segmentation of the human face based on the identified connected region and the generation of the sectional image comprise:

acquiring the maximum height and the maximum width of a communication area;

determining a compression ratio based on the maximum height, the maximum width and the height and width of the building block bottom plate;

2. The method for processing a human face image according to claim 1, wherein the adjusting the maximum height and the maximum width of the connected region, and the using the adjusted connected region as a matting image comprises:

3. The method of claim 2, wherein the mapping the scratch image to a corresponding building block particle color to generate a face building block image comprises:

dividing the sectional image into a plurality of characteristic subregions;

and performing threshold segmentation on the matched characteristic sub-regions according to the color characteristic values to generate a face cordwood image.

4. The face image processing method according to claim 3, wherein the feature sub-regions include an eye feature sub-region, an eyebrow feature sub-region, a nose feature sub-region, a mouth feature sub-region, and a cheek feature sub-region.

5. A user equipment, the user equipment comprising: a memory, a processor and a face image processing program stored on the memory and executable on the processor, the face image processing program when executed by the processor implementing the steps of the face image processing method according to any one of claims 1 to 4.

6. A storage medium, characterized in that the storage medium has stored thereon a face image processing program, which when executed by a processor implements the steps of the face image processing method according to any one of claims 1 to 4.

7. A face image processing device is characterized by comprising

The acquisition module is used for acquiring a target image to be processed;

the mapping module is used for mapping the sectional image to the corresponding building block particle color so as to generate a face building block image;

the recognition module is further used for recognizing the face in the target image based on a preset convolutional neural network model and acquiring the position of the face;

the segmentation module is also used for identifying a communication area where the face is located according to the position where the face is located;

the segmentation module is further used for obtaining the maximum height and the maximum width of the communication area;

and when the compressed height is smaller than the height of the building block bottom plate particle pixel or/and the compressed width is smaller than the width of the building block bottom plate particle pixel, adjusting the maximum height and the maximum width of the communication area, and taking the adjusted communication area as a sectional image.