WO2021120626A1

WO2021120626A1 - Image processing method, terminal, and computer storage medium

Info

Publication number: WO2021120626A1
Application number: PCT/CN2020/104638
Authority: WO
Inventors: 冯少江; 吕乐
Original assignee: 上海传英信息技术有限公司
Priority date: 2019-12-16
Filing date: 2020-07-24
Publication date: 2021-06-24
Also published as: CN111161131A

Abstract

The present application relates to an image processing method, a terminal, and a computer storge medium. The image processing method is applied to the terminal and comprises: obtaining an image to be processed; performing feature recognition on a portrait of said image to obtain at least two portrait features of said image; and performing facial beautification processing on a corresponding region of the at least one portrait feature, and/or performing no facial beautification processing on the corresponding region of the at least one portrait feature. According to the image processing method, the terminal, and the computer storage medium provided by the present application, the terminal obtains at least two portrait features obtained by performing feature recognition on a portrait of an image to be processed, and performs facial beautification processing on a corresponding region of the at least one portrait feature and/or perform no facial beautification processing on the corresponding region of the at least one portrait feature, so as to implement personalized facial beautification processing of a user according to user features, thereby improving the use experience of the user.

Description

Image processing method, terminal and computer storage medium

This patent application requires the priority of the Chinese patent application number 201911296529.1 filed on December 16, 2019, the applicant is Shanghai Chuanying Information Technology Co., Ltd., and the invention title is "an image processing method, terminal and computer storage medium". The full application is incorporated into this application by reference.

Technical field

This application relates to the field of terminals, in particular to an image processing method, terminal and computer storage medium.

Background technique

With the rapid development of mobile communication technology and the rapid popularization of smart terminals, there are more and more scenarios where smart terminals such as smart phones are used to take pictures. To some extent, smart terminals have replaced cameras as the mainstream tool for ordinary users to take pictures. In order to meet users' demands for high-quality images, the beauty function has gradually become a standard feature of camera applications in smart terminals. In related technologies, the smart terminal usually adopts unified beauty parameters for processing different users in the beauty mode, or uses unified beauty parameters for the entire person.

technical problem

For different users or different features of the same user, the beauty effect required for a particular feature may be different from other features. If the beauty effect is not distinguished, the user experience may be affected.

Technical solutions

The purpose of this application is to provide an image processing method, terminal, and computer storage medium, which can improve the user experience by performing personalized beauty processing on the user according to the characteristics of the user.

This application first provides an image processing method, which is applied to a terminal, and the method includes:

Obtain the image to be processed;

Performing feature recognition on the portrait in the image to be processed, and acquiring at least two portrait features in the image to be processed;

Performing beautification processing on the region corresponding to the at least one portrait feature and/or not performing beauty processing on the region corresponding to the at least one portrait feature.

Further, the characteristics include at least one of the following: gender, face shape, eyes, race, and age.

Further, the performing beautification processing on the region corresponding to the at least one portrait feature includes:

When performing beauty processing on the image to be processed, performing beauty reduction processing on the region corresponding to the at least one portrait feature, and/or adopting a beauty solution corresponding to the at least one portrait feature on the at least one portrait Perform beautification processing on the feature corresponding area.

Further, said adopting a beauty solution corresponding to the at least one portrait feature to perform beauty processing on the area corresponding to the at least one portrait feature includes:

Acquiring and outputting at least one beauty solution corresponding to the at least one portrait feature;

A selection of the at least one beauty solution is received, and beauty processing is performed on the region corresponding to the at least one portrait feature according to the selected beauty solution.

Further, the performing beauty processing on the region corresponding to the at least one portrait feature and/or not performing beauty processing on the region corresponding to the at least one portrait feature includes at least one of the following processes:

If the face shape is a round face, performing face-lifting and beautifying processing on the face area of the portrait in the image to be processed;

If the eyes are single eyelids, perform beauty treatment on the eye area of the portrait in the image to be processed;

If the age is an elderly person, wrinkle-preserving processing is performed on the facial area of the portrait in the image to be processed;

If the age is a young person, perform lip color rejuvenation processing on the lip region of the portrait in the image to be processed;

If the gender is male, then no beautification processing is performed on the beard area of the portrait in the image to be processed, or the corresponding beautification processing is performed;

If the gender is female, no beautification processing is performed on the brow-center mole area and/or the nose decoration area of the portrait in the image to be processed, or the corresponding beautification processing is performed.

Further, the performing beauty processing on the region corresponding to the at least one portrait feature and/or not performing beauty processing on the region corresponding to the at least one portrait feature includes:

Displaying the area corresponding to the at least one portrait feature;

Receiving a user's selection operation, and performing beautification processing and/or not performing beautification processing on the region corresponding to the portrait feature selected by the selection operation.

Further, the performing beauty reduction processing on the region corresponding to the at least one portrait feature includes:

The corresponding beauty level is gradually reduced when performing beauty processing from the edge to the center of the region corresponding to the at least one portrait feature;

Or, not performing beautification processing on the area corresponding to the at least one portrait feature; or,

The state of the region corresponding to the at least one portrait feature in the image to be processed after the beautification processing is restored to the state before the beautification processing.

Further, the performing feature recognition on the portrait in the to-be-processed image and acquiring at least two portrait features in the to-be-processed image includes:

Input the to-be-processed image into the trained portrait feature recognition neural network model to obtain at least two portrait features in the to-be-processed image output by the trained portrait feature recognition neural network model; wherein, the portrait feature recognition The neural network model is obtained by training based on historical images and corresponding historical portrait features.

This application also provides a terminal. The terminal includes a processor and a memory for storing a program; when the program is executed by the processor, the processor realizes the image processing method described above.

The present application also provides a computer storage medium that stores a computer program, and when the computer program is executed by a processor, the image processing method described above is implemented.

Beneficial effect

The image processing method, terminal, and computer storage medium of the present application acquire at least two portrait features obtained by feature recognition of a portrait in an image to be processed, and perform beauty processing and/or on an area corresponding to the at least one portrait feature No beautification processing is performed on the area corresponding to the at least one portrait feature, so as to implement personalized beautification processing for the user according to the user characteristics, thereby improving the user experience.

Further, the image processing method of the present application detects whether the image to be processed contains a preset feature identifier, and determines at least two portrait features in the image to be processed according to the detection result, which can realize rapid and accurate processing of the image to be processed. Character recognition is performed on the portrait in the image.

Furthermore, the image processing method provided by the present application performs beautification processing on the region corresponding to the portrait feature through different protection strategies, so as to realize the protection of the region corresponding to the portrait feature.

In addition, the image processing method provided by the present application displays at least one portrait feature corresponding region, so that the user can select one or several portrait feature corresponding regions to perform beauty processing, which is flexible in operation and further improves the user experience.

In addition, the image processing method provided by this application is flexible and convenient by providing users with a variety of beauty solutions to choose from, and further improves the user experience.

Description of the drawings

FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of this application;

FIG. 2 is a schematic structural diagram of a terminal provided by an embodiment of the application;

FIG. 3 is a schematic diagram of a specific flow of an image processing method provided by an embodiment of the application;

4 is a schematic diagram of a process of performing gender recognition according to face region data in an embodiment of the application;

FIG. 5 is a schematic diagram of the male beard area after beautification protection is performed in an embodiment of the application; FIG.

Fig. 6 is a schematic diagram of a woman's eyebrow nevus and nasal area after beauty protection is performed in an embodiment of the application.

The best implementation of this application

The technical solution of the present application will be further elaborated below in conjunction with the drawings and specific embodiments of the specification. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of this application. The terms used in the specification of the application herein are only for the purpose of describing specific embodiments, and are not intended to limit the application. The term "and/or" as used herein includes any and all combinations of one or more related listed items.

Referring to FIG. 1, an image processing method provided by an embodiment of this application is applicable to the situation of performing beautification processing on an image. The image processing method can be performed by an image processing apparatus provided by an embodiment of the application. In execution, the image processing device may be implemented in software and/or hardware. In specific applications, the image processing device may be a terminal such as a smart phone, a personal digital assistant, or a tablet computer. In this embodiment, taking the execution subject of the image processing method as the terminal as an example, the image processing method includes the following steps:

Step S101: the terminal obtains an image to be processed;

Here, the image to be processed is a single frame image, and the image to be processed may be a preview image collected by the terminal through a camera device such as a camera. Taking the terminal as a mobile phone as an example, the mobile phone uses the preview image collected by the camera as the image to be processed after the camera application is turned on and switched to the beauty mode. At this time, the preview image is displayed on the shooting preview interface of the camera application of the mobile phone. on. That is, after receiving the camera start instruction, the terminal switches to the beauty mode, and collects the preview image taken by the camera as the image to be processed. If the mobile phone shoots a person, an image including the person is displayed on the shooting preview interface of the camera application of the mobile phone.

Step S102: The terminal performs feature recognition on the portrait in the image to be processed, and acquires at least two portrait features in the image to be processed;

Specifically, the terminal performs feature recognition on the portrait in the image to be processed obtained in step S101, so as to correspondingly acquire at least two portrait features in the image to be processed.

In this embodiment, the characteristics include at least one of the following: gender, face shape, eyes, race, and age. Here, the race can refer to the classification based on skin color, specifically it can be yellow race, white race, black race and brown race, or can refer to the classification based on region, specifically it can be Asian, European People, Africans, Americans, etc., the age can refer to the type of age, such as young people, old people, etc. Here, the terminal may first obtain the face image in the image to be processed by performing face detection on the image to be processed, and then implement the face image in the image to be processed based on the face image. Feature recognition. It should be noted that, when the terminal performs feature recognition on the portrait in the image to be processed, the recognition may be performed by adopting a feature recognition model established based on an artificial intelligence algorithm, or by detecting the identification features of the portrait. In addition, for gender recognition, the terminal may collect the voice of the photographer, and obtain the gender recognition result of the voice of the photographer, so as to correspondingly realize the gender recognition of the portrait in the image to be processed. In practical applications, the terminal may also determine the portrait feature in the image to be processed according to the features input by the user.

Step S103: The terminal performs beauty processing on the region corresponding to the at least one portrait feature and/or does not perform beauty processing on the region corresponding to the at least one portrait feature.

Here, according to the difference of the portrait feature, the way of performing beautification processing on the image to be processed may be different, and correspondingly, the corresponding areas of the portrait feature that need to be beautified and not be beautified may be different. For different genders, if the gender is male, the portrait feature corresponding area may include a beard area; and/or, if the gender is female, the portrait feature corresponding area may include an eyebrow mole area and/or nose decoration area. For example, for a round face, the face area of a portrait may be used as the corresponding area of the face; for a single eyelid, the eye area of the portrait may be used as the corresponding area of the eye. In an embodiment, the performing beauty processing on the region corresponding to the at least one portrait feature may be performing beauty reduction processing on the region corresponding to the at least one portrait feature when performing beauty processing on the image to be processed , It is also possible to use a beautifying solution corresponding to the at least one portrait feature to perform beautification processing on the area corresponding to the at least one portrait feature. Understandably, if the beauty reduction process is performed on the region corresponding to the at least one portrait feature, after the beauty process of the to-be-processed image is completed, the effect of the region corresponding to the at least one portrait feature will be inconsistent with the previous effect. However, the effect change degree of the region corresponding to the at least one portrait feature will still be weaker than the effect change degree of other regions; if the beauty solution corresponding to the portrait feature is used to perform beautification processing on the personality feature region, the After the beauty processing of the image to be processed is completed, the effect of the region corresponding to the at least one portrait feature may better match the user's purpose. It should be noted that the performing beauty reduction processing on the region corresponding to the at least one portrait feature may include: gradually reducing the corresponding beauty level when performing beauty processing from the edge to the center of the region corresponding to the at least one portrait feature Or, not performing beauty processing on the region corresponding to the at least one portrait feature; or, restoring the state of the region corresponding to the at least one portrait feature in the image to be processed after the beauty processing to the state before the beauty processing . Taking the beautification treatment including the dermabrasion treatment as an example, when the dermabrasion treatment is performed on the male beard area, the level of the dermabrasion treatment can be reduced. The restoration of the state of the region corresponding to the at least one portrait feature in the image to be processed after the beautification processing to the state before the beautification processing can be understood as after the beautification processing is performed on the image to be processed, Restore the state of the region corresponding to the portrait feature in the image to be processed after the beautification processing to the state before the beauty process, so as not to perform beautification on the region corresponding to the portrait feature in the image to be processed. Color processing, so as to achieve the protection of local details. Here, if beauty processing is not performed on the region corresponding to the portrait feature, after the beauty processing of the image to be processed is completed, the effect of the region corresponding to the portrait feature will still remain consistent with the previous effect. In this way, by performing beautification processing on the area corresponding to the portrait feature through different protection strategies, the protection of the area corresponding to the portrait feature is realized.

To sum up, in the image processing method provided in the foregoing embodiment, the terminal acquires at least two portrait features obtained by feature recognition of a portrait in the image to be processed, and performs beauty processing and/or on the region corresponding to the at least one portrait feature No beautification processing is performed on the area corresponding to the at least one portrait feature, so as to implement personalized beautification processing for the user according to the user characteristics, thereby improving the user experience.

Implementation of this application

Based on the foregoing embodiment, in one embodiment, the performing feature recognition on the portrait in the to-be-processed image and acquiring at least two portrait features in the to-be-processed image includes: inputting the to-be-processed image after training The feature recognition neural network model for obtaining at least two portrait features in the to-be-processed image output from the trained feature recognition neural network model; wherein the feature recognition neural network model is based on historical images and corresponding historical portraits Features obtained through training. Understandably, the terminal may pre-store a feature recognition neural network model obtained by training historical images and corresponding historical portrait features using a neural network algorithm. When the image to be processed is used as the input of the feature recognition neural network model, The output of the feature recognition neural network model is the corresponding facial feature recognition result. It should be noted that the input of the image to be processed into the trained feature recognition neural network model may be the input of the face data in the image to be processed into the trained feature recognition neural network model. The establishment and training process of the feature recognition neural network model can refer to the prior art, and will not be repeated here.

In an embodiment, the terminal performing feature recognition on the portrait in the image to be processed, and acquiring at least two portrait features in the image to be processed, includes:

Detecting whether the portrait in the image to be processed includes a preset feature identifier, and obtaining a corresponding detection result;

Acquire at least two portrait features in the image to be processed according to the detection result.

Specifically, the terminal detects whether the portrait in the image to be processed includes a preset feature identifier, and obtains a corresponding detection result, so as to obtain at least two portrait features in the image to be processed according to the detection result. Here, the preset feature identification refers to an identification that can calibrate a certain feature of the user. For example, taking the feature as gender as an example, for males, the gender identification can be beard, throat, etc.; for women, The gender identification can be eyebrow moles, facial accessories such as nose decorations and veils, etc. Therefore, when it is detected that the image to be processed contains male identification such as beard or throat, the gender corresponding to the portrait in the image to be processed can be determined It is male; and when it is detected that the image to be processed contains female identification such as eyebrow mole or nose decoration, it can be determined that the gender corresponding to the portrait in the image to be processed is female. Taking the feature as age as an example, for the elderly and young people, the age can be distinguished by detecting whether there are wrinkles on the face, etc. In this way, by detecting whether the image to be processed contains a preset feature identifier, and determining at least two portrait features in the image to be processed according to the detection result, it is possible to quickly and accurately identify the features of the person in the image to be processed .

In an embodiment, the terminal performs feature recognition on the portrait in the image to be processed, and before acquiring at least two portrait features in the image to be processed, the method may further include: detecting the number of portraits in the image to be processed Whether the preset quantity condition is satisfied; if it is satisfied, the step of performing feature recognition on the portrait in the image to be processed and acquiring at least two portrait features in the image to be processed is performed. Here, the preset quantity condition may be one or more portraits. When the number of portraits in the image to be processed is two, each portrait may contain different features, so that different beautification processing solutions need to be adopted for different portraits. For example, the image to be processed may contain different male and female faces at the same time, and different portrait features are acquired for faces of different genders, and a corresponding beauty processing method is adopted for each region corresponding to the portrait feature.

It should be noted that, for different portrait features, the terminal may pre-store the corresponding relationship between the different portrait features and the corresponding beauty solutions. For example, if the eyes are single eyelids, the corresponding beauty solutions may be Dan Fengyan Beauty treatment; if the race is Asian, the corresponding beauty plan can be yellow skin beauty treatment, etc. In a specific embodiment, the performing beauty processing on the region corresponding to the at least one portrait feature and/or not performing beauty processing on the region corresponding to the at least one portrait feature includes at least one of the following processing: if the face shape is a round face , Then perform face-lifting and beautifying processing on the face area of the portrait in the image to be processed; if the eyes are single eyelids, perform the beautifying processing on the eye area of the portrait in the image to be processed; if the age is the elderly, Wrinkle-preserving processing is performed on the facial area of the portrait in the image to be processed; if the age is young, the lip color rejuvenation processing is performed on the lip area of the portrait in the image to be processed; if the gender is male, the lip color is not processed. The beard area of the portrait in the image to be processed is subjected to beautification processing, or corresponding beautification processing; if the gender is female, no beautification processing is performed on the eyebrow area and/or the nose area of the portrait in the image to be processed , Or corresponding beauty treatment.

The following takes the feature as gender as an example to describe in detail the process of performing beauty processing according to the feature recognition result. The beauty processing is performed on the region corresponding to the at least one portrait feature and/or the region corresponding to the at least one portrait feature is different. Performing beauty processing may be performing feature region recognition on the face in the image to be processed, and obtaining at least one feature region of the face in the face; wherein, the feature region of the face includes at least one feature region of five sense organs Determine the gender corresponding area in the face according to the gender and the at least one facial feature area; protect the gender corresponding area when performing beauty processing on the image to be processed. Here, the terminal may use the existing facial feature region recognition method to perform feature region recognition on the face in the image to be processed, so as to obtain at least one facial feature region in the face, and the face The feature area includes at least one feature area of the five sense organs. It should be noted that the facial feature regions may include facial feature regions such as eyes, mouth, nose, ears, and eyebrows, as well as non facial feature regions such as chin and forehead. Since the position of the region corresponding to the portrait feature is fixed and can be determined correspondingly by the region of the facial feature, it can be determined that the portrait feature in the face corresponds to the result of the recognition of the portrait feature and the at least one facial feature region. area. Taking the gender as male and the facial feature corresponding area as the beard area as an example, the terminal can pass through the mouth after determining the characteristic areas of the nose and mouth in the face, that is, the positions of the nose and mouth in the face. The lower area and the area between the upper part of the mouth and the lower part of the nose obtain the corresponding beard area. Understandably, under normal circumstances, the characteristics of beards are dark, such as black, which is lower in brightness than the skin area. Correspondingly, the brightness of pixels in the area under the mouth and between the area above the mouth and under the nose can be smaller than The pixels with the average value of the pixels in the face area and the difference between the two are greater than the preset first threshold are used as pixels representing the beard, and the beard area is obtained.

In an embodiment, the terminal performing beauty processing on the area corresponding to the at least one portrait feature and/or not performing beauty processing on the area corresponding to the at least one portrait feature includes:

Displaying the area corresponding to the at least one portrait feature;

Understandably, the terminal may determine multiple candidate portrait feature corresponding regions according to the feature recognition result, but the user may not need to use each candidate portrait feature corresponding region as the target portrait feature corresponding region for beautification. Processing, but only one or a few candidate portrait feature corresponding regions need to be used as the target portrait feature region for beauty processing. Therefore, after determining at least one portrait feature corresponding region, the terminal can display the at least one portrait feature corresponding region. For a feature corresponding area, the user selects one or several portrait feature corresponding areas from it as the target portrait feature corresponding area for beautification processing. It should be noted that after the terminal displays the at least one portrait feature corresponding area, it can simultaneously control the at least one portrait feature corresponding area to be in a selectable state, such as protruding and displaying each of the portrait feature corresponding areas Borders and so on. In this way, the terminal displays at least one portrait feature corresponding region, and the user selects a certain one or several portrait feature corresponding regions from the region for beautification processing, which is flexible in operation and further improves the user experience.

In one embodiment, said adopting the beauty solution corresponding to the at least one portrait feature to perform beauty processing on the region corresponding to the at least one portrait feature includes:

Understandably, for any portrait feature, the terminal may store at least one beauty solution. Accordingly, when the terminal needs to perform beauty processing on an area corresponding to a certain portrait feature, it can output the At least one beauty solution corresponding to the at least one portrait feature, and the user selects one or more beauty solutions therefrom to perform beauty processing on the region corresponding to the at least one portrait feature. For example, taking the feature as the face shape and the face shape as a round face as an example, the terminal may recommend various beauty solutions such as face-lifting and beautifying treatment, freckles beautifying treatment, whitening and beautifying treatment to the user, and the user can perform according to needs Choose the corresponding beauty plan. In this way, by providing users with a variety of beauty solutions to choose from, they are flexible and convenient, and the user experience is further improved.

In one embodiment, after the performing beautification processing on the area corresponding to the at least one portrait feature and/or not performing the beautification processing on the area corresponding to the at least one portrait feature, the method further includes:

Receiving user input on the to-be-processed image after beautification processing;

In response to the input, determining a target area selected on the to-be-processed image after the beautification processing;

The state of the target area in the image to be processed after the beautification processing is restored to the state before the beautification processing.

Understandably, in order to facilitate the user to flexibly select the area that needs to be protected, the terminal may receive the user's input on the to-be-processed image after the beautification process, so as to determine that the user is on the beautification-processed image to be processed. The selected target area further restores the state of the target area in the to-be-processed image after the beautification processing to the state before the beautification processing. Here, the terminal may display each area in the portrait after performing beautification processing on the image to be processed, for example, using a frame or highlighting to mark each area to facilitate the user's selection. In addition, the terminal may also determine the area selected by the sliding operation trajectory as the target area according to the sliding operation trajectory input by the user. In this way, the terminal determines the target area that does not require beautification processing according to the user's selection, so as to protect the target area, improve the flexibility of use, and further improve the user experience.

Based on the same inventive concept as the foregoing embodiment, an embodiment of the present application provides a terminal. As shown in FIG. 2, the terminal includes: a processor 110 and a memory 111 for storing a computer program that can run on the processor 110; Wherein, the processor 110 illustrated in FIG. 2 is not used to refer to the number of processors 110 as one, but only used to refer to the positional relationship of the processor 110 with respect to other devices. In practical applications, the processor 110 The number can be one or more; similarly, the memory 111 illustrated in FIG. 2 has the same meaning, that is, it is only used to refer to the positional relationship of the memory 111 with respect to other devices. In practical applications, the number of the memory 111 can be one or more. The processor 110 is configured to implement the image processing method applied to the foregoing terminal when running the computer program.

The terminal may further include: at least one network interface 112. The various components in the terminal are coupled together through the bus system 113. It can be understood that the bus system 113 is used to implement connection and communication between these components. In addition to the data bus, the bus system 113 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear description, various buses are marked as the bus system 113 in FIG. 2.

Among them, the memory 111 may be a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM, Read Only Memory), a programmable read-only memory (PROM, Programmable Read-Only Memory), an erasable programmable read-only memory (EPROM, Erasable Programmable Read-Only Memory, Electrically Erasable Programmable Read-Only Memory (EEPROM, Electrically Erasable Programmable Read-Only Memory), Magnetic Random Access Memory (FRAM, ferromagnetic random access memory), flash memory (Flash Memory), magnetic surface memory, CD-ROM, or CD-ROM, Compact Disc Read-Only Memory); Magnetic surface storage can be disk storage or tape storage. The volatile memory may be a random access memory (RAM, Random Access Memory), which is used as an external cache. By way of exemplary but not restrictive description, many forms of RAM are available, such as static random access memory (SRAM, Static Random Access Memory), Synchronous Static Random Access Memory (SSRAM, Synchronous Static Random Access Memory), Dynamic Random Access Memory (DRAM, Dynamic Random Access Memory), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM, Double Data Rate Synchronous Dynamic Random Access Memory, Enhanced Synchronous Dynamic Random Access Memory (ESDRAM, Enhanced Synchronous Dynamic Random Access Memory), synchronous connection dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), direct memory bus random access memory (DRRAM, Direct Rambus Random Access Memory). The memory 111 described in the embodiment of the present application is intended to include, but is not limited to, these and any other suitable types of memory.

The memory 111 in the embodiment of the present application is used to store various types of data to support the operation of the terminal. Examples of these data include: any computer programs used to operate on the terminal, such as operating systems and applications; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system contains various system programs, such as a framework layer, a core library layer, and a driver layer, which are used to implement various basic services and process hardware-based tasks. Applications can include various applications, such as media players (Media Player), browser (Browser), etc., used to implement various application services. Here, the program that implements the method of the embodiment of the present application may be included in the application program.

Based on the same inventive concept as the foregoing embodiment, this embodiment also provides a computer storage medium in which a computer program is stored. The computer storage medium may be a magnetic random access memory (FRAM, ferromagnetic random access memory) , Read Only Memory (ROM, Read Only Memory), Programmable Read Only Memory (PROM, Programmable Read-Only Memory), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM, Electrically Erasable Programmable Read-Only Memory), flash memory (Flash Memory), magnetic surface memory, optical disk, or CD-ROM (Compact Disc Read-Only Memory); it can also be a variety of devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, Tablet devices, personal digital assistants, etc. When the computer program stored in the computer storage medium is executed by the processor, the image processing method applied to the above terminal is implemented. Please refer to the description of the embodiment shown in FIG. 1 for the specific step flow implemented when the computer program is executed by the processor, which will not be repeated here.

Based on the same inventive concept as the foregoing embodiments, this embodiment uses specific examples to illustrate the technical solutions of the foregoing embodiments in detail. In this embodiment, the feature is gender as an example. FIG. 3 is a schematic diagram of a specific flow of an image processing method provided by an embodiment of the application, including the following steps:

Step S201: Switch to the beauty mode;

Specifically, after the user starts the camera application on the terminal and chooses to switch the photographing mode of the camera application to the beauty mode, the terminal correspondingly switches the photographing mode of the camera application to the beauty mode.

Step S202: Read the data of the preview frame image;

Specifically, the terminal reads the data of the preview frame image displayed on the preview interface of the camera application.

Step S203: face detection;

Specifically, the terminal performs face detection on the preview frame image to obtain the number of faces.

Step S204: Judge whether the number of faces is 1, if yes, go to step S205, otherwise go to step S208;

Specifically, the terminal judges whether the number of faces in the preview frame image is 1, and if so, execute step S205, otherwise, execute step S208.

Step S205: intercept the face area data;

Specifically, when the terminal detects that the number of faces in the preview frame image is 1, it intercepts the face area and converts it into face data in a specified format, and then extracts the feature points of the face, such as eyes, nose, and mouth. , Chin, ears and other main features.

Step S206: the gender recognition module performs gender recognition according to the face area data;

Here, a gender recognition module may be provided in the terminal, and the terminal inputs the face area data obtained in step S205 to the gender recognition module, and the gender recognition module can be regarded as a neural network model. Here, the process of performing gender recognition based on face region data in this embodiment can be seen in FIG. 4, which includes the following steps:

Step S301: Input the formatted face image data;

Specifically, input the formatted face image data into the input layer of the neural network model.

Step S302: convolution calculation;

Specifically, a convolution calculation is performed on a convolution layer of the neural network model. In this embodiment, the neural network model contains 4 convolutional layers as an example, and only one convolutional layer is convolutional calculation at a time. That is to say, the first convolutional layer of the neural network model performs the convolution calculation on the formatted person. The face image data is subjected to convolution calculation.

Step S303: batch standardization;

Specifically, batch normalization processing is performed on the features obtained by convolution in step S302, so that the obtained feature data distribution conforms to the normal distribution, thereby accelerating the convergence speed of the model and increasing the generalization ability of the model.

Step S304: feature activation;

Step S305: Maximum pooling;

Here, after activation and pooling, the obtained new feature map is used as the input data of the next convolutional layer.

Step S306: global average pooling;

Here, after 4 layers of convolution and pooling processing, global average pooling processing is performed on the obtained feature maps to replace the fully connected operation, so as to achieve the purpose of reducing model parameters.

Step S307: Output the probability array of gender classification.

Here, the probability array of gender classification is obtained after global average pooling, where the index with a larger array value is the final gender result of the gender recognition module, for example, the array index of male is 0, and the array index of female is 1.

Step S207: The gender recognition module delivers the gender recognition result and facial feature points to the beauty module;

Specifically, the gender recognition module sends the gender type and facial feature point information of the face obtained through neural network algorithm calculation to the beauty module. Here, a beautification module may be provided in the terminal to perform beautification processing on the image.

Step S208: the beautification module adapts the corresponding beautification parameters;

Here, when the number of detected human faces is 0 or greater than 1, the beautification module selects a set of default beautification parameters to perform beautification processing on the human face. When the number of detected human faces is 1, the beautification module divides the protection area of the corresponding gender according to the gender information obtained in step S207 and combined with the facial feature point information. When the beautification module processes the corresponding protection area, Select the corresponding beauty parameters to protect the protected area.

Step S209: the beautification module processes the image according to the adapted corresponding beautification parameters.

Here, when the number of detected faces is 0 or greater than 1, the beautification module performs beautification processing on the face according to the selected default beautification parameters. When the number of detected faces is 1, when the beauty module processes the corresponding protection area, it adopts a gradual processing scheme to smooth the edges of the protection area and reduce the processing of other parts of the protection area to achieve partial detail protection Effect. As shown in Figure 5, it is a schematic diagram after beautification protection is performed on the beard area of men (that is, the area will reduce the beautification of the skin). As shown in Figure 6, it is the eyebrow mole and nose decoration area of women. Schematic diagram after beauty protection (that is, the reduction of beautification on the brow area and the nose decoration area) to achieve the preservation of these local decorative details, and to perform beautification treatment on other areas of the face. Here, the screen of the terminal can display the image obtained by the beautification processing in real time, so that the user can see the effect of the beautification processing in real time.

To sum up, in the image processing method provided in the above embodiments, the Tensorflow framework based on deep learning, the gender recognition algorithm model trained through a large number of face image samples, and through TensorFlow The toco tool converts the server-side model to the Tflite model, and then transplants it to the Android terminal to realize offline real-time gender recognition on the mobile terminal. Load the gender recognition model in the camera on the Android side and call the execution entry of the model through the TensorFlow java side API, so as to intercept the detected face image, convert the format and pass it to the model, so as to perform algorithmic processing on each frame of image and output Gender recognition results, and finally adapt the beauty function of the corresponding gender to achieve different beauty effects for different genders. For example, beard protection will be added to the beauty of men, and eyebrow moles and nose decorations will be added to the beauty of women, thereby achieving both men and women. Differentiated smart beauty effect. It can be summarized that real-time gender recognition can be realized offline on the mobile terminal, and the beauty parameters of the corresponding gender can be adapted to beautify the face.

The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the various technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, All should be considered as the scope of this specification.

In this article, the terms "include", "include" or any other variants thereof are intended to encompass non-exclusive inclusion, in addition to including those elements listed, but also other elements not explicitly listed.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application. Should be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Industrial applicability

Claims

An image processing method applied to a terminal, wherein the method includes:

Obtain the image to be processed;

Performing feature recognition on the portrait in the image to be processed, and acquiring at least two portrait features in the image to be processed;

Performing beautification processing on the region corresponding to the at least one portrait feature and/or not performing beauty processing on the region corresponding to the at least one portrait feature.
The image processing method according to claim 1, wherein the characteristics include at least one of the following: gender, face shape, eyes, race, and age.
The image processing method according to claim 1, wherein said performing beautification processing on the region corresponding to the at least one portrait feature comprises:

When performing beauty processing on the image to be processed, performing beauty reduction processing on the region corresponding to the at least one portrait feature and/or adopting a beauty solution corresponding to the at least one portrait feature to perform the beauty treatment on the at least one portrait feature Perform beautification treatment on the corresponding area.
4. The image processing method according to claim 3, wherein said adopting a beautification solution corresponding to said at least one portrait feature to perform beautification processing on the area corresponding to said at least one portrait feature comprises:

Acquiring and outputting at least one beauty solution corresponding to the at least one portrait feature;

A selection of the at least one beauty solution is received, and beauty processing is performed on the region corresponding to the at least one portrait feature according to the selected beauty solution.
The image processing method according to claim 2, wherein said performing beautification processing on the region corresponding to the at least one portrait feature and/or not performing beauty processing on the region corresponding to the at least one portrait feature comprises at least one of the following processes: Species:

If the face shape is a round face, performing face-lifting and beautifying processing on the face area of the portrait in the image to be processed;

If the eyes are single eyelids, perform beauty treatment on the eye area of the portrait in the image to be processed;

If the age is an elderly person, wrinkle-preserving processing is performed on the facial area of the portrait in the image to be processed;

If the age is a young person, perform lip color rejuvenation processing on the lip region of the portrait in the image to be processed;

If the gender is male, then no beautification processing is performed on the beard area of the portrait in the image to be processed, or the corresponding beautification processing is performed;

If the gender is female, no beautification processing is performed on the brow-center mole area and/or the nose decoration area of the portrait in the image to be processed, or the corresponding beautification processing is performed.
The image processing method according to claim 1, wherein the performing beautification processing on the area corresponding to the at least one portrait feature and/or not performing the beautification processing on the area corresponding to the at least one portrait feature comprises:

Displaying the area corresponding to the at least one portrait feature;

Receiving a user's selection operation, and performing beautification processing and/or not performing beautification processing on the region corresponding to the portrait feature selected by the selection operation.
The image processing method according to claim 3, wherein said performing beauty reduction processing on the region corresponding to the at least one portrait feature comprises:

The corresponding beauty level is gradually reduced when performing beauty processing from the edge to the center of the region corresponding to the at least one portrait feature;

Or, not performing beautification processing on the area corresponding to the at least one portrait feature; or,

The state of the region corresponding to the at least one portrait feature in the image to be processed after the beautification processing is restored to the state before the beautification processing.
The image processing method according to claim 1, wherein said performing feature recognition on the portrait in the image to be processed and acquiring at least two portrait features in the image to be processed comprises:

Input the to-be-processed image into the trained feature recognition neural network model to obtain at least two portrait features in the to-be-processed image output by the trained feature recognition neural network model; wherein, the feature recognition neural network model It is obtained by training based on historical images and corresponding historical portrait features.
A terminal, comprising: a processor and a memory for storing a computer program that can be run on the processor, and when the processor runs the computer program, the image processing method according to claim 1 is implemented.
A computer storage medium, wherein a computer program is stored, and when the computer program is executed by a processor, the image processing method according to claim 1 is implemented.