WO2019000777A1

WO2019000777A1 - Internet-based face beautification system

Info

Publication number: WO2019000777A1
Application number: PCT/CN2017/109834
Authority: WO
Inventors: 甘俊英; 姜开永; 谭海英
Original assignee: 五邑大学
Priority date: 2017-06-27
Filing date: 2017-11-08
Publication date: 2019-01-03
Also published as: JP6773883B2; KR20190018483A; CN107392110A; JP2019523489A; KR102195922B1

Abstract

An Internet-based face beautification system, comprising an image and video acquisition unit, a face detection unit and a beautification processing unit; provided are an Internet-based platform and a face beautification system using "deep learning" technology, combining facial attractiveness value estimation and automatic beautification into one, realizing beautification of a face image and avoiding damaging the image. The present invention avoids the trouble of a user manually performing a complex image beautification operation, and simultaneously provides exchange between users, a view-sharing community and a personal space for showing oneself, satisfying multilevel user needs.

Description

Internet-based facial beautification system

Technical field

The invention relates to the field of internet, machine learning and biometric identification systems, and in particular to an internet-based facial beautification system.

Background technique

At present, with the development of smart phones and Internet technologies, social media has attracted more and more people's attention. Uploading personal life photos and travel photos on social media has become an indispensable part of people's lives, but for imaging original images. The dissatisfaction of quality or the “unconfidence” of personal appearance often requires retouching before uploading images to social media to improve the visual effect of the image. First of all, people don't know enough about their own personal beauty, lack of quantitative analysis and comparison. Secondly, although you can use software tools to beautify the image to improve the viewing effect, but usually master the work of a retouching software and achieve perfect retouching. It is very large, it takes a lot of time and effort, and after the image is beautified, there are often problems such as unsatisfactory local detail processing and overall display distortion. Face image beautification and face value prediction in existing solutions are separate and implemented in different software. Moreover, the existing face value scoring software only scores and predicts the face image, and does not give a basis for judging the score of the specific face and the pre-judgment score and providing a beautification plan for improving the visual effect, and using the beautification software to repair At the time of the picture, it is still necessary to rely on the individual subjective knowledge and constantly try to adjust the beauty of the face. How to combine face color prediction and face landscaping, according to the level of pre-judgment scores, the system gives targeted landscaping scheme and automatic face beautification, and avoiding the damage caused by image quality becomes an urgent problem to be solved. .

Summary of the invention

In order to solve the above, an object of the present invention is to provide a face landscaping system based on an Internet platform, which combines face color prediction and automatic landscaping to achieve beautification of face images and avoid damage to images.

The technical solution adopted by the present invention to solve the problem is:

Internet-based facial beautification system, including:

Image and video acquisition unit for acquiring images and videos in the client, processing and uploading to the server;

a face detection unit, configured to detect information uploaded by the image and video acquisition unit, and feed the result back to the client or the beautification processing unit;

The beautification processing unit is configured to process the face detection unit to detect the face information, and feed the result to the face detection unit or the client.

Further, the original image captured by the image and video acquisition unit is stored in a RAW high-definition format, and an image processing algorithm is called to reduce the resolution, sharpen and enhance the contrast, and obtain a low-definition image. After the software processing is completed, the two The image is uploaded to the server via the Internet. After the server receives the image, the low-definition image is used for face detection, face verification and face key point detection, and the high-definition image is used for face beautification.

Further, the image and video acquisition unit is configured to store image resources, including local storage and server-side storage. The local storage is used to store the image on the storage medium of the client, and the server-side storage stores the image resource in the public. On the cloud storage platform and the server storage platform, the storage capacity of the network cloud storage cloud platform is limited to 1000M, and the storage resource of the server storage platform is limited to 200M.

Further, the face detection unit includes remote remote face image detection, and the face detection algorithm is deployed on a remote server, and the API interface reserved by the algorithm is called on the local client to implement processing on the remote server, including the client. Image uploading, cloud face detection, and server image downloading, the client image uploading is used by the client to upload an image to the server, and the face detecting unit is configured to invoke a face detection algorithm pair deployed on the server. The received image is subjected to face detection processing, and the server image is transmitted to the server to feed back the processed image and face coordinates back to the client.

Further, the face detection unit includes cloud face key point detection, and is used for remotely detecting 68 key feature points representing the face shape of the face, so as to determine the facial facial area and perform feature extraction. The algorithm is low definition. The degree image is taken, and the extracted key point coordinates are fed back to the client and transmitted to the face beautification algorithm.

Further, the face detection unit includes a cloud face verification function for determining whether the original face image and the image beautified by the face beautification unit are the same person, and if the face beautification parameter is adjusted too large, the similarity is exceeded. If it is lower and lower than the system default similarity threshold of 0.4, it will be re-beautified.

Further, the beautification processing unit adopts a face-preferred prediction algorithm based on “deep learning” to predict the degree of beauty of the face image, and obtains a pre-judgment score representing the beauty level of the face, and the beautiful pre-judgment score adopts a percentage system, from 1 to 100. Divided into 100 points, the rating of 1 means the ugliest, the rating of 100 means the most beautiful, the higher the score, the higher the degree of beauty, and vice versa.

Further, the beautification processing unit includes a facial expression of the facial features, and the facial expression of the facial features is based on the prediction of the facial expression of the face, and the facial features and the face are extracted according to the facial features by the facial features. Key feature points, extract facial features, and predict the degree of beauty. The beautiful pre-judgment is based on the percentage system. It is divided into 100 points from 1 to 100. The rating is 1 for the ugliest. The rating for 100 is the most beautiful. The higher the score, the more beautiful. The higher the degree, the lower the opposite. Here, only the eyes, eyebrows, mouth, and nose of the facial features are selected for evaluation.

Further, the beautification processing unit includes a face beauty database, and the face beauty database stores facial structure shapes, facial features, skin color, skin texture, male face beauty standard templates, and females in major Asian countries and regions. As a priori knowledge of facial beauty prediction, the database has a number of prediction models based on face image training in different regions.

Further, the beautification processing unit includes a beautification implementation unit and an automatic beautification unit, and the beautification implementation unit is configured to guide the facial beautification algorithm to select a beautiful standard model from the face beauty database. The automatic beautification unit refers to the system automatically implementing facial five-point smooth adjustment according to the beautification implementation scheme, and does not change the original texture of the face surface.

Further, the system further includes:

The software basic functional unit, including user registration, login and personalized application setting functions, realizes login and maintenance of software accounts through face recognition technology and fingerprint recognition technology;

The instant messaging unit, including the third-party social media account, logs in to the instant chat for online dating chat, adds other users as friends, adopts the group management mode, and accepts the system information and the evaluation information of the netizens in the user's personal network space;

Personal homepage unit, including personal homepage and EXE web album display, can carry out personalized space dressing, text entry, EXE web album automatic play, with EXE web album online production, and support online download function.

The invention has the beneficial effects that the invention realizes the "deep learning" face beautification on the Internet platform, collects the user's face image through the client and performs compression processing, uploads to the server via the Internet, and performs the person on the server. Face detection, face verification, face key point detection and face beautification algorithm processing avoid the disadvantages of poor performance of mobile phones and the inability to run deep learning networks. At the same time, the beautiful face prediction and the face landscaping are combined to complete the beautiful prediction. At the same time, according to the relevant parameters set by the user, the system automatically recommends the optimization scheme, through image processing technology such as affine and deep learning, popular learning, etc. The machine learning algorithm performs local smoothing adjustment of the face and facial features to achieve a change in the face value to a higher beauty. The whole process is automatically completed by the system, and the local aesthetic smoothing adjustment is only performed for the face position, which does not affect the overall image. After the processing is completed, the processing trace is not left behind.

The invention, as a software, combines the Internet and biometrics technology, and has intelligent and fully automated processing functions, which can improve image processing speed and beautify quality.

DRAWINGS

Figure 1 is a block diagram of the system of the present invention;

2 is a flow chart of a picture beautification shown by the present invention.

Detailed ways

In order to make the technical problems, technical solutions and advantageous effects to be solved by the present invention more clearly, the present invention will be described in detail below with reference to the embodiments. The specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The embodiment of the present application provides an Internet-based system, which is applied to a mobile-faced beautification APP. The face-beautifying APP includes image capturing and video recording, image and video storage, cloud face detection, cloud face key point detection, and cloud Face verification, cloud face value prediction, cloud facial features prediction, face beauty database, face beautification scheme and face automatic beautification, live chat and personal cyberspace function; the face landscaping APP also includes software registration , login and personalized application settings,

After the software is started, the system login and registration page is opened by default. If you use the software for the first time, you need to register before you can enter the system. Otherwise, you cannot use any function of the system. The software supports third-party social media account registration and login. By default, you can register by phone number or you can register by email. After successful registration, after entering the system, you can set the software PIN code, graphic password, fingerprint identification and face recognition login method on the setting page. The software automatically records the login account and password.

Further, the software part function is implemented by an algorithm developed by the “deep learning” technology. Since the mobile phone performance is poor, the deep learning network cannot be run, and the software configures face detection, face verification, and face key on the cloud server. Point detection and face beautification algorithm, collect images on the client, and call the API interface reserved by the algorithm to transmit the image to the cloud server for algorithm implementation. After the detection algorithm is processed, the server feeds the result back to the client, and Displayed on the client. Since the algorithm is implemented on the server, which involves the process of image uploading and downloading, better network conditions are very important for improving the software experience.

Further, after the client collects the image and uploads it to the server, the server adopts the image. The "deep learning" algorithm performs face detection. If the face can be detected, the algorithm returns the face ROI coordinates, and the server returns to the client. The client calls the drawing algorithm to select the face in the form of a red frame on the image. Area; if no face is detected, the system will return an error message and ask for a replacement image to retest. The detection algorithm detects the face of the face and also detects 68 feature points on the face of the person. The server feeds back the detected information back to the client and transmits it to other algorithms. In the face beautification stage, the 68 key points returned to the client will not be marked on the face image. If you need to view it, you need to enable the corresponding function in the software settings.

Further, after the "deep learning" face detection algorithm detects the face and 68 feature points, the image processing technology is used to segment the facial features, locate the specific location and area of the facial features, and according to the image submitted by the user. Parameters, such as birthplace, gender, age, etc., the system retrieves the beautiful model of “deep learning” that has been trained in the corresponding regions, genders and ages from the database, and carries out the beautiful prediction of face and the beauty of facial features, if the user does not provide personal Image parameters, the system defaults to gender and age testing, and calls the public training model for fuzzy testing. Due to differences in appearance and aesthetic concepts among people in different countries and regions, the beautiful results predicted by fuzzy algorithms may have large deviations, and may also The landscaping schemes that lead to the system's later delivery do not match, such as: recommending the five-person landscaping scheme of Xinjiang people to Jiangsu and Zhejiang people.

Further, after the system predicts the beauty of the face and the facial features, the information is fed back to the client and displayed on the client's screen. The beautiful pre-judgment scores of the face and the facial features are in percent, of which 1 is the ugliest and 100 is the The most beautiful, the higher the score, the higher the degree of beauty, and vice versa. After receiving the beautiful score, the user can then set the face beautification parameters, such as setting the beautiful score to be reached, the age stage to be reached, etc., so that the system automatically selects the optimization model and provides an optimized reference scheme according to the set parameters. After the system obtains the parameters submitted by the user, it calls the beautiful model in the beautiful analysis database, and uses the affine change technology in the manifold learning method, deep learning and image processing to map the facial features to the set beauty degree model, and then adopt the depth. The learning method integrates the details such as illumination brightness, and realizes the perfect fusion of the processed facial organs and human faces. The user can also set parameters without standardization, and the algorithm performs standardization processing by default, and finely adjusts the facial features below the standard beauty level.

Further, after the image beautification process, the "deep learning" face verification algorithm is then used to determine whether the image after the beautification process is still the same person as the original image. The algorithm uses the similarity degree to measure. Because the face facial features have been adjusted, the face facial feature details change, and the re-tested feature points are different. The algorithm does not directly give the judgment result of the same person, but gives a The similarity is judged. The similarity is divided into 100 points, from 0 to 100, where 0 means completely different, 100 means identical, and the higher the score, the higher the similarity, and vice versa. By default, the system sets a face similarity threshold, which is 0.4. If the similarity is lower than the value after the image processing, the beautification process is performed again. This threshold can be modified in the system settings. The face verification function is also integrated in the software, and the user can upload two images at the same time for verification. At this time, after the system processes, the judgment result is the same person or not the same person.

Further, users can upload their original images and processed images to the software public platform display, and accept ratings from other users. The scoring system sets 10 points, from 1 to 10, with 1 being the ugliest and 10 being the most beautiful. The higher the score, the higher the degree of beauty, and vice versa. When the score is scored by the netizen, you can also answer the questions related to the judgment of the system's beauty, or you can leave a message below the image. The system will calculate the score statistical histogram, the mean value of the face value and the variance of the score value according to the participation of the netizen and the score value, which is convenient for the user to refer to in order to check the validity of the score. Users can also participate in PK competitions with other users, in the form of a collection of praises, in a limited time, to win more praises, and enter a higher round of PK competition.

Further, users can also upload images, videos and EXE electronic albums in their personal space. Electronic photo albums support online production and local uploading, and online photo albums support downloading. The images in the personal space album support batch color value scoring and beautification processing operations, and the user can choose to keep the score traces.

Further, in order to facilitate communication and communication between users, the software activates the instant chat function, and after the user receives a new vote and comment on the public display platform, the voting information and the comment information are received by the system. Forward to the chat function area, users can get relevant consulting information at any time.

Further, the posting and follow-up qualification examination is used to standardize the use of software, advocate civilized speech, and prohibit publication of statements that are detrimental to national interests and infringement on the legitimate rights and interests of others, and are implemented by third-party algorithms to detect and find violations of software usage rules, according to In case of serious circumstances, penalties such as deleting posts and prohibiting speeches.

The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present invention are included in the present invention. Within the scope of protection.

Claims

An internet-based facial beautification system characterized by comprising:

Image and video acquisition unit for acquiring images and videos, processing and uploading to the server;

a face detection unit, configured to detect information uploaded by the image and video acquisition unit, and feed the result back to the client or the beautification processing unit;

The beautification processing unit is configured to process the face detection unit to detect the face information, and feed the result to the face detection unit or the client.
The Internet-based facial landscaping system according to claim 1, wherein the original image captured by the image and video acquisition unit is stored in a RAW high-definition format, and an image processing algorithm is called to reduce the resolution and sharpen the image. And enhance the contrast to obtain low-definition images. After the software is processed, the two images are uploaded to the server via the Internet. After the server receives the image, the low-definition image is used for face detection and high-definition images for the face. beautify.
The Internet-based facial landscaping system according to claim 2, wherein the image and video acquisition unit is further configured to store image resources, including local storage and server-side storage, and the local storage is used to store the image. On the storage medium of the client, the server-side storage stores the image resources on the public cloud storage platform and the server storage platform.
The Internet-based facial landscaping system according to any one of claims 1 to 3, wherein the face detecting unit deploys the face detection algorithm to a remote server, and calls an algorithm reservation on the local client. The API interface performs image processing on the remote server, and the face detection unit includes a client image uploading unit, a cloud face detecting unit, and a server image downloading unit, and the client image uploading unit is configured to upload an image by the client. Going to the server, the cloud face detection unit is configured to invoke a face detection algorithm deployed on the server to perform face detection processing on the received image, where the server image downlink unit is used by the server to process the processed image, the person Face coordinates are fed back to the client.
The Internet-based facial landscaping system according to claim 4, wherein the face detecting unit further comprises a cloud face key point detecting unit for remotely detecting 68 key features representing the face shape of the face. Points to determine facial facial features and feature extraction, the above process is performed on low-resolution images, and the extracted keypoint coordinates are fed back to the client and transmitted to the face beautification algorithm.
The Internet-based facial landscaping system according to claim 5, wherein the face detecting unit further comprises a cloud face verification unit, configured to determine the original face image and the image after the beautification unit is beautified Whether it is the same person, if the similarity is too low and below the system default similarity threshold, then re-beautify.
The Internet-based facial landscaping system according to claim 4, wherein the beautification processing unit uses a face detection algorithm to predict the degree of beauty of the face image, and obtains a pre-judgment score representative of the beauty of the face, beautiful The pre-judgment score is based on the percentage system, which is divided into 100 points from 1 to 100. The rating of 1 means the ugliest, the rating of 100 means the most beautiful, the higher the score, the higher the degree of beauty, and vice versa.
The Internet-based facial landscaping system according to claim 7, wherein the beautification processing unit further comprises a facial features beautiful facial value predicting unit, wherein the facial features beautiful facial value predicting unit is predicted in a facial face value On the basis of the facial features, the facial features are extracted from the face features and the 68 key feature points of the face, and the facial features are extracted. The pre-judgment of the beauty degree is adopted. The beautiful pre-judgment is based on the percentage system and is divided into 100 from 1 to 100. The score, the rating of 1 means the ugliest, the rating of 100 means the most beautiful, the higher the score, the higher the degree of beauty, and vice versa.
The Internet-based facial landscaping system according to claim 8, wherein the beautification processing unit further comprises a face beauty database, and the face beauty database stores face structure shapes of major Asian countries and regions. The distribution of facial features, skin color, delicate skin texture, beautiful standard template for male face, beautiful standard template for female face, beautiful standard template for male facial features and beautiful standard template data for female facial features.
The Internet-based facial landscaping system according to claim 9, wherein the beautification processing unit comprises a beautification implementation unit and an automatic beautification unit, and the beautification implementation unit is configured to guide the facial beautification algorithm from the face beauty database. The beautiful standard model is selected, and the automatic beautification unit refers to the system automatically realizes the smooth adjustment of the facial features according to the beautification implementation scheme, and does not change the original texture of the face surface.
The Internet-based facial landscaping system according to claim 1, wherein the system further comprises:

The software basic functional unit, including user registration, login and personalized application setting functions, realizes login and maintenance of software accounts through face recognition technology and fingerprint recognition technology;

The instant messaging unit, including the third-party social media account, logs in to the instant chat for online dating chat, adds other users as friends, adopts the group management mode, and accepts the system information and the evaluation information of the netizens in the user's personal network space;

Personal homepage unit, including personal homepage and EXE web album display, can carry out personalized space dressing, text entry, EXE web album automatic play, with EXE web album online production, and support online download function.