CN110807108A - Asian face data automatic collection and cleaning method and system - Google Patents

Abstract

The invention discloses a method and system for automatically collecting and cleaning Asian face data. The method includes the following steps: presetting a plurality of Asian target person identifiers, obtaining official photo links of the Asian target person, and constructing a plurality of Asian target person. A data list identifying key information; searching and obtaining reference character data associated with the Asian target character identification and keywords according to the content of the data list; storing the reference character data in association with the corresponding Asian target character identification and keywords; cleaning the stored With reference to the person data, the target face data associated with the Asian target person is obtained. The invention realizes the automatic collection and cleaning of Asian face data, replaces the traditional heavy processes such as manual labeling and classification, greatly reduces the manpower and time cost of establishing the Asian face database, and also solves the problem of different categories in the existing face database. balance issue.

Description

A method and system for automatic collection and cleaning of Asian face data

technical field

The invention relates to the technical field of image processing and recognition, in particular to a method and system for automatically collecting and cleaning Asian face data.

Background technique

Most of the existing face recognition technologies are face recognition models trained based on deep learning methods. In order to improve the accuracy of recognition, the model needs to be trained using a database containing a large number of labeled face photos. The main source of face photos is It is downloaded and collected from the Internet through crawler means, and then the photos need to be marked and cleaned through complicated manual operations. This series of work not only requires very high computing and storage equipment, but also requires a lot of manpower and time costs. Due to their unique image resources and operating capital advantages, Internet giants have private large-scale face datasets, but so far there are very few large-scale public face datasets that ordinary users can obtain for free. The face datasets mainly include Youtube Face, CASIA-WebFace and MS-1M-Celeb.

What's more, most of the existing face data sets are dominated by European and American face data, which only contain a very small amount of Asian face data. Using this type of unbalanced face data set to train neural networks will easily lead to There is a potential problem of "racism" on the Internet. In addition, there are very few Asian face data sets that can be obtained online so far, and there are often problems such as a small number of characters and a small amount of face data. Therefore, under the premise of limited manpower and time cost, establishing a large-scale Asian face dataset has great scientific and commercial value.

SUMMARY OF THE INVENTION

In order to overcome the defects and deficiencies of the prior art, the present invention provides a method and system for automatically collecting and cleaning Asian face data, which automatically cleans the collected Asian face photo data, so as to achieve low time cost and per capita workload. It has less effect and can build an Asian face database with a high recall rate.

In order to achieve the above object, the present invention adopts the following technical solutions:

The present invention provides a method for automatically collecting and cleaning Asian face data, comprising the following steps:

Presetting a plurality of Asian target person identifiers, obtaining the official photo link of the Asian target person, and constructing a data list, the data list including the key information of the plurality of Asian target person identifiers;

According to the content of the data list, after adding keywords to the Asian target character identification and the character identification, a search is performed to obtain reference character data, and the specific steps include:

According to the official photo link of the Asian target person, obtain the official photo of the Asian target person;

Searching the Asian target person identification to obtain the associated reference person data in the case of a single identification;

Searching the Asian target person identification plus a variety of keywords, respectively obtaining the associated reference person data corresponding to different keywords on a single identification combination;

Associate the searched reference person data with the corresponding Asian target person identifiers and keywords, until all reference person data are associated and stored;

Clean the stored reference character data to obtain the target face data associated with the Asian target character.

As a preferred technical solution, the Asian target person identification adopts the name of different Asian target person or the number number used to distinguish different Asian target person, the official photo link of the Asian target person adopts URL link, and each row of the data list corresponds to An Asian target person logo and corresponding official photo URL link.

As a preferred technical solution, the reference character data after the search is stored in association with the corresponding Asian target character identifiers and keywords, and the specific steps are:

Create a main folder, the main folder is named by using the Asian target character identifier, and multiple sub-folders are created in the main folder, which are named by keywords;

The reference character data obtained through different search methods are associated and saved in the corresponding subfolders.

As a preferred technical solution, the specific steps of cleaning the stored reference character data are as follows:

Use image processing tools to check the readability and unify the format of all reference character data, and eliminate reference character data that cannot be read and written normally;

Delete the obtained data of repeated download reference characters, and keep the official photos of the Asian target characters;

Preliminary cleaning: use the face detection algorithm to perform face detection on the reference character data, and obtain the reference face data processed by face detection;

Deep cleaning: use the face recognition algorithm to detect the reference face data processed by face detection, update the official photo list of the Asian target person identification, and check whether it matches the official photo list of the Asian target person identification. If the reference face data after face detection processing matches, the reference face data after face detection processing will be retained as the target face data associated with the Asian target person.

As a preferred technical solution, the deletion of the obtained repeated download reference character data, the specific steps are:

After the reference character data is associated with the corresponding Asian target character identification and the keyword and is stored in the folder, it is a standard whether the file naming of the reference character data is the same as the described repeated download. One of them, and the rest of the duplicate reference character data will be deleted.

As a preferred technical solution, the use of a face detection algorithm to perform face detection on the reference character data, the specific steps are:

Through the face detection algorithm, locate the position of the key points of the face in the reference character data and detect the face frame,

If there is no face frame, delete the reference character data;

If there is a face frame, the reference face data in the face frame is cut out, and the reference character data is retained;

If there are multiple face frames, each reference face data in the multiple face frames is cut out, while the reference character data is retained, and the official photos corresponding to the Asian target characters and the face features of each reference face data are extracted, and calculated separately. The matching degree between the facial features of each reference face data and the facial features of the official photos corresponding to the Asian target person, and the reference person data corresponding to the reference face data with the highest matching degree is used as the reference person after face detection processing. Face data is preserved.

As a preferred technical solution, the specific steps of the deep cleaning are:

Extract the official photos of Asian target people and the facial features corresponding to the reference face data after preliminary cleaning based on the face recognition algorithm;

Calculate the matching degree between the facial features of the official photos of the Asian target person and the facial features corresponding to the reference face data after preliminary cleaning, and classify the reference facial data whose matching degree is greater than or equal to the first preset threshold into the target person. Official photo list, update the official photo list of the target person;

Match the face features of the remaining reference face data after preliminary cleaning with the face features in the updated official photo list of Asian target people one by one, retain the reference face data whose matching degree is greater than or equal to the second preset threshold, and delete The rest of the reference face data;

The first preset threshold is greater than the second preset threshold.

The present invention also provides a system for automatically collecting and cleaning Asian face data, comprising: a data list building module, a reference character data acquisition module, an associated storage module and a reference character data cleaning module;

The data list building module is used to obtain the official photo link of the Asian target person by presetting a plurality of Asian target person identifiers, and construct a data list;

The reference character data acquisition module is used to obtain reference character data associated with the Asian target character identifier and the character identifier keyword according to the content of the data list;

The associative storage module is used to associate and store the reference character data with the corresponding Asian target character identifiers and keywords;

The reference character data cleaning module is used for cleaning the stored reference character data to obtain target face data associated with the Asian target character.

As a preferred technical solution, the reference character data cleaning module includes a preliminary cleaning sub-module and a deep cleaning sub-module, and the preliminary cleaning sub-module is used to perform face detection on the reference character data by using a face detection algorithm, and obtain a face Detecting the processed reference face data, the deep cleaning submodule is used to detect the reference face data processed by face detection using a face recognition algorithm, update the official photo list of the Asian target person identification, and check whether it matches the Asian target person. Identify the official photo list match, and use the matched reference face data as the target face data associated with the Asian target person.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

(1) The present invention uses image processing tools to check the readability and unify the format of all reference character data, and delete the reference character data downloaded repeatedly, thereby improving the fluency and processing efficiency of the subsequent cleaning process.

(2) The present invention adopts a variety of search methods to obtain reference character data, thereby increasing the diversity and accuracy of obtaining reference character data.

(3) The present invention uses preliminary cleaning and deep cleaning to clean the data, obtains the target face data associated with the target person identification, and updates the official photo list of the Asian target person identification in the deep cleaning, that is, updates the reference database of reference, improves the human The accuracy of face feature alignment.

(4) The present invention performs automatic processing from the collection of Asian face data to the entire process of cleaning, which replaces the traditional heavy processes such as manual labeling and classification, greatly reduces the time cost of establishing the Asian face database, and also solves the problem of Problems such as category imbalance in the existing face database are solved.

Description of drawings

1 is a schematic flow chart of the Asian face data automatic collection and cleaning method of the present embodiment;

2 is a schematic diagram of a data list of the Asian face data automatic collection and cleaning method of the present embodiment;

3 is a schematic diagram of the effect when cleaning is not performed in the Asian face data automatic collection and cleaning method of the present embodiment;

4 is a schematic diagram of the effect of preliminary cleaning in the Asian face data automatic collection and cleaning method of the present embodiment;

FIG. 5 is a schematic diagram of the effect of collecting Asian face data in the method for automatically collecting and cleaning Asian face data according to this embodiment.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

Example

As shown in Figure 1, a method for automatic collection and cleaning of Asian face data includes the following steps:

S1. Preset multiple Asian target person identifiers, obtain the corresponding official photo link, and construct a data list including key information of multiple Asian target person identifiers;

The identifiers of multiple Asian target persons in step S1 are the names of different Asian target persons or a preset number number used to distinguish different Asian target persons; for example, Baidu provides a "Baidu Baike Star Popularity List". The list includes sub-modules such as "Mainland China Male Star List", "Mainland China Female Star List", "Hong Kong and Taiwan Southeast Asia Male Star List", "Hong Kong and Taiwan Southeast Asia Female Star List" and other sub-modules, which can be automatically Obtain the list of selected lists, record the names of the stars in the sub-module of Asian stars on the list in order on a data list, and optionally use an integer starting with 0 and gradually increasing as a digital label to distinguish Asian stars; The embodiment uses a public figure as an Asian target person identification to have two advantages, in addition to being able to easily obtain a large number of photos of the designated target person through the identification on the search engine, at the same time, any privacy and infringement problems caused by using these photos can be avoided;

For the official photo link in step S1, in this embodiment, Baidu Encyclopedia is used to provide the URL link of the downloaded official photo of the Asian target person. For example, the "Baidu Baike Star Popularity List" will display the star's official photos and names in turn according to the star's real-time popularity. Clicking on the star's official photo or name will enter the corresponding Baidu Baike introduction page, and the star can be obtained through crawler technology. The URL link to the official photo is displayed on the page and the link is recorded on the data list.

As shown in Figure 2, in the schematic diagram of the list of obtaining reference character data, the reference character data includes key information of multiple Asian target character identifiers, and each row corresponds to an Asian target character identifier and its corresponding official photo URL link. To the right are numeric labels, names, and URL links, separated by tabs '\t';

S2, according to the content of the data list, search after adding keywords to the Asian target person identifier and the person identifier to obtain reference person data;

The specific steps of step S2 are:

According to the URL link of the official photo of the Asian target person, use computer technology to obtain the official photo of the Asian target person, wherein the computer technology is optional but not limited to web crawler, downloader tools, etc.;

Search for the Asian target person identification, and obtain at least one reference person data associated with a single identification; specifically, if the Asian target person identification is the name of the target person, then the reference person data is the relevant photo of the Asian target person, such as , the present embodiment can adopt the process of using a Python script to simulate manual search and download pictures, and can easily obtain a large amount of reference character data by performing a search on the Asian target character identification on the Baidu image search engine;

In this embodiment, the target person identifier plus keywords are also searched to obtain at least one associated reference character data corresponding to different keywords on a single identifier combination, wherein the keywords are optional but not limited to glasses, hats, actors, For singers (professional), etc., various reference character data such as "target character name + glasses", "target character name + hat", "target character name + occupation" can be obtained, which can increase the diversity and accuracy of the obtained reference character data. sex;

S3, successively associate and store at least one reference character data with the corresponding Asian target character identifier and the data added with the keyword, until all the reference character data are associated and stored;

Specifically, photos are collected and stored for each Asian target person identification in the data list, for example:

Create a main folder and name it with the Asian target person identifier (such as the digital label corresponding to the Asian target person), then create multiple subfolders in the main folder and name them with keywords (such as the English spelling of keywords) ;

Save at least one reference character data of Asian target characters obtained through different search methods in a corresponding subfolder;

S4, automatically cleaning the stored reference character data in turn to obtain target face data associated with the Asian target character;

The specific steps of step S4 include:

S41. Use an image processing tool to check the readability and unify the format of all reference character data, and remove a small part of the reference character data that cannot be read and written normally due to download errors, format errors, etc. The image processing tool is optional but Not limited to MATLAB, Python, OpenCV, Photoshop and other image processing software or programming languages; for example, when batch downloading pictures from the Internet through the crawler method, it is often affected by network fluctuations and anti-crawling mechanisms, resulting in incomplete content of the downloaded pictures, Download errors and other problems, such pictures generally cannot be read and written by the software normally, which will seriously affect the smoothness of the cleaned data and should be eliminated to improve the efficiency of data processing; in addition, for the convenience of subsequent processing and data management, in the data cleaning work Before starting, MATLAB can be used to unify all reference character data into a common JPEG format;

S42. Delete the repeatedly downloaded reference data of the Asian target person obtained by using different search methods, and retain the official photo of the Asian target person;

Among them, the repeated download is based on the same file name as the reference data; for example, to search for a public figure, first create a main folder, and the search methods can be "someone's name, one's name + hat, one's name + glasses, one's name Name + singer", etc., so create several subfolders name, hat, glass, job, etc. in the main folder, save the official photos in the standard subfolder, and obtain reference character data in the search engine through different combinations. The engine inevitably returns some reference character data with the same name and content. This kind of repeated data can easily lead to overfitting of the neural network and seriously affect the face recognition performance of the network. Therefore, the purpose of this step is to target the subfolders in All the files with the same name, delete the duplicate and keep only one; the official photos in the standard subfolder of this embodiment are retained, and the official photos downloaded from the URL can be named standard. The photo retains the image naming of the source network, so that the official photo naming in the standard subfolder does not repeat with other images; or the official photo naming in the standard subfolder can not be changed, and a judgment link is added. If one of the images is repeated The picture is located in the standard subfolder, and the pictures in the standard subfile are kept.

The specific steps of step S42 include:

Use the program to sequentially record the file naming of the reference person data in the subfolder associated with the Asian target person, wherein the program is optional but not limited to be written in MATLAB, Python and other languages;

If there are duplicate downloads, only one of the reference character data will be retained, and the remaining duplicate reference character data will be deleted;

S43. Perform face detection work on the reference character data based on the face detection algorithm, so that the cleaned reference character data only includes the reference face data, wherein the face detection work includes face detection, face correction, and face alignment and other series of steps, the reference character data before cleaning includes reference face data and reference non-face data;

The specific steps of step S43 include:

Perform face detection on the reference character data based on the face detection algorithm, and obtain the reference face data processed by face detection. The face detection algorithm may adopt but not limited to deep learning methods such as MTCNN, and deep learning methods such as MTCNN. It includes a series of steps such as face detection, face correction, and face alignment. Among them, face detection can remove some non-face data, such as pictures such as glasses and hats, and face correction and face alignment can The laterally skewed faces are corrected and aligned to improve the processing efficiency of subsequent face feature matching.

As shown in FIG. 3 , when the cleaning work is not performed, the reference character data includes reference face data and reference non-face data. The preliminary cleaning work performed in this embodiment is to remove the reference non-face data in the reference character data. Commonly used The methods include the MTCNN deep learning method and the face detection toolkit that comes with the OpenCV software. The specific principle is to detect and locate the five key points (eyes, nose, mouth corners) of the face in the photo through the algorithm and return the face. frame, you can judge whether the reference person data is the reference face data according to whether the face frame is returned; if the face frame is not returned, delete the photo; if a face frame is returned, cut out a face frame If more than one face frame is returned, the reference face data in more than one face frame will be cut out, and the reference face data processed by face detection will be retained, and then based on The face recognition algorithm selects the reference face data with the highest matching degree with the official photo of the Asian target person, and deletes the rest of the reference face data;

Further, in view of the situation that the above-mentioned face detection algorithm returns more than one face frame, it is necessary to first create a folder named after the photo name, and then cut out the faces in each face frame in the photo, and use the available The differentiating methods are stored in the folder in turn, and then, based on the face recognition algorithm, the official photo of the photo corresponding to the Asian target person and the face features of each face frame photo in the folder are extracted, and the face of each face frame photo is calculated separately. The matching degree between the features and the facial features corresponding to the official photos of the Asian target person, keep the face frame photo with the highest matching degree, and delete the remaining face frame photos, move the retained photos out of the folder and replace the original reference person data, and delete the folder at the same time, as shown in Figure 4, it can be seen that most of the reference non-face data has been eliminated after preliminary cleaning;

S44 , based on the face recognition algorithm, detect whether the reference face data after preliminary cleaning matches the official photo of the Asian target person identification, if not, delete the reference face data after preliminary cleaning, and retain the reference person after preliminary cleaning if they match face data in order to obtain the target face data associated with the target person; specifically, after preliminary cleaning of the reference person data, there is no guarantee that the obtained reference face data belong to the target person, so it is necessary to analyze the obtained reference face data. Perform in-depth data cleaning, that is, retain the target face data with the same identification as the Asian target person, delete the target face data different from the Asian target person identification, so as to complete the final data cleaning work;

The specific steps of step S44 include:

Extract the facial features of the official photos of Asian target people and their corresponding cleaned reference face data based on the face recognition algorithm;

Calculate the degree of matching between the facial features of the official photos of the Asian target person and the facial features of the initially cleaned reference facial data, and classify the cleaned reference facial data whose matching degree is greater than or equal to the first preset threshold into the The official photo list of the target person. At this time, the official photo list of the target person is not only the official photo, but also includes the target person's photo with a high degree of matching obtained by adding a keyword to the target person. In the face data, the face data of the Asian target person obtained through the keyword search is compared with the official photo of the target person obtained only through the URL link of the official photo of the Asian target person in step S2, and it is easy to appear that the matching degree is low. For example, the official photo is a frontal face photo without accessories such as hats and glasses, and the photos searched by adding keywords (such as hats, glasses) (after face recognition screening) are not decorated with hats, glasses, etc. Comparing the official photos of accessories such as glasses is prone to match deviation. Therefore, it is necessary to add the target person's photo with a high degree of matching obtained by adding a keyword search to the target person's official photo list and update the target person's official photo list. List of photos to increase the accuracy of face feature comparison;

Match the remaining preliminarily cleaned reference face data with the updated face features of the data in the official photo list of the target person one by one, and retain the remaining preliminarily cleaned reference face data whose matching degree is greater than or equal to the second preset threshold , and the remaining reference face data after preliminary cleaning is deleted.

In this embodiment, a face recognition algorithm is used to sequentially extract the feature vector of the Asian target person corresponding to the initially cleaned reference face data, and at the same time, the first preset threshold is set to 0.9, and then the feature vector of the reference face data and The feature vectors of the official photos are matched one by one. In the first round, the reference face data with a matching degree greater than or equal to 0.9 can be screened out, and these photos can be considered as the official photos of the target person; then, the second preset threshold value Set it to 0.7, and then match the feature vector of the remaining reference face data with the feature vector of the photos in the official photo queue selected in the first round, and match the feature vector of any photo in the official photo queue. As long as it is greater than or equal to 0.7, the remaining reference face data is retained, otherwise it is deleted; the first preset threshold and the second preset threshold used in the embodiment can be adjusted according to actual conditions.

As shown in Figure 5, after two rounds of cleaning, a high-purity Asian face database can be obtained. Most of the photos in the folder identified by each Asian target person belong to the target person, and there are very few noise photos.

This embodiment also provides a system for automatically collecting and cleaning Asian face data, including: a data list building module, a reference character data acquisition module, an associated storage module, and a reference character data cleaning module;

In this embodiment, it includes: a data list building module, a reference character data acquisition module, an associated storage module and a reference character data cleaning module;

In this embodiment, the data list building module is used to obtain the official photo link of the Asian target person by presetting multiple identifiers of the Asian target person, and construct a data list; the reference person data acquisition module is used to obtain information related to the Asian target person according to the content of the data list. The reference character data associated with the identification and character identification keywords; the associative storage module is used to associate and store the reference character data with the corresponding Asian target character identification and keywords; the reference character data cleaning module is used to clean the stored reference Person data, get the target face data associated with the Asian target person.

In this embodiment, the reference character data cleaning module includes a preliminary cleaning sub-module and a deep cleaning sub-module, and the preliminary cleaning sub-module is used to perform face detection on the reference character data by using a face detection algorithm, and obtain a face detection processing. With reference to face data, the deep cleaning sub-module is used to detect the reference face data processed by face detection using the face recognition algorithm, update the official photo list of the Asian target person identification, and check whether it matches the official photo list of the Asian target person identification. Use the matched reference face data as the target face data associated with the Asian target person.

In this embodiment, the entire process from the collection of Asian face data to the cleaning process replaces the traditional heavy processes such as manual labeling and classification through automatic processing, which greatly reduces the time and cost of establishing the Asian face database, and also solves the problem of existing problems. There are problems such as category imbalance in the face database, which promotes the development and progress of corresponding technologies.

The above-mentioned embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited by the above-mentioned embodiments, and any other changes, modifications, substitutions, combinations, The simplification should be equivalent replacement manners, which are all included in the protection scope of the present invention.

Claims (9)

1. A method for automatically collecting and cleaning Asian face data is characterized by comprising the following steps:

presetting a plurality of Asian target character identifications, acquiring an official photo link of the Asian target character, and constructing a data list, wherein the data list comprises a plurality of Asian target character identification key information;

according to the content of the data list, after keywords are added to the Asian target character identification and the character identification, searching is carried out to obtain reference character data, and the specific steps comprise:

obtaining an official photo of the Asian target character according to the Asian target character official photo link;

searching the Asian target character identification to obtain associated reference character data under the condition of single identification;

adding a plurality of keywords to the Asian target character identification for searching, and respectively acquiring associated reference character data corresponding to different keywords on a single identification combination;

performing associated storage on the searched reference character data, the corresponding Asian target character identification and the keywords until all the reference character data are stored in an associated manner;

and cleaning the stored reference character data to obtain target face data associated with the Asian target characters.

2. The method of claim 1, wherein said asian target character identifiers are different asian target character names or different asian target character numbers, said asian target character official photo links are URL linked, and each row of said data list corresponds to an asian target character identifier and a corresponding official photo URL link.

3. The Asian face data automatic collection and cleaning method as claimed in claim 1, wherein the storing of the searched reference character data in association with the corresponding Asian target character identifier and keywords comprises the following specific steps:

creating a main folder, wherein the main folder is named by adopting an Asian target character identifier, and a plurality of subfolders are created in the main folder and are named by adopting keywords respectively;

and storing the reference person data acquired by different searching modes in corresponding subfolders in a related manner.

4. The Asian face data automatic collection and cleaning method as claimed in claim 1, wherein the cleaning of the stored reference character data comprises the following steps:

performing readability inspection and format unification on all reference character data by adopting a picture processing tool, and removing the reference character data which cannot be read and written normally;

deleting the obtained repeatedly downloaded reference character data, and reserving official photos of the Asian target characters;

primary cleaning: performing face detection on the reference figure data by adopting a face detection algorithm to obtain reference face data subjected to face detection processing;

deep cleaning: and detecting reference face data after face detection processing by adopting a face recognition algorithm, updating an Asian target figure identification official photo list, checking whether the reference face data are matched with the Asian target figure identification official photo list, if not, deleting the reference face data after face detection processing, and if so, keeping the reference face data after face detection processing as target face data associated with the Asian target figure.

5. The Asian face data automatic collection and cleaning method according to claim 4, wherein the deleting of the repeatedly downloaded reference character data comprises the following specific steps:

and after the reference character data, the corresponding Asian target character identification and the keywords are stored in a folder in an associated mode, the repeated downloading adopts the standard that whether the file names of the reference character data are the same or not, if the repeated downloading exists, one of the repeatedly downloaded reference character data is reserved, and the rest repeatedly downloaded reference character data is deleted.

6. The Asian face data automatic collection and cleaning method as claimed in claim 4, wherein the face detection of the reference person data by the face detection algorithm comprises the following steps:

positioning the positions of key points of the human face in the reference person data through a human face detection algorithm and detecting a human face frame;

if the face frame does not exist, deleting the reference character data;

if a face frame exists, cutting out reference face data in the face frame, and keeping the reference character data;

if a plurality of face frames exist, cutting out each reference face data in the plurality of face frames respectively, simultaneously retaining the reference character data, extracting the official photo corresponding to the Asian target character and the face features of each reference face data, respectively calculating the matching degree of the face features of each reference face data and the face features of the official photo corresponding to the Asian target character, and retaining the reference character data corresponding to the reference face data with the highest matching degree as the reference face data after face detection processing.

7. The Asian face data automatic collection and cleaning method as claimed in claim 4, wherein the deep cleaning comprises the following specific steps:

extracting the official photos of the Asian target characters and the face features of the reference face data corresponding to the preliminarily cleaned Asian target characters based on a face recognition algorithm;

respectively calculating the matching degree of the face features of the Asian target character official photo and the face features of the reference face data corresponding to the preliminarily cleaned face features, classifying the reference face data with the matching degree being greater than or equal to a first preset threshold value into a target character official photo list, and updating the target character official photo list;

matching the face features of the reference face data after the residual preliminary cleaning with the face features in the updated Asian target character official photo list one by one, keeping the reference face data with the matching degree larger than or equal to a second preset threshold value, and deleting the rest reference face data;

the first preset threshold is greater than a second preset threshold.

8. An Asian face data automatic collection and cleaning system, comprising: the system comprises a data list construction module, a reference character data acquisition module, an association storage module and a reference character data cleaning module;

the data list building module is used for obtaining Asian target character official photo links by presetting a plurality of Asian target character identifications and building a data list;

the reference character data acquisition module is used for acquiring reference character data associated with the Asian target character identification and the character identification key words according to the content of the data list;

the association storage module is used for performing association storage on the reference character data, the corresponding Asian target character identification and the keywords;

the reference character data cleaning module is used for cleaning the stored reference character data to obtain target face data associated with the Asian target characters.

9. The Asian face data automatic collection and cleaning system according to claim 8, wherein the reference character data cleaning module comprises a preliminary cleaning sub-module and a deep cleaning sub-module, the preliminary cleaning sub-module is used for performing face detection on the reference character data by adopting a face detection algorithm to obtain the reference face data after face detection processing, the deep cleaning sub-module is used for detecting the reference face data after face detection processing by adopting a face recognition algorithm, updating the Asian target character identification official photo list, checking whether the reference face data are matched with the Asian target character identification official photo list, and taking the matched reference face data as the target face data associated with the Asian target character.

Images