CN108021669B - Image classification method and apparatus, electronic device, computer-readable storage medium - Google Patents

Abstract

The present application relates to an image classification method and device, an electronic device, and a computer-readable storage medium. The first-level face that meets preset conditions is obtained from an image according to the size of the face area, and the first-level face is classified into the first-level face. Recognition to get the face category corresponding to the first-level face. Then obtain the second-level face in the image, perform face recognition on the second-level face, and obtain the face category corresponding to the second-level face, and the second-level face is the first-level face in the image. outside face. Finally, according to the face category corresponding to the first-level face and the face category corresponding to the second-level face, the images are classified into corresponding face categories. The face categories obtained in this way will be more accurate, and there will not be a large number of unimportant face categories, which will improve the effectiveness of the classification results and better meet the actual needs of users. Finally, the images are classified into corresponding face categories.

Description

Image classification method and apparatus, electronic device, computer-readable storage medium

technical field

The present application relates to the field of computer technology, and in particular, to an image classification method and apparatus, an electronic device, and a computer-readable storage medium.

Background technique

With the popularization of electronic devices and the rapid development of the mobile Internet, the number of users of electronic devices is increasing. The photo album function has become one of the commonly used applications of electronic devices, and is an application that is frequently used by users. A large number of images are stored in the photo albums of electronic devices. The traditional photo albums of electronic devices provide various image browsing and classification functions. For example, personal image classification based on facial features is a popular image display method at present. However, the traditional image classification method will generate many useless classifications, resulting in waste of resources, which does not meet the actual needs of users.

SUMMARY OF THE INVENTION

Embodiments of the present application provide an image classification method and apparatus, electronic device, and computer-readable storage medium, which can improve the effectiveness of classification results and better meet the actual needs of users.

An image classification method comprising:

According to the size of the face area, a first-level face that satisfies a preset condition is obtained from an image, and face recognition is performed on the first-level face to obtain a face category corresponding to the first-level face;

Obtain the second-level face in the image, perform face recognition on the second-level face, and obtain the face category corresponding to the second-level face, and the second-level face is the The faces in the image except the first-level faces;

The images are classified into corresponding face categories according to the face categories corresponding to the first-level faces and the face categories corresponding to the second-level faces.

An image classification device, the device includes:

The first-level face recognition module is used to obtain a first-level face that meets preset conditions from an image according to the size of the face area, perform face recognition on the first-level face, and obtain the first-level face The face category corresponding to the face;

The second-level face recognition module is used to obtain the second-level face in the image, perform face recognition on the second-level face, and obtain the face category corresponding to the second-level face, The second-level human face is a human face other than the first-level human face in the image;

The image classification module is configured to classify the images into corresponding face categories according to the face categories corresponding to the first-level faces and the face categories corresponding to the second-level faces.

An electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the instructions are executed by the processor, the processor causes the processor to execute the steps of the image classification method as described above.

A computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the image classification method as described above when executed by a processor.

The above-mentioned image classification method and device, electronic device, and computer-readable storage medium, first, obtain a first-level face that satisfies a preset condition from an image according to the size of the face area, and perform face recognition on the first-level face, Get the face category corresponding to the first-level face. Then obtain the second-level face in the image, perform face recognition on the second-level face, and obtain the face category corresponding to the second-level face, and the second-level face is the first-level face in the image. outside face. Finally, according to the face category corresponding to the first-level face and the face category corresponding to the second-level face, the images are classified into corresponding face categories. The faces in the image are extracted in stages, and the first-level faces are extracted in descending order of face area, and the extracted first-level faces need to meet preset conditions, so , the conditions for extracting the first-level face are very strict, and face recognition is performed on the first-level face to obtain the face category corresponding to the first-level face. Then continue to obtain the second-level face from the image, and then perform face recognition on the second-level face to obtain the face category corresponding to the second-level face. The face categories obtained in this way will be more accurate, and there will not be a large number of unimportant face categories, which will improve the effectiveness of the classification results and better meet the actual needs of users. Finally, the images are classified into corresponding face categories.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1A is an application scenario diagram of an image classification method in one embodiment;

1B is an internal structure diagram of an electronic device in one embodiment;

2 is a flowchart of an image classification method in one embodiment;

3 is a flow chart of a method for obtaining a first-level face from an image in descending order of face area;

4 is a flowchart of a method for obtaining a face category corresponding to a second-level face in one embodiment;

5 is a flowchart of a method for obtaining a face category corresponding to a second-level face in one embodiment;

6 is a flowchart of a method for obtaining a face category corresponding to a second-level face and a third-level face method in one embodiment;

7 is a flowchart of a method for obtaining a face category corresponding to a third-level face in one embodiment;

8 is a schematic structural diagram of an image classification apparatus in an embodiment;

Fig. 9 is the structural representation of the first-level face recognition module in Fig. 8;

Fig. 10 is the structural representation of the second-level face recognition module in Fig. 8;

11 is a schematic structural diagram of another second-level face recognition module in FIG. 8;

FIG. 12 is a block diagram of a partial structure of a mobile phone related to an electronic device provided in an embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

FIG. 1A is an application scenario diagram of an image processing method in an embodiment. As shown in FIG. 1A , the application environment includes an electronic device 110 and a server 120 . The terminal 110 and the server 120 are connected through a network. An image is stored in the electronic device 110 , and the image may be stored in the internal memory of the electronic device 110 or in an SD (Secure Digital Memory Card, Secure Digital Memory Card) card built in the electronic device 110 . The electronic device 110 can obtain a first-level face that meets a preset condition from an image according to the size of the face area, perform face recognition on the first-level face, and obtain a face category corresponding to the first-level face. Obtain the second-level face in the image, perform face recognition on the second-level face, and obtain the face category corresponding to the second-level face, and the second-level face is the image except the first-level face. face. According to the face category corresponding to the first-level face and the face category corresponding to the second-level face, the images are classified into corresponding face categories. Of course, in the above image classification method, the electronic device 110 can also initiate an image classification request to the server 120 , and the image classification is completed on the server 120 , and the server 120 sends the image classification result to the electronic device 110 .

FIG. 1B is a schematic diagram of the internal structure of an electronic device in one embodiment. As shown in FIG. 1B , the electronic device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory is used for storing data, programs, etc., and at least one computer program is stored in the memory, and the computer program can be executed by the processor to implement the image classification method applicable to the electronic device provided in the embodiments of the present application. The memory may include a non-volatile storage medium such as a magnetic disk, an optical disk, and a read-only memory (Read-Only Memory, ROM), or a random-access-memory (Random-Access-Memory, RAM) and the like. For example, in one embodiment, the memory includes a non-volatile storage medium and internal memory. The nonvolatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement an image classification method provided by the following embodiments. Internal memory provides a cached execution environment for operating system computer programs in non-volatile storage media. The network interface can be an Ethernet card or a wireless network card, etc., and is used to communicate with external electronic devices. The electronic device may be a mobile phone, a tablet computer, a personal digital assistant or a wearable device, and the like.

In one embodiment, as shown in FIG. 2, an image classification method is provided, and the method is applied to the electronic device in FIG. 1A as an example for description, including:

Step 202: Obtain a first-level face that meets a preset condition from an image according to the size of the face area, perform face recognition on the first-level face, and obtain a face category corresponding to the first-level face.

The electronic device obtains the images to be classified from the local or cloud album, and then obtains the first-level faces that meet the preset conditions from the images according to the size of the face area from the images to be classified. The images to be classified include single-person images and multi-person group photo images. There is only one face in a single-person image, and multiple faces are included in a group photo image. Among them, for a group photo image of multiple people, according to the order of the area size of the faces in the image, it is judged from large to small whether the faces meet the preset conditions, and if so, the faces that meet the conditions are the first-level faces. The first-level face refers to the face of the core person in each image. When extracting the first-level first-level face for each image for the first time, only one face is extracted from each image as the first-level face. Perform face recognition on the acquired first-level face of each image according to the face recognition algorithm, and obtain the face category corresponding to the first-level face of each image. The face category refers to the identity of the person corresponding to the first-level face recognition result. Specifically, after performing face recognition on the first-level face, it is obtained that the identity corresponding to the first-level face is Zhang San, that is, the face category corresponding to the first-level face is obtained as Zhang San.

Step 204: Acquire the second-level face in the image, perform face recognition on the second-level face, and obtain the face category corresponding to the second-level face, where the second-level face is the first-level person removed from the image. face other than face.

After the first-level face of each image is obtained from the images to be classified, the second-level face in the image is obtained. Among them, the second-level face is the face in the image except the first-level face. For a single-person image, there are no faces except the first-level face, but for a group photo image of multiple people, there are faces other than the first-level face.

The second-level face in the image is processed, the clarity of the second-level face in the image is calculated respectively, and it is judged whether the clarity of the second-level face meets a preset condition. If so, perform face recognition on the second-level face that satisfies the preset definition condition, and obtain the face category corresponding to the second-level face. If the clarity of the second-level face in the image does not reach the clarity threshold that can identify the identity corresponding to the face, continue to search for the first-level face similar to the second-level face in other images in the album. If a first-level face similar to the second-level face is found, the second-level face is classified into a face category corresponding to the first-level face. If the first-level face similar to the second-level face is not found, look for a face similar to the second-level face in other images in the album, and calculate the appearance of the face similar to the second-level face. number of times. If the number of times reaches the set threshold, face recognition is performed on the second-level face, and the face category corresponding to the second-level face is obtained. If the number of times does not reach the set threshold, the second-level face is identified. For the third-level face.

Step 206: Classify the images into corresponding face categories according to the face categories corresponding to the first-level faces and the face categories corresponding to the second-level faces.

The first-level face and the second-level face in the image are classified by the above steps, and those that can be classified are classified into corresponding face categories. Then, according to the face categories that the first-level face and the second-level face in the image can be classified into, the image is classified into the corresponding face category. Specifically, if the first-level face and the second-level face of an image are classified into 3 categories respectively, the image will appear in these 3 categories at the same time. For example, an image includes 3 faces, namely Zhang San, Li Si and Wang Wu. Among them, it is assumed that the first-level face of this image is Zhang San, and the face category corresponding to the obtained first-level face is Zhang San. Assume that the second-level faces except the first-level faces are also recognized because their clarity meets the preset conditions, and they are classified into the corresponding face categories, which are the classification of Li Si and Wang Wu. . Then after the image is classified, it will eventually be displayed in the Zhang San atlas, the Li Si atlas, and of course in the Wang Wu atlas. Because the faces corresponding to these three identities all meet certain conditions and undergo face recognition, they are classified into the corresponding face categories.

In this embodiment, first, a first-level face that satisfies a preset condition is obtained from an image according to the size of the face area, and face recognition is performed on the first-level face to obtain the person corresponding to the first-level face. face category. Then obtain the second-level face in the image, perform face recognition on the second-level face, and obtain the face category corresponding to the second-level face, and the second-level face is the first-level face in the image. outside face. Finally, according to the face category corresponding to the first-level face and the face category corresponding to the second-level face, the images are classified into corresponding face categories. The faces in the image are extracted in stages, and the first-level faces are extracted in descending order of face area, and the extracted first-level faces need to meet preset conditions, so , the conditions for extracting the first-level face are very strict, and face recognition is performed on the first-level face to obtain the face category corresponding to the first-level face. Then continue to obtain the second-level face from the image, and then perform face recognition on the second-level face to obtain the face category corresponding to the second-level face. The face categories obtained in this way will be more accurate, and there will not be a large number of unimportant face categories, which will improve the effectiveness of the classification results and better meet the actual needs of users. Finally, the images are classified into corresponding face categories.

In one embodiment, as shown in FIG. 3 , the first-level face that satisfies a preset condition is obtained from the image according to the size of the face area, including:

Step 302: Obtain the face with the largest face area from the multi-person group photo image, and determine whether the face meets the preset condition.

Obtain the face with the largest face area from the multi-person group photo image, and then determine whether the face meets the preset conditions. The preset conditions include but are not limited to the following: determine the shooting angle and focus of the face with the largest face area, determine whether the shooting angle is a frontal face, determine whether the shooting focus is the face with the largest face area, and determine whether the face has the largest face area. whether the face is closest to the camera.

Step 304, if yes, take the face with the largest face area as the first-level face of the group photo image.

Step 306, if not, then continue to sequentially obtain the face with the second largest face area from the group photo image, determine whether the face meets the preset conditions, and repeat the execution until the face reaches the minimum threshold or the first-level person is obtained. until the face.

If the judgment result is that the above preset conditions are met at the same time, it means that the face with the largest face area should be the face corresponding to the core character, so the face with the largest face area is used as the first-level face of the multi-person group photo image. And mark it as the first-level face.

If the judgment result is that the above preset conditions are not met, it means that the face with the largest face area is not the face corresponding to the core character, that is, the first-level face, and then continue to obtain the face area of the second group from the group photo image. face, to judge whether the face meets the preset conditions, if the judgment result is that the above preset conditions are met at the same time, it means that the face with the second face area should be the first-level face, so the person with the second face area The face is used as the first-level face of the multi-person group photo image, and is marked as the first-level face.

If the face with the second face area does not meet the preset condition, it means that the face with the second face area is not the first-level face. Continue to obtain the face with the next face area from the multi-person group photo image, and repeat the execution until the face reaches the minimum threshold or the first-level face is obtained. That is, a minimum threshold of face area is preset. If there is no face that meets the preset conditions among the faces whose face area is larger than the minimum threshold, then stop acquiring the first-level face for this image. .

In the embodiment of the present application, since it is more difficult to determine the first-level face from the multi-person group photo image compared with the single-person image, it is determined whether the first-level face can be used as the first-level face according to the face area in descending order. , so that the first-level face is not missed. And preset conditions are set, and only the faces that meet these preset conditions can become the first-level faces, so that the obtained results of the first-level faces are more accurate.

In one embodiment, step 302 obtains the face with the largest face area from the multi-person group photo image, and determines whether the face meets the preset condition, including: obtaining the face with the largest face area from the multi-person group photo image, determining Whether the angle and focus of the face meet the preset conditions.

In the embodiment of the present application, the face with the largest face area is obtained from the group photo image of multiple people, and then it is determined whether the face meets the preset condition. The preset conditions include but are not limited to the following: determine the shooting angle and focus of the face with the largest face area, determine whether the shooting angle is a frontal face, determine whether the shooting focus is the face with the largest face area, and determine whether the face has the largest face area. whether the face is closest to the camera. Only the faces that meet these preset conditions can become the first-level faces, so that the obtained results of the first-level faces are more accurate.

In one embodiment, as shown in FIG. 4 , step 204 obtains the second-level face in the image, performs face recognition on the second-level face, and obtains the face category corresponding to the second-level face, including:

Step 402, acquiring the second-level face in the image.

Because the second-level face is the face except the first-level face in the image, after obtaining the first-level face of each image from the multi-person group photo image, the first-level face in each image is obtained. The face other than the human face means that the second-level face in the image is obtained.

Step 404, respectively calculating the sharpness of the second-level face in the image.

Step 406, judging whether the clarity of the second-level face meets a preset condition.

Step 408: If yes, perform face recognition on the second-level face that meets the preset definition condition, and obtain the face category corresponding to the second-level face.

Calculate the sharpness of the second-level faces obtained from the multi-person group photo images, respectively, and determine whether the sharpness of the second-level faces satisfies a preset condition. The preset condition may be a threshold of clarity that can identify the identity corresponding to the face. For example, for a second-level face, the identity of the person corresponding to the face can be identified, that is, the clarity meets the preset condition. Of course, the preset condition may also be other specific sharpness thresholds, and if the sharpness threshold exceeds the sharpness threshold, the sharpness satisfies the preset condition. For a single-person image, it is not necessary to extract the first-level face. The face in the single-person image can be regarded as the second-level face, and the method of processing the second-level face is used for processing. Therefore, the single-person face is directly processed first. Determine the sharpness of the faces in the image.

Determine whether the clarity of the second-level face satisfies the preset condition, and if the preset condition is met, perform face recognition on the second-level face that meets the preset clarity condition, and obtain the corresponding second-level face face category.

In the embodiment of the present application, the condition of clarity is used for the second-level face to determine whether face recognition and classification are required. Because the condition of clarity is used to screen the second-level faces, only the second-level faces whose clarity meets the preset conditions will be recognized and classified, so the screening conditions are relatively strict, and the screened faces There are fewer categories, thus greatly reducing the face classification of secondary faces, improving the validity of the classification results, and more in line with the actual needs of users.

In one embodiment, as shown in Figure 5, the method further includes:

Step 410, when it is determined that the clarity of the second-level face does not meet the preset condition, search for a first-level face similar to the second-level face in other images in the album.

If the definition of the second-level face in the image does not meet the preset condition, continue to search for the first-level face similar to the second-level face in other images in the album. For example, a first-level face extracted from a single-person image or a group photo image of multiple people, if the first-level face is similar to the second-level face in the image, the second-level person in the image can be The face is classified into the face category corresponding to the first-level face. The concept of similarity can be defined as the similarity of facial features reaching a set threshold. For example, when the similarity of facial features reaches 80%, it is defined as similar. Of course, other reasonable values can also be set.

Step 412: If a first-level face similar to the second-level face is found, classify the second-level face into a face category corresponding to the first-level face.

If a first-level face similar to the second-level face is found in other images in the album, for example, a first-level face is found, and the second-level face is similar to the first-level face If the degree reaches 90%, then the first-level face that is similar to the second-level face is found, and the second-level face is classified into the face category corresponding to the first-level face.

In this embodiment, the second-level face that can find the first-level face similar to the second-level face in other images in the album is classified into the category corresponding to the first-level face . The first-level face is used as the classification standard, so that there will not be too many other face classifications, which helps to improve the accuracy of face classification for images.

In one embodiment, as shown in FIG. 6 , after searching for a first-level face similar to a second-level face in other images in the album, it includes:

Step 602: If the first-level face similar to the second-level face is not found, search for a face similar to the second-level face in other images in the album.

If no first-level face similar to the second-level face is found in other images in the album, look for a face similar to the second-level face in other images in the album. The face may not be First class face. The concept of similarity can be defined as the similarity of facial features reaching a set threshold. For example, when the similarity of facial features reaches 80%, it is defined as similar. Of course, other reasonable values can also be set. If all the faces in the other images in the album are searched for a face whose similarity with the second-level face reaches 80%.

Step 604: Count the appearance times of faces similar to the second-level faces.

Count the number of occurrences of finding a face that is 80% similar to the second-level face from all the faces in other images in the album.

Step 606, if the number of times reaches the set threshold, perform face recognition on the second-level face, and obtain the face category corresponding to the second-level face.

If the counted number of occurrences of similar faces reaches the set threshold, select a clearest face from these similar faces, and perform face recognition on the clearest face to obtain the clearest face. The corresponding identity, the identity is the face category corresponding to this type of similar face, that is, the face category corresponding to the second-level face. The preset number of times can be set to 5 times, of course, in other embodiments, it can also be set to other reasonable times, such as 3 times, 4 times, 6 times, 10 times and so on.

Step 608, if the number of times does not reach the set threshold, identify the second-level face as the third-level face.

If the counted number of appearances of similar faces does not reach the set threshold, the second-level face is temporarily identified as a third-level face. The third-level face includes all faces that are not classified into face categories.

In this embodiment, the second-level face that is similar to the first-level face and the second-level face cannot be found in other images in the album, but similar faces can be found in the album, and they are similar For the second-level face whose number of times of face appearance reaches the set threshold, the clearest face is extracted from these similar faces for face recognition and the face category is obtained. The face category is also obtained for the second-level face that meets the above conditions, and the second-level face that does not meet the above two conditions for face classification is classified into the third-level face for preparation. Subsequently, the third-level face is classified again. Through layer-by-layer screening, all eligible second-level faces are classified, which ensures the accuracy and completeness of the classification results.

In one embodiment, as shown in FIG. 7 , after identifying the second-level human face as the third-level human face, it includes:

Step 702: Calculate the number of times that the third-level face in the album and the first-level face belonging to different face categories appear on the same image.

After the above classification, the third-level faces are all faces that are not classified into the face category. In all the images in the album, the number of times that each third-level face and the first-level face belonging to different face categories appear on the same image are calculated respectively, that is, the third-level face with the same identity and the different The number of times the first-level faces of the identity were grouped together. For example, to count the number of times the third-level face with identity A took photos with first-level faces of different identities, specifically, assuming that there are 10 first-level faces in the album, including Zhang San, Li Si and Wang Wu, etc. First class face. Then count the number of times the third-level face with identity A takes a photo with any one of these 10 first-level faces. There is an image in the album that is a group photo of A with Zhang San, Li Si, and Wang Wusi, so the number of times that the third-level face with identity A and the first-level face of different identities are taken together is 3 times. If there is another image in the album that is a group photo of A and Zhang San, and another image is a group photo of A and Wang Wu. Then, at this time, the number of times the third-level face with identity A and the first-level face with different identities take photos together is 5 times.

Step 704, judging whether the number of times reaches the set threshold.

To set the threshold, it can be set that the number of times that the third-level face in the album and the first-level face belonging to different face categories appear on the same image is at least 5 times. Of course, in other embodiments, it can also be Set to other reasonable times, such as 3 times, 4 times, 6 times, 7 times, 8 times, 9 times, 10 times, etc.

Step 706, if yes, perform face recognition on the third-level face to obtain the face category corresponding to the third-level face.

Step 708, if not, keep the third-level face.

Determine whether the number of times the third-level face and the first-level face belonging to different face categories appear on the same image has reached a set threshold (for example, 5 times), and if it reaches 5 times, the third-level face Perform face recognition to obtain the face category corresponding to the third-level face.

In this embodiment, if the number of times that the third-level face with identity A and the first-level face with different identities take a group photo together exceeds the set threshold, face recognition will be performed on the third-level face with identity A, Get the face category corresponding to the third-level face. If it does not reach 5 times, it will remain as a suspected passerby's face. Wait for the next time when an image is added to the album, and then recalculate the number of times that the third-level face appears on the same image as the first-level face belonging to a different face category. In this way, the third-level face is classified, so that the third-level face also has the opportunity to perform face classification to obtain the corresponding face category. This effectively avoids face recognition only on the first-level face and the second-level face in each image, ensures the integrity and integrity of the scheme, and improves the accuracy of the final image classification result. sex, effectiveness.

In one embodiment, an image classification method is also provided, which is described by taking the method applied to the electronic device in FIG. 1A as an example, specifically:

(1) Determine how many faces are in the image to be classified.

(2) If there is only one face in the picture, skip to step (3) to start execution. If there are multiple faces in the image, that is, a group photo image, then obtain the face area from the multi-person group photo image. For the largest face, it is judged whether the angle and focus of the face meet the preset conditions; If not, continue to obtain the second-largest face from the multi-person group photo image, determine whether the face meets the preset conditions, and repeat the execution until the face area reaches the minimum threshold or the first-level face is obtained.

(3) Obtain the second-level face in the image; calculate the clarity of the second-level face in the image respectively; judge whether the clarity of the second-level face meets the preset condition; Perform face recognition on the conditional second-level face to obtain the face category corresponding to the second-level face.

(4) When it is judged that the clarity of the second-level face does not meet the preset conditions, search for a first-level face similar to the second-level face in other images in the album; If the first-level faces are similar to the first-level faces, the second-level faces are classified into the face categories corresponding to the first-level faces.

(5) If the first-level face similar to the second-level face is not found, search for a face similar to the second-level face in other images in the album; calculate the face similar to the second-level face The number of face appearances; if the number of times reaches the set threshold, face recognition is performed on the second-level face, and the face category corresponding to the second-level face is obtained; if the number of times does not reach the set threshold, the The second-level face is identified as the third-level face.

(6) Calculate the number of times that the third-level face in the album and the first-level face belonging to different face categories appear on the same image; determine whether the number of times reaches the set threshold; Face recognition, get the face category corresponding to the third-level face; if not, keep it as the third-level face.

(7) According to the face category corresponding to the first-level face, the face category corresponding to the second-level face, and the face category corresponding to the third-level face, the images are classified into corresponding face categories .

In one embodiment, as shown in FIG. 8 , an image classification apparatus 800 is provided. The apparatus includes: a first-level face recognition module 802 , a second-level face recognition module 804 and an image classification module 806 . in,

The first-level face recognition module 802 is used to obtain the first-level face that meets the preset conditions from the image according to the size of the face area, perform face recognition on the first-level face, and obtain the information of the first-level face. corresponding face category.

The second-level face recognition module 804 is used to obtain the second-level face in the image, perform face recognition on the second-level face, and obtain the face category corresponding to the second-level face, the second-level face is the face except the first-level face in the image.

The image classification module 806 is configured to classify the images into corresponding face categories according to the face categories corresponding to the first-level faces and the face categories corresponding to the second-level faces.

In one embodiment, as shown in FIG. 9, the first-level face recognition module 802 includes:

The first-level face judging module 802a of the multi-person group photo image is used to obtain the face with the largest face area from the multi-person group photo image, and determine whether the face meets the preset condition.

The first-level face determination module 802b of the group photo image is configured to use the face with the largest face area as the first-level face of the group photo image if the face with the largest face area meets the preset condition.

The loop module 802c is used for if the face with the largest face area does not meet the preset condition, then continue to obtain the face with the second largest face area from the multi-person group photo image, determine whether the face meets the preset condition, and repeat the execution Until the face area reaches the minimum threshold or the first-level face is obtained.

In one embodiment, the first-level face judging module 802a of the group photo image is further configured to obtain the face with the largest face area from the group photo image, and determine whether the angle and focus of the face meet preset conditions .

In one embodiment, as shown in FIG. 10 , the second-level face recognition module 804 includes:

The second-level face acquisition module 8041 is used to acquire the second-level human face in the image.

The sharpness calculation module 8042 is used to calculate the sharpness of the second-level human face in the image respectively.

The judgment module 8043 is used for judging whether the clarity of the second-level face satisfies the preset condition.

The face recognition module 8044 is configured to perform face recognition on the second-level face that satisfies the preset clarity condition if the clarity of the second-level face meets the preset condition, and obtain the corresponding second-level face face category.

In one embodiment, as shown in FIG. 11 , the second-level face recognition module 804 further includes:

A similar first-level face finding module 8045 is used to search for a first-level face similar to the second-level face in other images in the album when it is judged that the clarity of the second-level face does not meet the preset condition human face.

The second-level face classification module 8046 is configured to classify the second-level face into a face category corresponding to the first-level face if a first-level face similar to the second-level face is found.

In one embodiment, the second-level face recognition module is further configured to: if the first-level face similar to the second-level face is not found, search for the second-level face in other images in the album Similar faces; count the number of appearances of faces similar to the second-level face; if the number of times reaches the set threshold, perform face recognition on the second-level face to obtain the person corresponding to the second-level face face category; if the number of times does not reach the set threshold, the second-level face is identified as the third-level face.

In one embodiment, the second-level face recognition module is also used to: calculate the number of times that the third-level face in the album and the first-level face belonging to different face categories appear on the same image; determine whether the number of times reaches Set the threshold; if yes, perform face recognition on the third-level face to obtain the face category corresponding to the third-level face; if the number of times does not reach the set threshold, it will remain as the third-level face.

The division of each module in the above image classification apparatus is only for illustration. In other embodiments, the image classification apparatus may be divided into different modules as required to complete all or part of the functions of the above image classification apparatus.

A computer program product containing instructions, when run on a computer, causes the computer to perform the image classification method described above.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of the image classification methods provided in the foregoing embodiments.

The embodiments of the present application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the image classification methods provided by the above embodiments when the computer program is executed. A step of.

The embodiments of the present application also provide an electronic device. As shown in FIG. 12 , for the convenience of description, only the parts related to the embodiments of the present application are shown, and the specific technical details are not disclosed, please refer to the method part of the embodiments of the present application. The electronic device can be any terminal device including a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales, a sales terminal), a vehicle-mounted computer, a wearable device, etc. The electronic device is a mobile phone as an example :

FIG. 12 is a block diagram of a partial structure of a mobile phone related to an electronic device provided by an embodiment of the present application. Referring to FIG. 12 , the mobile phone includes: a radio frequency (RF) circuit 910 , a memory 920 , an input unit 930 , a display unit 940 , a sensor 950 , an audio circuit 960 , a wireless fidelity (WiFi) module 970 , and a processor 980 , and power supply 990 and other components. Those skilled in the art can understand that the structure of the mobile phone shown in FIG. 12 does not constitute a limitation on the mobile phone, and may include more or less components than shown, or combine some components, or arrange different components.

The RF circuit 910 can be used for receiving and sending signals during sending and receiving of information or during a call. After receiving the downlink information of the base station, it can be processed by the processor 980; it can also send the uplink data to the base station. Typically, the RF circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, RF circuitry 910 may communicate with networks and other devices via wireless communications. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access) Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE)), email, Short Messaging Service (Short Messaging Service, SMS) and the like.

The memory 920 can be used to store software programs and modules, and the processor 980 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 920 . The memory 920 may include a program storage area and a data storage area at the first level, wherein the program storage area may store an operating system, an application program required for at least one function (such as an application program for a sound playback function, an application program for an image playback function, etc. ), etc.; the data storage area can store data (such as audio data, address book, etc.) created according to the use of the mobile phone. Additionally, memory 920 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The input unit 930 may be used to receive input numerical or character information, and generate key signal input related to user settings and function control of the mobile phone 900 . Specifically, the input unit 930 may include a touch panel 931 and other input devices 932 . The touch panel 931, also referred to as a touch screen, can collect the user's touch operations on or near it (such as the user using a finger, a stylus, etc., any suitable object or accessory on or near the touch panel 931) operation), and drive the corresponding connection device according to the preset program. In one embodiment, the touch panel 931 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the touch controller. To the processor 980, and can receive the command sent by the processor 980 and execute it. In addition, the touch panel 931 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 931 , the input unit 930 may further include other input devices 932 . Specifically, other input devices 932 may include, but are not limited to, one or more of physical keyboards, function keys (such as volume control keys, switch keys, etc.), and the like.

The display unit 940 may be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 940 may include a display panel 941 . In one embodiment, the display panel 941 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. In one embodiment, the touch panel 931 may cover the display panel 941, and when the touch panel 931 detects a touch operation on or near it, the touch panel 931 transmits it to the processor 980 to determine the type of the touch event, and then the processor 980 determines the type of the touch event according to the The type of touch event provides a corresponding visual output on display panel 941 . Although in FIG. 12, the touch panel 931 and the display panel 941 are used as two independent components to realize the input and input functions of the mobile phone, in some embodiments, the touch panel 931 and the display panel 941 can be integrated to form a Realize the input and output functions of the mobile phone.

Cell phone 900 may also include at least one sensor 950, such as light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 941 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 941 and/or when the mobile phone is moved to the ear. or backlight. The motion sensor can include an acceleration sensor. The acceleration sensor can detect the magnitude of acceleration in all directions, and can detect the magnitude and direction of gravity when it is stationary. It can be used for applications that recognize the posture of the mobile phone (such as switching between horizontal and vertical screens), and vibration recognition related functions (such as Pedometer, tapping), etc.; in addition, the mobile phone can also be equipped with other sensors such as gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc.

Audio circuit 960, speaker 961 and microphone 962 may provide an audio interface between the user and the cell phone. The audio circuit 960 can transmit the received audio data converted electrical signal to the speaker 961, and the speaker 961 converts it into a sound signal for output; on the other hand, the microphone 962 converts the collected sound signal into an electrical signal, which is converted by the audio circuit 960 After receiving, the audio data is converted into audio data, and then the audio data is output to the processor 980 for processing, and can be sent to another mobile phone via the RF circuit 910, or the audio data can be output to the memory 920 for subsequent processing.

WiFi is a short-distance wireless transmission technology. The mobile phone can help users to send and receive emails, browse web pages, and access streaming media through the WiFi module 970. It provides users with wireless broadband Internet access. Although FIG. 12 shows the WiFi module 970, it can be understood that it is not a necessary component of the mobile phone 900 and can be omitted as required.

The processor 980 is the control center of the mobile phone, using various interfaces and lines to connect various parts of the entire mobile phone, by running or executing the software programs and/or modules stored in the memory 920, and calling the data stored in the memory 920. Various functions of the mobile phone and processing data, so as to monitor the mobile phone as a whole. In one embodiment, the processor 980 may include one or more processing units. In one embodiment, the processor 980 may integrate an application processor and a modem processor, wherein the first stage of the application processor is to process the operating system, user interface and application programs, etc.; the first stage of the modem processor is to Handle wireless communications. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 980.

The mobile phone 900 also includes a power supply 990 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 980 through a power management system, so as to manage charging, discharging, and power consumption management functions through the power management system.

In one embodiment, the mobile phone 900 may further include a camera, a Bluetooth module, and the like.

Any reference to a memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Suitable nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM).

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims (10)

1. An image classification method, comprising:

acquiring a face with the largest face area from a multi-person synthetic image, and judging whether the face meets a preset condition or not; if so, taking the face with the largest face area as a first-level face of the multi-person group photo image; if not, continuously acquiring the face with the face area of the second order from the multi-person group photo image, judging whether the face meets the preset condition, and repeatedly executing until the face area reaches the minimum threshold or the first-level face is acquired; acquiring a face from a single image as a first-level face;

carrying out face recognition on the first-level face to obtain a face category corresponding to the first-level face;

for a multi-person group photo image, acquiring a second-level face meeting a definition preset condition in the image, and performing face recognition on the second-level face to obtain a face type corresponding to the second-level face, wherein the second-level face is a face except for a first-level face in the image;

and dividing the image into corresponding face categories according to the face category corresponding to the first-level face and the face category corresponding to the second-level face.

2. The method according to claim 1, wherein the obtaining a face with a largest face area from the multi-person photographic image and determining whether the face meets a preset condition comprises:

and acquiring the face with the largest face area from the multi-person group photo image, and judging whether the angle and the focus of the face meet preset conditions.

3. The method according to claim 1, wherein for the multi-person group photo image, acquiring a second-level face meeting a preset definition condition in the image, and performing face recognition on the second-level face to obtain a face class corresponding to the second-level face, comprises:

acquiring a second-level face in the image;

respectively calculating the definition of the second-level face in the image;

judging whether the definition of the second-level face meets a preset condition or not;

and if so, carrying out face recognition on the second-level face meeting the definition preset condition to obtain the face type corresponding to the second-level face.

4. The method of claim 3, further comprising:

when the definition of the second-level face does not meet the preset condition, searching a first-level face similar to the second-level face in other images in the album;

and if a first-level face similar to the second-level face is found, dividing the second-level face into the face type corresponding to the first-level face.

5. The method of claim 4, wherein after finding the first level face similar to the second level face in the other images in the album, comprising:

if the first-level face similar to the second-level face is not found, searching faces similar to the second-level face in other images in the album;

calculating the occurrence times of the human faces similar to the second-level human faces;

if the times reach a set threshold value, carrying out face recognition on the second-level face to obtain a face type corresponding to the second-level face;

and if the times do not reach a set threshold value, identifying the second-level face as a third-level face.

6. The method of claim 5, wherein after said identifying said second level face as a third level face, comprising:

calculating the times of the third-level face and the first-level faces belonging to different face categories appearing on the same image in the photo album;

judging whether the times reach a set threshold value or not;

if so, carrying out face recognition on the third-level face to obtain a face class corresponding to the third-level face;

and if not, keeping the face as the third-level face.

7. The method according to claim 1, wherein the obtaining a face with a largest face area from the multi-person photographic image and determining whether the face meets a preset condition comprises:

and acquiring the face with the largest face area from the multi-person group photo image, and judging whether the face with the largest face area is closest to the lens.

8. An image classification apparatus, characterized in that the apparatus comprises:

the first-stage face recognition module is used for acquiring a face with the largest face area from the multi-person synthetic image and judging whether the face meets a preset condition or not; if so, taking the face with the largest face area as a first-level face of the multi-person group photo image; if not, continuously acquiring the face with the face area of the second order from the multi-person group photo image, judging whether the face meets the preset condition, and repeatedly executing until the face area reaches the minimum threshold or the first-level face is acquired; acquiring a face from a single image as a first-level face; acquiring a first-level face meeting preset conditions from an image according to the sequence of the face areas from large to small, and performing face recognition on the first-level face to obtain a face type corresponding to the first-level face;

the second-level face recognition module is used for acquiring a second-level face meeting the definition preset condition in the image for the multi-person group photo image, and performing face recognition on the second-level face to obtain a face type corresponding to the second-level face, wherein the second-level face is a face except the first-level face in the image;

and the image classification module is used for classifying the image into corresponding face classes according to the face class corresponding to the first-level face and the face class corresponding to the second-level face.

9. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program, wherein the computer program, when executed by the processor, causes the processor to perform the steps of the image classification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the image classification method according to any one of claims 1 to 7.

Images