WO2019052316A1

WO2019052316A1 - Image processing method and apparatus, computer-readable storage medium and mobile terminal

Info

Publication number: WO2019052316A1
Application number: PCT/CN2018/101502
Authority: WO
Inventors: 柯秀华; 曹威; 王俊
Original assignee: Oppo广东移动通信有限公司
Priority date: 2017-09-15
Filing date: 2018-08-21
Publication date: 2019-03-21
Also published as: CN107679559B; CN107679559A

Abstract

The embodiments of the present application relate to an image processing method and apparatus, a computer-readable storage medium and a mobile terminal. The method comprises: when an image clustering request is received, acquiring a target human face image corresponding to the image clustering request; acquiring characteristic information about a first human face in the target human face image; and clustering the target human face image according to the characteristic information about the first human face. The received image clustering request comprises: when a characteristic recognition model is updated, receiving a first image clustering request; when first clustering information sent by a server is received, receiving a second image clustering request, wherein the first clustering information is the server's clustering information about a set of second human face images uploaded by a mobile terminal; and when it is detected that an image is newly added, receiving a third clustering request.

Description

Image processing method, device, computer readable storage medium, and mobile terminal

Cross-reference to related applications

The present application claims priority to Chinese Patent Application No. 201710850225.X, filed on Sep. 15, 2017, entitled "Image Processing Method, Apparatus, Computer Readable Storage Medium, and Mobile Terminal", which The entire contents are incorporated herein by reference.

Technical field

The present application relates to the field of computer technology, and in particular, to an image processing method, apparatus, computer readable storage medium, and mobile terminal.

Background technique

With the rapid development of intelligent mobile terminals, the functions of intelligent mobile terminals are more and more complete, and the performance in smart mobile is more and more perfect. After the user takes a picture with the smart mobile terminal, the smart mobile terminal can upload the image captured by the user to the server, so that the server classifies according to the image information. The images are grouped according to the image time information, or according to the image location information, or according to the face information contained in the image, so that the user can view the images in different categories.

Summary of the invention

The embodiment of the present application provides an image processing method, apparatus, computer readable storage medium, and mobile terminal, which can cluster images according to facial feature information.

An image processing method comprising:

When the image clustering request is received, acquiring a target face image corresponding to the image clustering request;

Obtaining first facial feature information in the target facial image;

And clustering the target face image according to the first face feature information;

The receiving the image clustering request includes:

Receiving a first image clustering request when the feature recognition model is updated;

Receiving, by the first cluster information sent by the server, a second image clustering request, where the first clustering information is clustering information of a second facial image set uploaded by the server to the mobile terminal;

When a new image is detected, a third clustering request is received.

An image processing apparatus comprising:

a first acquiring module, configured to: when receiving an image clustering request, acquire a target facial image corresponding to the image clustering request;

a second acquiring module, configured to acquire first facial feature information in the target facial image;

a clustering module, configured to cluster the target facial image according to the first facial feature information;

The receiving the image clustering request includes:

When a new image is detected, a third clustering request is received.

A computer readable storage medium having stored thereon a computer program that, when executed by a processor, implements the operations of the methods described above.

A mobile terminal comprising a memory and a processor, the memory storing computer readable instructions that, when executed by the processor, cause the processor to operate as described above.

The image processing method, the device, the computer readable storage medium, and the mobile terminal acquire a corresponding target face image according to the image clustering request when receiving different image clustering requests, and implement different image clustering requests to obtain different images. The target face image does not acquire all images in the mobile terminal, saves mobile terminal resources, and reduces mobile terminal resource consumption.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present application, and other drawings can be obtained according to the drawings without any creative work for those skilled in the art.

1 is a schematic diagram of an application environment of an image processing method in an embodiment;

2 is a sequence diagram of interaction between the mobile terminal 110 of FIG. 1 and the first server 120 and the second server 130 in an embodiment;

3 is a flow chart of an image processing method in an embodiment;

4 is a block diagram showing the structure of an image processing apparatus in an embodiment;

Figure 5 is a block diagram showing the structure of an image processing apparatus in another embodiment;

FIG. 6 is a block diagram showing a partial structure of a mobile phone related to a mobile terminal provided by an embodiment of the present application.

Detailed ways

In order to make the objects, technical solutions, and advantages of the present application more comprehensible, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the application and are not intended to be limiting.

It will be understood that the terms "first", "second" and the like, as used herein, may be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, the first acquisition module may be referred to as a second acquisition module without departing from the scope of the present application, and similarly, the second acquisition module may be referred to as a first acquisition module. Both the first acquisition module and the second acquisition module are acquisition modules, but they are not the same acquisition module.

FIG. 1 is a schematic diagram of an application environment of an image processing method in an embodiment. As shown in FIG. 1, the application environment includes a mobile terminal 110, a first server 120, and a second server 130. Images can be stored in the memory of the mobile terminal 110 and the SD (Secure Digital Memory Card) card. The mobile terminal 110 can perform face recognition on the image and extract a face image included in the stored image. The mobile terminal 110 may extract face feature information in the face image according to the feature recognition model, and perform similarity matching on the extracted face feature information, thereby clustering the face image. The mobile terminal 110 may also upload the face image to the first server 120. The first server 120 extracts the face feature information in the face image according to the feature recognition model, and uploads the extracted face feature information to the second server 130, the second server. 130 may cluster the facial feature information uploaded by the first server 120. The second server 130 may return the result of clustering the facial feature information to the mobile terminal 110, so that the mobile terminal 110 clusters the clustering result of the second server 130 with the face image and the clustering result of the mobile terminal 110 with the face image. Compare. The feature recognition models of the mobile terminal 110 and the second server 130 may be the same or different.

In one embodiment, the first server 120 and the second server 130 may be the same server, ie the mobile terminal 110 will recognize the face image upload server. The server performs face feature recognition on the received face image to obtain face feature information, and the server clusters the face feature information, and sends the clustering result to the mobile terminal 110.

FIG. 2 is a sequence diagram of interaction between the mobile terminal 110 of FIG. 1 and the first server 120 and the second server 130 in one embodiment. As shown in FIG. 2, the process of interacting with the first server 120 and the second server 130 by the mobile terminal 110 mainly includes the following operations:

(1) The mobile terminal 110 detects the feature recognition model update and clusters the face image set in the mobile terminal 110.

When the mobile terminal 110 detects the feature recognition model update in the mobile terminal 110, the face image stored in the mobile terminal 110 may be acquired, and the updated feature recognition model is used to extract the face feature information in the face image, and the face feature information is performed. Clustering.

(2) The mobile terminal 110 detects that the mobile terminal 110 adds an image and clusters the newly added image.

When the mobile terminal 110 detects that there is a new image, and the newly added image includes a human face, the face feature information in the newly added image is extracted, and the face feature information of the newly added image and the person who has clustered the face image are added. Face feature information is matched to cluster new images.

(3) The mobile terminal 110 uploads the face image to the first server 120.

The mobile terminal 110 may upload the face image included in the image stored by the mobile terminal 110 to the first server 120. The mobile terminal 110 may upload the face image included in the memory storage image to the first server 120, and the mobile terminal 110 may also upload the face image included in the SD card storage image to the first server 120, and the mobile terminal 110 also The memory image and the face image included in the SD card storage image may be uploaded to the first server 120.

(4) The first server 120 extracts face feature information in the face image.

After receiving the face image uploaded by the mobile terminal 110, the first server 120 may extract the face feature information from the face image according to the feature recognition model. The face feature recognition model in the mobile terminal 110 and the face feature recognition model in the server may be the same or different.

(5) The first server 120 transmits the face feature information to the second server 130.

The first server 120 sends the acquired facial feature information to the second server 130, so that the second server 130 can perform clustering according to the facial feature information.

(6) The mobile terminal 110 transmits a clustering request to the second server 130.

After the mobile terminal 110 uploads the face image, the clustering request may be sent to the second server 130.

(7) The second server 130 clusters the face feature information.

If the second server 130 receives the facial feature information sent by the first server 120, and the second server 130 receives the clustering request sent by the mobile terminal 110, the second server 130 may cluster the facial feature information. The clustering of the facial feature information by the second server 130 includes: matching the facial feature information to the similarity degree, and if the similarity exceeds the specified value, dividing the facial feature information into a group. The algorithm for clustering the face feature information by the mobile terminal 110 and the algorithm for clustering the face feature information by the second server 130 may be the same or different.

(8) The second server 130 returns the clustering result to the mobile terminal 110.

After the second server 130 clusters the facial feature information, the clustering result may be sent to the mobile terminal 110.

(9) The mobile terminal 110 updates the cluster information of the image in the mobile terminal 110 according to the clustering result sent by the second server 130, and clusters the un-uploaded face image.

After receiving the clustering result sent by the second server 130, the mobile terminal 110 may cluster the face image uploaded by the first server 120 according to the clustering result. The mobile terminal 110 may also acquire facial feature information of the un-uploaded face image, and cluster the un-uploaded face image.

In the embodiment of the present application, the mobile terminal and the server can perform feature recognition on the face image, acquire face feature information, and perform face clustering on the face image according to the face feature information. 3 is a flow chart of an image processing method in one embodiment. As shown in FIG. 3, an image processing method is applied to a mobile terminal, including:

Operation 302, when receiving an image clustering request, acquiring a target face image corresponding to the image clustering request.

After receiving the image clustering request, the mobile terminal may acquire the target face image corresponding to the image clustering request. The receiving, by the mobile terminal, the image clustering request includes:

(1) When the feature recognition model is updated, the first image clustering request is received.

(2) receiving the second clustering request when the first clustering information sent by the server is received, where the first clustering information is clustering information of the second facial image set uploaded by the server to the mobile terminal.

(3) When a new image is detected, a third clustering request is received.

The first image clustering request is triggered when the mobile terminal detects the feature recognition model update; the mobile terminal triggers the second image clustering request when receiving the image clustering information sent by the server; the mobile terminal detects the new image The third image clustering request is triggered. The feature recognition model can be used to extract face feature information from the face image, and the mobile terminal can cluster the face image according to the extracted face feature information. When the feature recognition model is updated, the server can send a new version of feature recognition to the mobile terminal. The mobile terminal installs a new version of the feature recognition model according to the server The new version of the feature recognition model is issued for feature recognition model update. After receiving the face image uploaded by the mobile terminal, the server performs feature recognition on the face image according to the feature recognition model, acquires face feature information, and then clusters the face feature information, and the image clustering result sent by the server to the mobile terminal That is, the first cluster information delivered by the server. The feature recognition model of the mobile terminal and the feature recognition model of the server may be the same or different. The face image of the mobile terminal upload server is a face image stored in the memory of the mobile terminal. The image clustering information includes an image ID (Identification, ID card) and an image grouping mark. After receiving the image clustering information, the mobile terminal searches for an image according to the image ID, and divides the image into corresponding groups according to the image grouping mark.

After receiving the image clustering request, the mobile terminal may acquire the target face image corresponding to the image clustering request. The above target face image is the face image to be clustered.

When the feature recognition model is updated, that is, the algorithm update of the face feature information is extracted from the face image, the mobile terminal acquires all the images stored in the current mobile terminal, including the memory image and the SD card storage image. A face scan is performed on the memory image and the SD card storage image to identify the face image included in the memory image and the SD card storage image. The above-mentioned face scanning refers to recognizing a face from an image according to a face recognition algorithm, and acquiring a included face image. When the mobile terminal receives the first cluster information sent by the server, that is, when the mobile terminal receives the group information of the face image sent by the server, the mobile terminal acquires the SD card to store the image, and performs face scanning on the SD card stored image. Identify the face image contained in the SD card storage image. Before the server sends the image clustering information, the mobile terminal may upload the face image included in the memory image to the server, and the server performs feature recognition on the acquired face image according to the feature recognition model to obtain the facial feature in the face image. The information is used to cluster the face image according to the face feature information, and the image clustering information is sent to the mobile terminal. When the mobile terminal adds an image, the mobile terminal acquires a new image, performs face scanning on the newly added image, and recognizes the face image included in the newly added image.

When the feature recognition model is updated, the target face image is the first face image set in the mobile terminal, that is, the mobile terminal memory and the face image stored in the SD card. When receiving the first clustering request sent by the server, the target face image is a face image other than the second face image set in the first face image set. When a new image is detected, face recognition is performed on the newly added image. If the newly added image includes a face, the new image is added as the target face image.

Operation 304: Acquire first face feature information in the target face image.

After acquiring the target face image, the mobile terminal may identify the target face image feature according to the feature recognition model, and extract the first face feature information in the target face image. The face feature information refers to information for identifying a unique face, including contour features of a face, facial features, and the like.

Operation 306, clustering the target face image according to the first face feature information.

After acquiring the first facial feature information of the target facial image, the target facial image may be clustered according to the first facial feature information. When the feature recognition model is updated, the mobile terminal directly performs similarity matching on the first facial feature information of the first facial image set, and clusters the first facial feature information according to the matching result. After receiving the first cluster information sent by the server, the mobile terminal may perform similarity matching on the first face feature information of the target face image and the face feature information of the second face image set, according to the matching result. A face feature information is clustered. When the newly added image is detected, the mobile terminal may perform similarity matching between the first facial feature information of the target facial image and the facial feature information of the clustered facial image, and the first facial feature information according to the matching result. Perform clustering.

The image processing method in the embodiment of the present application, when receiving different image clustering requests, acquires a corresponding target face image according to the image clustering request, and implements different image clustering requests to acquire different target face images, which are not acquired. All images in the mobile terminal save mobile terminal resources and reduce mobile terminal resource consumption.

In an embodiment, the first image clustering request is received, and the acquiring the first facial feature information in the target facial image includes: performing facial recognition on the target facial image according to the updated feature recognition model, and acquiring the first facial face. Feature information; or, the stored face feature information of the target face image is converted into the first face feature information according to the feature information conversion model.

When the feature recognition model in the mobile terminal is updated, that is, the algorithm for extracting the face feature information of the face image is updated, the mobile terminal may acquire the set of the face image in the stored image as the target face image, that is, the memory image and the SD in the mobile terminal. The set of face images in the card image serves as the target face image.

The mobile terminal may perform face recognition on the target face image by using the updated feature recognition model to acquire first face feature information in the target face image. The face feature information extracted by the updated feature recognition model is inconsistent with the face feature information extracted by the original feature recognition model. After the updated feature recognition model is used to re-extract the first face feature information of the target face, the mobile terminal may The re-extracted first face feature information clusters the target face image.

In an embodiment, after the feature recognition model is updated in the mobile terminal, the mobile terminal may convert the stored face feature information of the target face image into the face feature information corresponding to the updated feature recognition model, that is, the target face image. The stored face feature information is converted into the first face feature information. The mobile terminal may re-cluster the target facial image according to the first facial feature information.

The image processing method in the embodiment of the present application re-extracts the face feature information of the face image after the feature recognition model is updated in the mobile terminal. After the feature recognition model is updated, the face images are clustered in time, which improves the timeliness of face image clustering.

In an embodiment, when the second image clustering request is received, acquiring the target facial image corresponding to the image clustering request includes: acquiring a facial image other than the second facial image set in the first facial image set as the target person a face image; performing clustering on the target face image according to the first face feature information includes: matching the first face feature information with the second face feature information in the second image set, if the similarity exceeds a preset value, Obtaining a first image corresponding to the second facial feature information, and dividing the target facial image into corresponding clustering packets of the first image in the first clustering information.

The mobile terminal can upload the face image in the memory image to the server, and the server can perform image clustering on the received face image, and send the clustering result to the mobile terminal. If the mobile terminal detects that the image clustering information is sent by the server, and the image set corresponding to the image clustering information sent by the server is not equal to the first facial image set, the mobile terminal second party is updated according to the image clustering information sent by the server. The clustering information of the face image set, and then acquiring the face image other than the second face image set in the first face image set, that is, the face image in the SD card and the memory person not included in the image clustering information sent by the server Face image. The mobile terminal can acquire the facial feature information of the target facial image, and perform similarity matching between the facial feature information of the target facial image and the facial feature information of the second facial image set. If the similarity is greater than a preset threshold, And dividing the target face image into a cluster group corresponding to the face feature information of the second face image set.

The image processing method in the embodiment of the present invention, after receiving the image clustering information sent by the server, clusters the face image of the un-uploaded server by using the image clustering information sent by the server as a standard, and improves the face of the face. The accuracy of image clustering.

In an embodiment, after receiving the image clustering information sent by the server, the mobile terminal updates the local image clustering information according to the image clustering information sent by the server. If the image clustering information sent by the server is inconsistent with the local image clustering information, detecting whether the image has a user operation flag in the local image clustering information, and if so, the mobile terminal saves the image. Local clustering information, and uploading local clustering information of the image to the server, so that the server overwrites the original image clustering information with the local clustering information of the image; if not, the mobile terminal overwrites the local with the image clustering information sent by the cloud Image clustering information.

In the image processing method of the embodiment of the present application, when the image clustering information of the mobile terminal is updated according to the image clustering information sent by the server, if the image clustering result of the same image is detected to be different, and the clustering result on the mobile terminal is When the user operates, the user operation is retained, and the clustering information of the user operation is uploaded to the server. The above method not only ensures multi-end data synchronization between the mobile terminal and the server, but also avoids data confusion, and retains user operations, thereby improving user stickiness.

In an embodiment, when the third image clustering request is received, acquiring the target facial image corresponding to the image clustering request includes: performing face recognition on the newly added image, and if the newly added image includes a human face, the person is included The new image of the face is used as the target face image.

When the mobile terminal detects that there is a new image, the face scan is performed on the newly added image to identify whether the newly added image is a face image, and if so, the face feature information in the newly added image is acquired; if not, the new image is not made. deal with. After obtaining the facial feature information in the newly added image, the facial feature information of the clustered image of the newly added image is compared with the similarity degree, and if the similarity exceeds the preset value, the new image is divided. Image clustering corresponding to the clustered image.

The information processing method in the embodiment of the present application compares the face feature information of the newly added image with the face feature information of the clustered image when detecting the newly added image of the mobile terminal, thereby ensuring the aggregation of the newly added image. Timeliness of the class.

In one embodiment, when the mobile terminal stores the image to be deleted, the mobile terminal detects whether the image has image clustering information, and if so, deletes the image clustering information corresponding to the image. For example, the first group Group1 in the mobile terminal includes an image 1.jpg, and the second group Group2 also includes an image 1.jpg. When the image 1.jpg is deleted from the mobile terminal, it corresponds to the first group Group1 and the second group Group. The image 1.jpg is deleted, that is, the image 1.jpg is no longer displayed in the first group Group1 and the second group Group2 of the mobile terminal album.

In an embodiment, before acquiring the target face image corresponding to the image clustering request, when at least one of the following conditions is met, the target face image corresponding to the image clustering request is acquired:

(1) The time difference between the current time and the time when the target face image was last acquired exceeds the preset time length.

(2) The current time is the preset time.

(3) The mobile terminal is in a charging state.

After receiving the image clustering request, the mobile terminal may detect whether the current condition meets the preset condition, and if the current condition is met, the target face image corresponding to the image clustering request is acquired; if the current condition is detected, When the preset condition is met, when the mobile terminal satisfies the preset condition, the image to be scanned is determined according to the type of the face scan.

In an embodiment, after receiving the image clustering request, the mobile terminal detects whether the current condition meets the preset condition. If not, the mobile terminal detects whether the current condition meets the preset condition according to the preset time interval.

In an embodiment, the mobile terminal detects whether the current condition meets the preset condition according to the preset time interval. If the mobile terminal detects that the mobile terminal meets the preset condition, and then detects whether the mobile terminal receives the image clustering request, if the mobile terminal receives the The image clustering request acquires a target face image corresponding to the image clustering request.

The foregoing preset condition includes: acquiring a time when the mobile terminal last acquired the target face image, and detecting a time difference between the current time and the last time the target face image is acquired exceeds a preset duration. For example, the preset duration is 48 hours, the current time is 10:18 on August 11, 2017, and the last time to get the target face image is 9:5 on August 8, 2017, the time difference is 73 hours and 13 minutes. If the preset time is longer than 48 hours, the mobile terminal satisfies the preset condition and acquires the target face image corresponding to the image clustering request. The mobile terminal may also detect whether the current time is a preset time. If the current time is the preset time, the target face image corresponding to the image clustering request is acquired. For example, if the preset time is 2:00AM to 5:00AM, and the mobile terminal detects that the current time is 3:15AM, the target face image corresponding to the image clustering request is acquired. When the mobile terminal is currently in a charging state, the mobile terminal may also acquire a target face image corresponding to the image clustering request.

The image processing method in the embodiment of the present application determines whether the mobile terminal satisfies a preset condition before acquiring the target face image corresponding to the image clustering request, and acquires the target face corresponding to the image clustering request when the preset condition is met. image. Since the mobile terminal takes a long time to cluster the image and occupies the CPU resources of the mobile terminal, the mobile terminal consumes power. Enabling mobile terminal image clustering when the mobile terminal is charging can avoid the situation that the mobile terminal consumes too much power. When the mobile terminal is at a preset time, the image clustering of the mobile terminal is started, which can avoid the situation that the mobile terminal image clustering consumes a large amount of CPU resources and causes the mobile terminal to be stuck. Image clustering is performed when the time difference from the last image clustering exceeds the specified duration, which ensures the timeliness of image clustering.

In an embodiment, the image processing method includes: when detecting the face information change in the target face image, reacquiring the first face feature information in the target face image, and targeting the target face according to the first face feature information. Images are clustered.

The mobile terminal can identify the face state of the face in the face image, and after receiving the image clustering information sent by the server, the mobile terminal can adjust the person in the face image in the mobile terminal according to the image clustering information sent by the server. Face status. When the mobile terminal detects that the face state changes in the target face image, for example, the face state changes from display to hidden, or changes from hidden to displayed, the mobile terminal may reacquire the first face feature information in the target face image. The target face image is clustered according to the re-acquired first face feature information.

The image processing method in the embodiment of the present application re-acquires the face feature information in the face image when the face information is changed in the face image, and re-clusters the face image according to the face feature information in the face image. It can realize the timeliness of clustering face images.

Fig. 4 is a block diagram showing the structure of an image processing apparatus in an embodiment. As shown in FIG. 4, an image processing apparatus includes:

The first obtaining module 402 is configured to acquire a target face image corresponding to the image clustering request when receiving the image clustering request.

The second obtaining module 404 is configured to acquire first facial feature information in the target facial image.

The clustering module 406 is configured to cluster the target facial image according to the first facial feature information.

Receiving the image clustering request includes: receiving the first image clustering request when the feature recognition model is updated; receiving the second image clustering request when receiving the first clustering information sent by the server, where the first clustering information is a server pair The clustering information of the second face image set uploaded by the mobile terminal; when the newly added image is detected, the third clustering request is received.

In an embodiment, when the first image clustering request is received, acquiring the first facial feature information in the target facial image includes: performing face recognition on the target facial image according to the updated feature recognition model, and acquiring the first person The face feature information; or, the stored face feature information of the target face image is converted into the first face feature information according to the feature information conversion model.

Fig. 5 is a block diagram showing the structure of an image processing apparatus in another embodiment. As shown in FIG. 5, an image processing apparatus includes a first acquisition module 502, a second acquisition module 504, a clustering module 506, and a processing module 508. The first obtaining module 502, the second obtaining module 504, and the clustering module 506 have the same functions as the corresponding modules.

The first obtaining module 502 is further configured to acquire second cluster information of the second face set of the mobile terminal.

The processing module 508 is configured to process the second cluster information according to the type of the comparison result when the comparison result of the first cluster information and the second cluster information is different.

In an embodiment, the first obtaining module 502 is further configured to acquire a target face image corresponding to the image clustering request if the time difference between the current time and the time when the target face image was last acquired exceeds a preset duration; or If the current time is the preset time, the target face image corresponding to the image clustering request is acquired; or the mobile terminal is in the charging state, and the target face image corresponding to the image clustering request is acquired.

In an embodiment, the second obtaining module 504 is further configured to re-acquire the first facial feature information in the target facial image when detecting the change of the facial information in the target facial image. The clustering module 506 is configured to cluster the target face image according to the first face feature information.

The embodiment of the present application also provides a computer readable storage medium. One or more non-transitory computer readable storage media containing computer executable instructions that, when executed by one or more processors, cause the processor to perform an image processing method as described above .

The embodiment of the present application further provides a mobile terminal. As shown in FIG. 6 , for the convenience of description, only the parts related to the embodiments of the present application are shown. If the specific technical details are not disclosed, please refer to the method part of the embodiment of the present application. The mobile terminal can be any mobile device, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), an on-board computer, a wearable device, or the like, and the mobile terminal is used as a mobile phone as an example. :

FIG. 6 is a block diagram showing a partial structure of a mobile phone related to a mobile terminal provided by an embodiment of the present application. Referring to FIG. 6, the mobile phone includes: a radio frequency (RF) circuit 610, a memory 620, an input unit 630, a display unit 640, a sensor 650, an audio circuit 660, a wireless fidelity (WiFi) module 670, and a processor 680. And power supply 690 and other components. It will be understood by those skilled in the art that the structure of the mobile phone shown in FIG. 6 does not constitute a limitation to the mobile phone, and may include more or less components than those illustrated, or a combination of certain components, or different component arrangements.

The RF circuit 610 can be used for receiving and transmitting information during the transmission or reception of information, and can receive and send the downlink information of the base station, and then send the uplink data to the base station. Generally, RF circuits include, but are not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, RF circuitry 610 can also communicate with the network and other devices via wireless communication. The above wireless communication may use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (Code Division). Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), e-mail, Short Messaging Service (SMS), and the like.

The memory 620 can be used to store software programs and modules, and the processor 680 executes various functional applications and data processing of the mobile phone by running software programs and modules stored in the memory 620. The memory 620 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as an application of a sound playing function, an application of an image playing function, etc.); The data storage area can store data (such as audio data, address book, etc.) created according to the use of the mobile phone. Moreover, memory 620 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The input unit 630 can be configured to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the handset 600. Specifically, the input unit 630 may include a touch panel 631 and other input devices 632. The touch panel 631, which may also be referred to as a touch screen, can collect touch operations on or near the user (such as a user using a finger, a stylus, or the like on the touch panel 631 or near the touch panel 631. Operation) and drive the corresponding connection device according to a preset program. In one embodiment, the touch panel 631 can include two portions of a touch detection device and a touch controller. Wherein, the touch detection device detects the touch orientation of the user, and detects a signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts the touch information into contact coordinates, and sends the touch information. The processor 680 is provided and can receive commands from the processor 680 and execute them. In addition, the touch panel 631 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 631, the input unit 630 may also include other input devices 632. Specifically, other input devices 632 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.).

The display unit 640 can be used to display information input by the user or information provided to the user as well as various menus of the mobile phone. The display unit 640 can include a display panel 641. In one embodiment, the display panel 641 can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like. In one embodiment, the touch panel 631 can cover the display panel 641. When the touch panel 631 detects a touch operation thereon or nearby, the touch panel 631 transmits to the processor 680 to determine the type of the touch event, and then the processor 680 is The type of touch event provides a corresponding visual output on display panel 641. Although in FIG. 6, the touch panel 631 and the display panel 641 are two independent components to implement the input and input functions of the mobile phone, in some embodiments, the touch panel 631 may be integrated with the display panel 641. Realize the input and output functions of the phone.

The handset 600 can also include at least one type of sensor 650, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 641 according to the brightness of the ambient light, and the proximity sensor may close the display panel 641 and/or when the mobile phone moves to the ear. Or backlight. The motion sensor may include an acceleration sensor, and the acceleration sensor can detect the magnitude of the acceleration in each direction, and the magnitude and direction of the gravity can be detected at rest, and can be used to identify the gesture of the mobile phone (such as horizontal and vertical screen switching), and vibration recognition related functions (such as Pedometer, tapping, etc.; in addition, the phone can also be equipped with gyroscopes, barometers, hygrometers, thermometers, infrared sensors and other sensors.

Audio circuitry 660, speaker 661, and microphone 662 can provide an audio interface between the user and the handset. The audio circuit 660 can transmit the converted electrical data of the received audio data to the speaker 661 for conversion to the sound signal output by the speaker 661; on the other hand, the microphone 662 converts the collected sound signal into an electrical signal by the audio circuit 660. After receiving, it is converted into audio data, and then processed by the audio data output processor 680, sent to another mobile phone via the RF circuit 610, or outputted to the memory 620 for subsequent processing.

WiFi is a short-range wireless transmission technology, and the mobile phone can help users to send and receive emails, browse web pages, and access streaming media through the WiFi module 670, which provides users with wireless broadband Internet access. Although FIG. 6 shows the WiFi module 670, it can be understood that it does not belong to the essential configuration of the mobile phone 600 and can be omitted as needed.

The processor 680 is the control center of the handset, and connects various portions of the entire handset using various interfaces and lines, by executing or executing software programs and/or modules stored in the memory 620, and invoking data stored in the memory 620, executing The phone's various functions and processing data, so that the overall monitoring of the phone. In one embodiment, processor 680 can include one or more processing units. In one embodiment, processor 680 can integrate an application processor and a modem processor, where the application processor primarily processes an operating system, user interface, and applications, etc.; the modem processor primarily processes wireless communications. It will be appreciated that the above described modem processor may also not be integrated into the processor 680.

The handset 600 also includes a power source 690 (such as a battery) that supplies power to the various components. Preferably, the power source can be logically coupled to the processor 680 via a power management system to manage functions such as charging, discharging, and power management through the power management system.

In one embodiment, the handset 600 may also include a camera, a Bluetooth module, and the like.

In the embodiment of the present application, the processor 680 included in the mobile terminal implements the image processing method as described above when executing a computer program stored in the memory.

Any reference to a memory, storage, database or other medium used herein may include non-volatile and/or volatile memory. Suitable non-volatile memories can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), which acts as an external cache. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronization. Link (Synchlink) DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM). The above-mentioned embodiments are merely illustrative of several embodiments of the present application, and the description thereof is more specific and detailed, but is not to be construed as limiting the scope of the claims. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the spirit and scope of the present application. Therefore, the scope of the invention should be determined by the appended claims.

Claims

An image processing method, comprising:

When the image clustering request is received, acquiring a target face image corresponding to the image clustering request;

Obtaining first facial feature information in the target facial image; and

And clustering the target face image according to the first face feature information;

The receiving the image clustering request includes:

Receiving a first image clustering request when the feature recognition model is updated;

Receiving, by the first cluster information sent by the server, a second image clustering request, where the first clustering information is clustering information of a second facial image set uploaded by the server to the mobile terminal;

When a new image is detected, a third clustering request is received.
The method according to claim 1, wherein when the first image clustering request is received, the acquiring the first facial feature information in the target facial image comprises:

Performing face recognition on the target face image according to the updated feature recognition model, and acquiring the first face feature information;

Or converting the stored face feature information of the target face image into the first face feature information according to the feature information conversion model.
The method according to claim 1, wherein when the second image clustering request is received, the acquiring the target face image corresponding to the image clustering request comprises:

Acquiring a face image other than the second face image set in the first face image set as the target face image;

The clustering the target facial image according to the first facial feature information includes:

Matching the first facial feature information with the second facial feature information in the second image set, and if the similarity exceeds a preset value, acquiring a first image corresponding to the second facial feature information, and the target person The face image is divided into corresponding clustering packets of the first image in the first clustering information.
The method according to claim 1, wherein when the third image clustering request is received, the acquiring the target face image corresponding to the image clustering request comprises:

Face recognition is performed on the newly added image, and if the newly added image includes a face, a new image including the face is used as the target face image.
The method of claim 1 further comprising:

Obtaining, by the mobile terminal, second cluster information of the second face set; and

When the comparison result of the first cluster information and the second cluster information is different, the second cluster information is correspondingly processed according to the type of the comparison result.
The method according to any one of claims 1 to 5, wherein before the acquiring the target face image corresponding to the image clustering request, the method further comprises:

Obtaining a target face image corresponding to the image clustering request if a time difference between a current time and a time when the target face image was last acquired exceeds a preset time length;

Or, if the current time is a preset time, acquiring a target face image corresponding to the image clustering request;

Or, the mobile terminal is in a charging state, and acquires a target face image corresponding to the image clustering request.
The method according to any one of claims 1 to 5, wherein the method further comprises:

When the face information in the target face image is detected to be changed;

Reacquiring the first face feature information in the target face image; and

And clustering the target face image according to the first face feature information.
An image processing apparatus, comprising:

a first acquiring module, configured to: when receiving an image clustering request, acquire a target facial image corresponding to the image clustering request;

a second acquiring module, configured to acquire first facial feature information in the target facial image; and

a clustering module, configured to cluster the target facial image according to the first facial feature information;

The receiving the image clustering request includes:

Receiving a first image clustering request when the feature recognition model is updated;

Receiving, by the first cluster information sent by the server, a second image clustering request, where the first clustering information is clustering information of a second facial image set uploaded by the server to the mobile terminal;

When a new image is detected, a third clustering request is received.
A computer readable storage medium having stored thereon a computer program, wherein the computer program is executed by a processor to perform the operations of the method of any one of claims 1 to 7.
A mobile terminal comprising a memory and a processor, the memory storing computer readable instructions, wherein when the computer readable instructions are executed by the processor, the processor performs the following operations:

When the image clustering request is received, acquiring a target face image corresponding to the image clustering request;

Obtaining first facial feature information in the target facial image; and

And clustering the target face image according to the first face feature information;

The receiving the image clustering request includes:

Receiving a first image clustering request when the feature recognition model is updated;

Receiving, by the first cluster information sent by the server, a second image clustering request, where the first clustering information is clustering information of a second facial image set uploaded by the server to the mobile terminal;

When a new image is detected, a third clustering request is received.
The mobile terminal of claim 10, when the computer readable instructions are executed by the processor, causing the processor to: when the first image clustering request is received, the obtaining The first face feature information in the target face image includes:

Performing face recognition on the target face image according to the updated feature recognition model, and acquiring the first face feature information;

Or converting the stored face feature information of the target face image into the first face feature information according to the feature information conversion model.
The mobile terminal of claim 10, when said computer readable instructions are executed by said processor, causing said processor to:

When the second image clustering request is received, the acquiring the target facial image corresponding to the image clustering request includes:

Acquiring a face image other than the second face image set in the first face image set as the target face image;

The clustering the target facial image according to the first facial feature information includes:

Matching the first facial feature information with the second facial feature information in the second image set, and if the similarity exceeds a preset value, acquiring a first image corresponding to the second facial feature information, and the target person The face image is divided into corresponding clustering packets of the first image in the first clustering information.
The mobile terminal of claim 10, when said computer readable instructions are executed by said processor, causing said processor to:

When the third image clustering request is received, the acquiring the target facial image corresponding to the image clustering request includes:

Face recognition is performed on the newly added image, and if the newly added image includes a face, a new image including the face is used as the target face image.
The mobile terminal of claim 10, when said computer readable instructions are executed by said processor, such that said processor further performs the following operations:

Obtaining, by the mobile terminal, second cluster information of the second face set; and

When the comparison result of the first cluster information and the second cluster information is different, the second cluster information is correspondingly processed according to the type of the comparison result.
The mobile terminal according to any one of claims 10 to 14, wherein when the computer readable instructions are executed by the processor, causing the processor to perform acquisition of a target face image corresponding to the image clustering request Previously, the following actions were also taken:

Obtaining a target face image corresponding to the image clustering request if a time difference between a current time and a time when the target face image was last acquired exceeds a preset time length;

Or, if the current time is a preset time, acquiring a target face image corresponding to the image clustering request;

Or, the mobile terminal is in a charging state, and acquires a target face image corresponding to the image clustering request.
The mobile terminal according to any one of claims 10 to 14, wherein the computer readable instructions are executed by the processor such that the processor further performs the following operations:

When the face information in the target face image is detected to be changed;

Reacquiring the first face feature information in the target face image; and

And clustering the target face image according to the first face feature information.