CN112417197A - Sorting method, sorting device, machine readable medium and equipment - Google Patents
Sorting method, sorting device, machine readable medium and equipment Download PDFInfo
- Publication number
- CN112417197A CN112417197A CN202011406007.5A CN202011406007A CN112417197A CN 112417197 A CN112417197 A CN 112417197A CN 202011406007 A CN202011406007 A CN 202011406007A CN 112417197 A CN112417197 A CN 112417197A
- Authority
- CN
- China
- Prior art keywords
- similarity
- face
- cluster
- score
- retrieved
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/55—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Abstract
The invention discloses a sorting method, which comprises the following steps: clustering the retrieved pictures similar to the retrieved face pictures in the database to obtain a plurality of cluster clusters; acquiring face similarity scores between the retrieved face pictures and each cluster; and sequencing the retrieved pictures of the plurality of cluster clusters according to the face similarity score to obtain a sequencing result. The invention greatly improves the understandability of the retrieval result on the premise of not influencing the effectiveness of the retrieval result, and ensures that the user can not reject the retrieval result subjectively as much as possible.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to a sorting method, a sorting device, a machine readable medium and equipment.
Background
In the face retrieval engine service, the precise retrieval is often realized through high-dimensional face feature vectors extracted by a computer deep neural network. However, high-dimensional face vectors generated by computer deep neural networks are not usually very intuitive and interpretative. It is straight to say that the portrait information observed by the computer is richer, more detailed and more abstract than that observed by the human eye.
The problem caused by the non-intuitive and non-interpretable nature of the high-dimensional face feature vector is that the retrieved recent face image is often not in accordance with the common knowledge of human cognition, and is difficult to understand and accept. For example, sometimes the most recent pictures that a face image may retrieve are all pictures of people of different gender from him; the most recent picture retrieved for a picture of yellow race is a picture of some white or black race, etc.
Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, it is an object of the present invention to provide a sorting method, apparatus, machine-readable medium and device, which are used to solve the problems of the prior art.
To achieve the above and other related objects, the present invention provides a sorting method, comprising:
clustering the retrieved pictures similar to the retrieved face pictures in the database to obtain a plurality of cluster clusters;
acquiring face similarity scores between the retrieved face pictures and each cluster;
and sequencing the retrieved pictures of the plurality of cluster clusters according to the face similarity score to obtain a sequencing result.
Optionally, if the face similarity score of the cluster at the first position in the sorting result is smaller than the similarity score threshold, the method further includes:
determining the biological attribute of the retrieved face picture;
acquiring biological attribute similarity scores between the retrieved face pictures and each cluster;
obtaining a total similarity score based on the human face similarity score and the biological attribute similarity score;
and sequencing the searched pictures of the plurality of clustering clusters according to the total similarity score.
Optionally, the face similarity of the cluster at the first position in the sorting result is divided into the maximum value of the face similarity between each picture in the cluster and the retrieved face picture; the face similarity of other cluster is the average value of the face similarity scores between all the pictures in the cluster and the retrieved face picture.
Optionally, the attribute similarity of each cluster is an average value of attribute similarity scores between all pictures in the cluster and the retrieved face picture.
Optionally, the obtaining a total similarity score based on the face similarity score and the attribute similarity score includes:
adding the face similarity and the biological attribute similarity to obtain a similarity total score;
or carrying out weighted average on the human face similarity and the biological attribute similarity to obtain a similarity total score.
Optionally, the biological attribute similarity score comprises at least one of: sex similarity score, age similarity score, race similarity score; if the biological attribute comprises two or three of gender, age and race, the biological attribute similarity score is the sum of biological attribute similarity scores corresponding to the biological attribute; or a weighted average of the corresponding biological attribute similarity scores.
To achieve the above and other related objects, the present invention provides a sorting apparatus comprising:
the clustering module is used for clustering the pictures which are retrieved from the database and are similar to the retrieved face pictures to obtain a plurality of clustering clusters;
the similarity score acquisition module is used for acquiring the similarity score of the face between the retrieved face picture and each cluster;
and the sequencing module is used for sequencing the retrieved pictures of the plurality of cluster clusters according to the face similarity score to obtain a sequencing result.
Optionally, the apparatus further comprises:
the biological attribute determining module is used for determining the biological attribute of the retrieved face picture when the face similarity score of the cluster most positioned at the first position in the sequencing result is smaller than a similarity score threshold value;
the attribute similarity score acquisition module is used for acquiring biological attribute similarity scores between the retrieved face pictures and each cluster;
the similarity total score acquisition module is used for obtaining a similarity total score based on the human face similarity score and the biological attribute similarity score;
and the sorting module sorts the searched pictures of the plurality of clustering clusters according to the total similarity score.
Optionally, the face similarity of the cluster at the first position in the sorting result is divided into the maximum value of the face similarity between each picture in the cluster and the retrieved face picture; the face similarity of other cluster is the average value of the face similarity scores between all the pictures in the cluster and the retrieved face picture.
Optionally, the attribute similarity of each cluster is an average value of attribute similarity scores between all pictures in the cluster and the retrieved face picture.
Optionally, the obtaining a total similarity score based on the face similarity score and the attribute similarity score includes:
adding the face similarity and the biological attribute similarity to obtain a similarity total score;
or carrying out weighted average on the human face similarity and the biological attribute similarity to obtain a similarity total score.
Optionally, the biological attribute similarity score comprises at least one of: sex similarity score, age similarity score, race similarity score; if the biological attribute comprises two or three of gender, age and race, the biological attribute similarity score is the sum of biological attribute similarity scores corresponding to the biological attribute; or a weighted average of the corresponding biological attribute similarity scores.
To achieve the above and other related objects, the present invention also provides a sorting apparatus comprising:
one or more processors; and
one or more machine-readable media having instructions stored thereon that, when executed by the one or more processors, cause the apparatus to perform one or more of the methods described previously.
To achieve the above objects and other related objects, the present invention also provides one or more machine-readable media having instructions stored thereon, which when executed by one or more processors, cause an apparatus to perform one or more of the methods described above.
As described above, the sorting method, apparatus, machine-readable medium and device provided by the present invention have the following advantages:
the invention discloses a sorting method, which comprises the following steps: clustering the retrieved pictures similar to the retrieved face pictures in the database to obtain a plurality of cluster clusters; acquiring face similarity scores between the retrieved face pictures and each cluster; and sequencing the retrieved pictures of the plurality of cluster clusters according to the face similarity score to obtain a sequencing result. The invention greatly improves the understandability of the retrieval result on the premise of not influencing the effectiveness of the retrieval result, and ensures that the user can not reject the retrieval result subjectively as much as possible.
Drawings
FIG. 1 is a flow chart of a sorting method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a sorting method according to another embodiment of the present invention;
FIG. 3 is a diagram illustrating a hardware structure of a sorting apparatus according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a hardware structure of a sorting apparatus according to another embodiment of the present invention;
fig. 5 is a schematic diagram of a hardware structure of a terminal device according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a hardware structure of a terminal device according to an embodiment of the present invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
As shown in fig. 1, a method of sorting includes:
s11, clustering the pictures similar to the retrieved face pictures retrieved from the database to obtain a plurality of cluster clusters;
s12, obtaining the face similarity score between the retrieved face picture and each cluster;
s13, sorting the retrieved pictures of the plurality of cluster clusters according to the face similarity score to obtain a sorting result. The sorting may be according to the size of the face similarity score.
The invention greatly improves the understandability of the retrieval result on the premise of not influencing the effectiveness of the retrieval result, and ensures that the user can not reject the retrieval result subjectively as much as possible.
In step S11, the retrieved face pictures are input into the retrieval engine by the user, and then one or more face pictures are obtained. Before face picture retrieval, the retrieved face pictures can be screened. For example, the quality score of the retrieved face picture is calculated, and when the quality score is smaller than a set threshold, the retrieved face picture can be considered to be not satisfactory.
Of course, the retrieved face picture may also be filtered, for example, the quality score of the retrieved face picture is calculated, and when the quality score is smaller than the set threshold, the retrieved face picture may be considered to be not satisfactory.
After the retrieved pictures are clustered, a plurality of cluster clusters can be obtained, each cluster comprises a plurality of pictures, each picture and the retrieved face picture have a face similarity score, and each cluster has a maximum face similarity score. The cluster corresponding to the largest face similarity score is used as a first cluster and is arranged at the first position of the sequencing result, the sequence of other clusters is arranged from large to small according to the face similarity score of each cluster, and the pictures in each cluster can also be arranged from large to small according to the face similarity score between each picture and the retrieved face picture. The face similarity score of the first cluster is the largest face similarity score in the cluster.
And comparing the size between the face similarity between the first cluster and the retrieved face picture and a set similarity score threshold, and if the face similarity between the first cluster and the retrieved face picture is greater than the set similarity score threshold, taking the picture corresponding to the largest face similarity score in the first cluster as the target picture.
If the face similarity score between the first cluster and the retrieved face picture is smaller than a set similarity score threshold, calculating face similarity scores between other cluster and the retrieved picture; the face similarity between the retrieved face picture and each cluster is divided into an average value of the face similarity between the retrieved face picture and each picture in each cluster. For example, the similarity between the retrieved picture and the picture a1 in the cluster a is B1, the similarity between the retrieved picture and the picture a2 in the cluster a is B2, the similarity between the retrieved picture and the picture A3 in the cluster a is B3, and the face similarity between the retrieved face picture and each cluster is (a1+ a2+ A3)/3.
In an embodiment, if the maximum face similarity score is smaller than the similarity score threshold, as shown in fig. 2, the method further includes:
s21, determining the biological attribute of the retrieved face picture;
wherein the biological attribute comprises at least one of: sex, age, race;
s22, acquiring attribute similarity scores between the retrieved face pictures and each cluster;
and the attribute similarity of each cluster is the average value of the attribute similarity between all the pictures in the cluster and the retrieved face picture. The biological attribute similarity score includes at least one of: sex similarity score, age similarity score, race similarity score;
s23, obtaining a total similarity score based on the face similarity score and the biological attribute similarity score;
if the biological attribute is one of sex, age and race, the biological attribute similarity is divided into biological attribute similarity scores corresponding to various attributes. For example, if the biological attribute is gender, the biological attributes are classified into gender-similar scores.
Wherein the gender similarity score can be calculated by the following method:
acquiring the face features of the retrieved face pictures, and calculating the gender similarity of the face features and the pictures in the cluster, wherein the higher the similarity is, the larger the gender similarity is; the similarity may be reflected by a cosine distance or a hamming distance.
The age similarity score can be calculated by the following method:
acquiring the face features of the retrieved face pictures, and calculating the age similarity between the face features and the pictures in the cluster clusters, wherein the higher the similarity is, the larger the gender similarity score is; the similarity may be reflected by a cosine distance or a hamming distance.
The ethnic similarity score can be calculated by the following method:
acquiring the face features of the retrieved face pictures, and calculating the ethnic similarity between the face features and the pictures in the cluster clusters, wherein the higher the similarity is, the larger the gender similarity score is; the similarity may be reflected by a cosine distance or a hamming distance.
In one embodiment, if the biological attributes include gender and age, the biological attribute similarity score is a gender similarity score plus an age similarity score; or a weighted average of gender similarity and age similarity.
Wherein the obtaining of the similarity total score based on the face similarity score and the attribute similarity score comprises:
adding the face similarity and the attribute similarity to obtain a total similarity score; or carrying out weighted average on the face similarity and the attribute similarity to obtain a similarity total score.
S24, the pictures of the plurality of cluster clusters are sorted according to the total similarity score.
For example, the retrieved pictures may be sorted from large to small according to the total similarity score.
In an embodiment, the method further comprises: displaying the sequencing result and the face similarity score/similarity total score of each cluster; and when the face similarity score/the similarity total score is displayed, the face similarity score/the similarity total score of other cluster clusters is obtained by subtracting the difference of the face similarity score/the similarity total score between the first cluster and the other cluster clusters from the face similarity score/the similarity total score of the cluster at the first position of the sequencing result. For example, the face similarity score/similarity of the first cluster is generally designated as a, the face similarity score/similarity of the second cluster is generally designated as B, the face similarity score/similarity of the first cluster is generally designated as C, the displayed ranking results are the first cluster, the second cluster and the third cluster, the face similarity score/similarity of the first cluster is generally designated as a, the face similarity score/similarity of the second cluster is generally designated as a-B, and the face similarity score/similarity of the third cluster is generally designated as a-C.
And after finishing sequencing the pictures, outputting a sequencing result, and if the face similarity scores need to be output simultaneously, assigning a similarity score to each face picture again according to the sequencing result.
As shown in fig. 3, a sorting apparatus includes:
the clustering module 31 is configured to cluster the retrieved pictures similar to the retrieved face pictures in the database to obtain a plurality of cluster clusters;
a similarity score obtaining module 32, configured to obtain a face similarity score between the retrieved face picture and each cluster;
and the sorting module 33 is configured to sort the retrieved pictures of the multiple cluster clusters according to the face similarity score to obtain a sorting result. The sorting may be according to the size of the face similarity score.
The invention greatly improves the understandability of the retrieval result on the premise of not influencing the effectiveness of the retrieval result, and ensures that the user can not reject the retrieval result subjectively as much as possible.
The retrieved face pictures are input into a retrieval engine by a user, and then one or more face pictures are obtained. Before face picture retrieval, the retrieved face pictures can be screened. For example, the quality score of the retrieved face picture is calculated, and when the quality score is smaller than a set threshold, the retrieved face picture can be considered to be not satisfactory.
Of course, the retrieved face picture may also be filtered, for example, the quality score of the retrieved face picture is calculated, and when the quality score is smaller than the set threshold, the retrieved face picture may be considered to be not satisfactory.
After the retrieved pictures are clustered, a plurality of cluster clusters can be obtained, each cluster comprises a plurality of pictures, each picture and the retrieved face picture have a face similarity score, and each cluster has a maximum face similarity score. The cluster corresponding to the largest face similarity score is used as a first cluster and is arranged at the first position of the sequencing result, the sequence of other clusters is arranged from large to small according to the face similarity score of each cluster, and the pictures in each cluster can also be arranged from large to small according to the face similarity score between each picture and the retrieved face picture. The face similarity score of the first cluster is the largest face similarity score in the cluster.
And comparing the size between the face similarity between the first cluster and the retrieved face picture and a set similarity score threshold, and if the face similarity between the first cluster and the retrieved face picture is greater than the set similarity score threshold, taking the picture corresponding to the largest face similarity score in the first cluster as the target picture.
If the face similarity score between the first cluster and the retrieved face picture is smaller than a set similarity score threshold, calculating face similarity scores between other cluster and the retrieved picture; the face similarity between the retrieved face picture and each cluster is divided into an average value of the face similarity between the retrieved face picture and each picture in each cluster. For example, the similarity between the retrieved picture and the picture a1 in the cluster a is B1, the similarity between the retrieved picture and the picture a2 in the cluster a is B2, the similarity between the retrieved picture and the picture A3 in the cluster a is B3, and the face similarity between the retrieved face picture and each cluster is (a1+ a2+ A3)/3.
In an embodiment, as shown in fig. 4, the apparatus further includes:
a biological attribute determining module 41, configured to determine a biological attribute of the retrieved face picture when the face similarity score of the cluster most located at the first position in the sorting result is smaller than a similarity score threshold;
wherein the biological attribute comprises at least one of: sex, age, race;
an attribute similarity score obtaining module 42, configured to obtain a biological attribute similarity score between the retrieved face picture and each cluster;
and the attribute similarity of each cluster is the average value of the attribute similarity between all the pictures in the cluster and the retrieved face picture. The biological attribute similarity score includes at least one of: sex similarity score, age similarity score, race similarity score;
a similarity total score obtaining module 43, configured to obtain a similarity total score based on the face similarity score and the biological attribute similarity score;
if the biological attribute is one of sex, age and race, the biological attribute similarity is divided into biological attribute similarity scores corresponding to various attributes. For example, if the biological attribute is gender, the biological attributes are classified into gender-similar scores.
Wherein the gender similarity score can be calculated by the following method:
acquiring the face features of the retrieved face pictures, and calculating the gender similarity of the face features and the pictures in the cluster, wherein the higher the similarity is, the larger the gender similarity is; the similarity may be reflected by a cosine distance or a hamming distance.
The age similarity score can be calculated by the following method:
acquiring the face features of the retrieved face pictures, and calculating the age similarity between the face features and the pictures in the cluster clusters, wherein the higher the similarity is, the larger the gender similarity score is; the similarity may be reflected by a cosine distance or a hamming distance.
The ethnic similarity score can be calculated by the following method:
acquiring the face features of the retrieved face pictures, and calculating the ethnic similarity between the face features and the pictures in the cluster clusters, wherein the higher the similarity is, the larger the gender similarity score is; the similarity may be reflected by a cosine distance or a hamming distance.
In one embodiment, if the biological attributes include gender and age, the biological attribute similarity score is a gender similarity score plus an age similarity score; or a weighted average of gender similarity and age similarity.
Wherein the obtaining of the similarity total score based on the face similarity score and the attribute similarity score comprises:
adding the face similarity and the attribute similarity to obtain a total similarity score; or carrying out weighted average on the face similarity and the attribute similarity to obtain a similarity total score.
And the sorting module sorts the searched pictures of the plurality of clustering clusters according to the total similarity score. For example, the retrieved pictures may be sorted from large to small according to the total similarity score.
And after finishing sequencing the pictures, outputting a sequencing result, and if the face similarity scores need to be output simultaneously, assigning a similarity score to each face picture again according to the sequencing result.
In one embodiment, the apparatus further comprises: the display module is used for displaying the sequencing result and the face similarity score/similarity total score of each cluster; and when the face similarity score/the similarity total score is displayed, the face similarity score/the similarity total score of other cluster clusters is obtained by subtracting the difference of the face similarity score/the similarity total score between the first cluster and the other cluster clusters from the face similarity score/the similarity total score of the cluster at the first position of the sequencing result. For example, the face similarity score/similarity of the first cluster is generally designated as a, the face similarity score/similarity of the second cluster is generally designated as B, the face similarity score/similarity of the first cluster is generally designated as C, the displayed ranking results are the first cluster, the second cluster and the third cluster, the face similarity score/similarity of the first cluster is generally designated as a, the face similarity score/similarity of the second cluster is generally designated as a-B, and the face similarity score/similarity of the third cluster is generally designated as a-C.
An embodiment of the present application further provides an apparatus, which may include: one or more processors; and one or more machine readable media having instructions stored thereon that, when executed by the one or more processors, cause the apparatus to perform the method of fig. 1. In practical applications, the device may be used as a terminal device, and may also be used as a server, where examples of the terminal device may include: smart phones, tablet computers, e-book readers, MP3 (Moving picture experts group audiolayer iii) players, MP4 (Moving picture experts group audiolayer iv) players, laptop portable computers, car-mounted computers, desktop computers, set-top boxes, smart televisions, wearable devices, and the like, and the embodiments of the present application are not limited to specific devices.
The present application further provides a non-transitory readable storage medium, where one or more modules (programs) are stored in the storage medium, and when the one or more modules are applied to a device, the device may be caused to execute instructions (instructions) of steps included in the method in fig. 1 according to the present application.
Fig. 5 is a schematic diagram of a hardware structure of a terminal device according to an embodiment of the present application. As shown, the terminal device may include: an input device 1100, a first processor 1101, an output device 1102, a first memory 1103, and at least one communication bus 1104. The communication bus 1104 is used to implement communication connections between the elements. The first memory 1103 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and the first memory 1103 may store various programs for performing various processing functions and implementing the method steps of the present embodiment.
Alternatively, the first processor 1101 may be, for example, a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, and the first processor 1101 is coupled to the input device 1100 and the output device 1102 through a wired or wireless connection.
Optionally, the input device 1100 may include a variety of input devices, such as at least one of a user-oriented user interface, a device-oriented device interface, a software programmable interface, a camera, and a sensor. Optionally, the device interface facing the device may be a wired interface for data transmission between devices, or may be a hardware plug-in interface (e.g., a USB interface, a serial port, etc.) for data transmission between devices; optionally, the user-facing user interface may be, for example, a user-facing control key, a voice input device for receiving voice input, and a touch sensing device (e.g., a touch screen with a touch sensing function, a touch pad, etc.) for receiving user touch input; optionally, the programmable interface of the software may be, for example, an entry for a user to edit or modify a program, such as an input pin interface or an input interface of a chip; the output devices 1102 may include output devices such as a display, audio, and the like.
In this embodiment, the processor of the terminal device includes a module for executing functions of each module in each device, and specific functions and technical effects may refer to the foregoing embodiments, which are not described herein again.
Fig. 6 is a schematic hardware structure diagram of a terminal device according to an embodiment of the present application. FIG. 6 is a specific embodiment of the implementation of FIG. 5. As shown, the terminal device of the present embodiment may include a second processor 1201 and a second memory 1202.
The second processor 1201 executes the computer program code stored in the second memory 1202 to implement the method described in fig. 1 in the above embodiment.
The second memory 1202 is configured to store various types of data to support operations at the terminal device. Examples of such data include instructions for any application or method operating on the terminal device, such as messages, pictures, videos, and so forth. The second memory 1202 may include a Random Access Memory (RAM) and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
Optionally, a second processor 1201 is provided in the processing assembly 1200. The terminal device may further include: communication component 1203, power component 1204, multimedia component 1205, speech component 1206, input/output interfaces 1207, and/or sensor component 1208. The specific components included in the terminal device are set according to actual requirements, which is not limited in this embodiment.
The processing component 1200 generally controls the overall operation of the terminal device. The processing assembly 1200 may include one or more second processors 1201 to execute instructions to perform all or part of the steps of the data processing method described above. Further, the processing component 1200 can include one or more modules that facilitate interaction between the processing component 1200 and other components. For example, the processing component 1200 can include a multimedia module to facilitate interaction between the multimedia component 1205 and the processing component 1200.
The power supply component 1204 provides power to the various components of the terminal device. The power components 1204 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the terminal device.
The multimedia components 1205 include a display screen that provides an output interface between the terminal device and the user. In some embodiments, the display screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the display screen includes a touch panel, the display screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
The voice component 1206 is configured to output and/or input voice signals. For example, the voice component 1206 includes a Microphone (MIC) configured to receive external voice signals when the terminal device is in an operational mode, such as a voice recognition mode. The received speech signal may further be stored in the second memory 1202 or transmitted via the communication component 1203. In some embodiments, the speech component 1206 further comprises a speaker for outputting speech signals.
The input/output interface 1207 provides an interface between the processing component 1200 and peripheral interface modules, which may be click wheels, buttons, etc. These buttons may include, but are not limited to: a volume button, a start button, and a lock button.
The sensor component 1208 includes one or more sensors for providing various aspects of status assessment for the terminal device. For example, the sensor component 1208 may detect an open/closed state of the terminal device, relative positioning of the components, presence or absence of user contact with the terminal device. The sensor assembly 1208 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact, including detecting the distance between the user and the terminal device. In some embodiments, the sensor assembly 1208 may also include a camera or the like.
The communication component 1203 is configured to facilitate communications between the terminal device and other devices in a wired or wireless manner. The terminal device may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In one embodiment, the terminal device may include a SIM card slot therein for inserting a SIM card therein, so that the terminal device may log onto a GPRS network to establish communication with the server via the internet.
As can be seen from the above, the communication component 1203, the voice component 1206, the input/output interface 1207 and the sensor component 1208 referred to in the embodiment of fig. 6 can be implemented as the input device in the embodiment of fig. 5.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.
Claims (14)
1. A method of sorting, comprising:
clustering the retrieved pictures similar to the retrieved face pictures in the database to obtain a plurality of cluster clusters;
acquiring face similarity scores between the retrieved face pictures and each cluster;
and sequencing the retrieved pictures of the plurality of cluster clusters according to the face similarity score to obtain a sequencing result.
2. The sorting method according to claim 1, wherein if the face similarity score of the cluster located at the first position in the sorting result is smaller than the similarity score threshold, the method further comprises:
determining the biological attribute of the retrieved face picture;
acquiring biological attribute similarity scores between the retrieved face pictures and each cluster;
obtaining a total similarity score based on the human face similarity score and the biological attribute similarity score;
and sequencing the searched pictures of the plurality of clustering clusters according to the total similarity score.
3. The sorting method according to claim 1 or 2, wherein the face similarity of the cluster at the first position in the sorting result is the maximum value of the face similarity between each picture in the cluster and the retrieved face picture; the face similarity of other cluster is the average value of the face similarity scores between all the pictures in the cluster and the retrieved face picture.
4. The ordering method according to claim 3, wherein the attribute similarity of each cluster is an average of attribute similarity scores between all pictures in the cluster and the retrieved face picture.
5. The sorting method according to claim 2, wherein the obtaining a similarity total score based on the face similarity score and the attribute similarity score comprises:
adding the face similarity and the biological attribute similarity to obtain a similarity total score;
or carrying out weighted average on the human face similarity and the biological attribute similarity to obtain a similarity total score.
6. The ranking method of claim 5, wherein the biological attribute similarity score comprises at least one of: sex similarity score, age similarity score, race similarity score; if the biological attribute comprises two or three of gender, age and race, the biological attribute similarity score is the sum of biological attribute similarity scores corresponding to the biological attribute; or a weighted average of the corresponding biological attribute similarity scores.
7. A sequencing apparatus, comprising:
the clustering module is used for clustering the pictures which are retrieved from the database and are similar to the retrieved face pictures to obtain a plurality of clustering clusters;
the similarity score acquisition module is used for acquiring the similarity score of the face between the retrieved face picture and each cluster;
and the sequencing module is used for sequencing the retrieved pictures of the plurality of cluster clusters according to the face similarity score to obtain a sequencing result.
8. The sequencing apparatus of claim 7, further comprising:
the biological attribute determining module is used for determining the biological attribute of the retrieved face picture when the face similarity score of the cluster most positioned at the first position in the sequencing result is smaller than a similarity score threshold value;
the attribute similarity score acquisition module is used for acquiring biological attribute similarity scores between the retrieved face pictures and each cluster;
the similarity total score acquisition module is used for obtaining a similarity total score based on the human face similarity score and the biological attribute similarity score;
and the sorting module sorts the searched pictures of the plurality of clustering clusters according to the total similarity score.
9. The sorting device according to claim 7 or 8, characterized in that the face similarity of the cluster at the first position in the sorting result is the maximum value of the face similarity between each picture in the cluster and the retrieved face picture; the face similarity of other cluster is the average value of the face similarity scores between all the pictures in the cluster and the retrieved face picture.
10. The sorting device according to claim 9, wherein the attribute similarity of each cluster is an average of attribute similarity scores between all pictures in the cluster and the retrieved face picture.
11. The sorting apparatus according to claim 8, wherein said deriving a similarity total score based on the face similarity score and the attribute similarity score comprises:
adding the face similarity and the biological attribute similarity to obtain a similarity total score;
or carrying out weighted average on the human face similarity and the biological attribute similarity to obtain a similarity total score.
12. The sequencing apparatus of claim 11, wherein the biological attribute similarity score comprises at least one of: sex similarity score, age similarity score, race similarity score; if the biological attribute comprises two or three of gender, age and race, the biological attribute similarity score is the sum of biological attribute similarity scores corresponding to the biological attribute; or a weighted average of the corresponding biological attribute similarity scores.
13. A sequencing apparatus, comprising:
one or more processors; and
one or more machine-readable media having instructions stored thereon that, when executed by the one or more processors, cause the apparatus to perform the method recited by one or more of claims 1-6.
14. One or more machine-readable media having instructions stored thereon, which when executed by one or more processors, cause an apparatus to perform the method recited by one or more of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011406007.5A CN112417197B (en) | 2020-12-02 | 2020-12-02 | Sorting method, sorting device, machine readable medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011406007.5A CN112417197B (en) | 2020-12-02 | 2020-12-02 | Sorting method, sorting device, machine readable medium and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112417197A true CN112417197A (en) | 2021-02-26 |
| CN112417197B CN112417197B (en) | 2022-02-25 |
Family
ID=74830231
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011406007.5A Active CN112417197B (en) | 2020-12-02 | 2020-12-02 | Sorting method, sorting device, machine readable medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112417197B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114022934A (en) * | 2021-11-04 | 2022-02-08 | 南威软件股份有限公司 | Real-time portrait document gathering method, system and medium based on majority principle |
Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101667248A (en) * | 2008-09-04 | 2010-03-10 | 索尼株式会社 | Image processing apparatus, imaging apparatus, image processing method, and program |
| US20120106853A1 (en) * | 2010-11-01 | 2012-05-03 | Microsoft Corporation | Image search |
| CN103207898A (en) * | 2013-03-19 | 2013-07-17 | 天格科技（杭州）有限公司 | Method for rapidly retrieving similar faces based on locality sensitive hashing |
| CN103399951A (en) * | 2013-08-19 | 2013-11-20 | 山东大学 | Semi-supervised image reordering method with self-feedback characteristic based on heterogeneous diagram |
| CN103902694A (en) * | 2014-03-28 | 2014-07-02 | 哈尔滨工程大学 | Clustering and query behavior based retrieval result sorting method |
| CN104143088A (en) * | 2014-07-25 | 2014-11-12 | 电子科技大学 | Face identification method based on image retrieval and feature weight learning |
| CN105631039A (en) * | 2016-01-15 | 2016-06-01 | 北京邮电大学 | Picture browsing method |
| CN106033549A (en) * | 2015-03-16 | 2016-10-19 | 北京大学 | Reordering method in vehicle retrieval and apparatus thereof |
| CN106126581A (en) * | 2016-06-20 | 2016-11-16 | 复旦大学 | Cartographical sketching image search method based on degree of depth study |
| CN106649710A (en) * | 2016-12-20 | 2017-05-10 | 北京奇虎科技有限公司 | Picture pushing method, device and mobile terminal |
| CN106815566A (en) * | 2016-12-29 | 2017-06-09 | 天津中科智能识别产业技术研究院有限公司 | A kind of face retrieval method based on multitask convolutional neural networks |
| CN108009465A (en) * | 2016-10-31 | 2018-05-08 | 杭州海康威视数字技术股份有限公司 | A kind of face identification method and device |
| CN108829900A (en) * | 2018-07-31 | 2018-11-16 | 成都视观天下科技有限公司 | A kind of Research on face image retrieval based on deep learning, device and terminal |
| CN109993102A (en) * | 2019-03-28 | 2019-07-09 | 北京达佳互联信息技术有限公司 | Similar face retrieval method, apparatus and storage medium |
| CN110704659A (en) * | 2019-09-30 | 2020-01-17 | 腾讯科技（深圳）有限公司 | Image list sorting method and device, storage medium and electronic device |
| CN110825899A (en) * | 2019-09-18 | 2020-02-21 | 武汉纺织大学 | Clothing image retrieval method integrating color features and residual network depth features |
| CN111242040A (en) * | 2020-01-15 | 2020-06-05 | 佳都新太科技股份有限公司 | Dynamic face clustering method, device, equipment and storage medium |
| CN111310725A (en) * | 2020-03-13 | 2020-06-19 | 恒睿(重庆)人工智能技术研究院有限公司 | Object identification method, system, machine readable medium and device |
-
2020
- 2020-12-02 CN CN202011406007.5A patent/CN112417197B/en active Active
Patent Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101667248A (en) * | 2008-09-04 | 2010-03-10 | 索尼株式会社 | Image processing apparatus, imaging apparatus, image processing method, and program |
| US20120106853A1 (en) * | 2010-11-01 | 2012-05-03 | Microsoft Corporation | Image search |
| CN103207898A (en) * | 2013-03-19 | 2013-07-17 | 天格科技（杭州）有限公司 | Method for rapidly retrieving similar faces based on locality sensitive hashing |
| CN103399951A (en) * | 2013-08-19 | 2013-11-20 | 山东大学 | Semi-supervised image reordering method with self-feedback characteristic based on heterogeneous diagram |
| CN103902694A (en) * | 2014-03-28 | 2014-07-02 | 哈尔滨工程大学 | Clustering and query behavior based retrieval result sorting method |
| CN104143088A (en) * | 2014-07-25 | 2014-11-12 | 电子科技大学 | Face identification method based on image retrieval and feature weight learning |
| CN106033549A (en) * | 2015-03-16 | 2016-10-19 | 北京大学 | Reordering method in vehicle retrieval and apparatus thereof |
| CN105631039A (en) * | 2016-01-15 | 2016-06-01 | 北京邮电大学 | Picture browsing method |
| CN106126581A (en) * | 2016-06-20 | 2016-11-16 | 复旦大学 | Cartographical sketching image search method based on degree of depth study |
| CN108009465A (en) * | 2016-10-31 | 2018-05-08 | 杭州海康威视数字技术股份有限公司 | A kind of face identification method and device |
| CN106649710A (en) * | 2016-12-20 | 2017-05-10 | 北京奇虎科技有限公司 | Picture pushing method, device and mobile terminal |
| CN106815566A (en) * | 2016-12-29 | 2017-06-09 | 天津中科智能识别产业技术研究院有限公司 | A kind of face retrieval method based on multitask convolutional neural networks |
| CN108829900A (en) * | 2018-07-31 | 2018-11-16 | 成都视观天下科技有限公司 | A kind of Research on face image retrieval based on deep learning, device and terminal |
| CN109993102A (en) * | 2019-03-28 | 2019-07-09 | 北京达佳互联信息技术有限公司 | Similar face retrieval method, apparatus and storage medium |
| CN110825899A (en) * | 2019-09-18 | 2020-02-21 | 武汉纺织大学 | Clothing image retrieval method integrating color features and residual network depth features |
| CN110704659A (en) * | 2019-09-30 | 2020-01-17 | 腾讯科技（深圳）有限公司 | Image list sorting method and device, storage medium and electronic device |
| CN111242040A (en) * | 2020-01-15 | 2020-06-05 | 佳都新太科技股份有限公司 | Dynamic face clustering method, device, equipment and storage medium |
| CN111310725A (en) * | 2020-03-13 | 2020-06-19 | 恒睿(重庆)人工智能技术研究院有限公司 | Object identification method, system, machine readable medium and device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114022934A (en) * | 2021-11-04 | 2022-02-08 | 南威软件股份有限公司 | Real-time portrait document gathering method, system and medium based on majority principle |
| CN114022934B (en) * | 2021-11-04 | 2023-06-27 | 清华大学 | Real-time portrait filing method, system and medium based on majority principle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112417197B (en) | 2022-02-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110471858B (en) | Application program testing method, device and storage medium | |
| EP4083817A1 (en) | Video tag determination method, device, terminal, and storage medium | |
| CN111598012B (en) | Picture clustering management method, system, device and medium | |
| CN111310725A (en) | Object identification method, system, machine readable medium and device | |
| CN111898495B (en) | Dynamic threshold management method, system, device and medium | |
| CN111339737B (en) | Entity linking method, device, equipment and storage medium | |
| CN111626229A (en) | Object management method, device, machine readable medium and equipment | |
| CN112417197B (en) | Sorting method, sorting device, machine readable medium and equipment | |
| CN110022397B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN111178455B (en) | Image clustering method, system, device and medium | |
| CN112000264B (en) | Dish information display method and device, computer equipment and storage medium | |
| CN112989210A (en) | Insurance recommendation method, system, equipment and medium based on health portrait | |
| CN111275683B (en) | Image quality grading processing method, system, device and medium | |
| CN113077262A (en) | Catering settlement method, device, system, machine readable medium and equipment | |
| CN110363187B (en) | Face recognition method, face recognition device, machine readable medium and equipment | |
| CN111639705B (en) | Batch picture marking method, system, machine readable medium and equipment | |
| CN112347982A (en) | Video-based unsupervised difficult case data mining method, device, medium and equipment | |
| CN108021648B (en) | Question searching method and device and intelligent terminal | |
| CN112580472A (en) | Rapid and lightweight face recognition method and device, machine readable medium and equipment | |
| CN112257812A (en) | Method and device for determining labeled sample, machine readable medium and equipment | |
| CN112232890A (en) | Data processing method, device, equipment and storage medium | |
| CN110909190B (en) | Data searching method and device, electronic equipment and storage medium | |
| CN111079662A (en) | Figure identification method and device, machine readable medium and equipment | |
| KR20200101055A (en) | Method for displaying visual object regarding contents and electronic device thereof | |
| CN111710011B (en) | Cartoon generation method and system, electronic device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |