CN113269154A

CN113269154A - Image identification method, device, equipment and storage medium

Info

Publication number: CN113269154A
Application number: CN202110729630.2A
Authority: CN
Inventors: 于志鹏; 梁鼎
Original assignee: Beijing Sensetime Technology Development Co Ltd
Current assignee: Beijing Sensetime Technology Development Co Ltd
Priority date: 2021-06-29
Filing date: 2021-06-29
Publication date: 2021-08-17
Anticipated expiration: 2041-06-29
Also published as: CN113269154B; WO2023273437A1

Abstract

The application relates to an image recognition method, an image recognition device and a storage medium, wherein the image recognition method comprises the following steps: acquiring an image to be identified; responding to a specific relation between the image to be recognized and a first template, and acquiring a second template matched with the first template, wherein the first template is a reference image used for image recognition or a feature of the reference image, and the first template and the second template meet a preset similar relation; and obtaining an identification result corresponding to the image to be identified based on the image to be identified, the first template and the second template.

Description

Image identification method, device, equipment and storage medium

Technical Field

The present application relates to the field of image recognition technologies, and in particular, to an image recognition method, an image recognition apparatus, an image recognition device, and a storage medium.

Background

With the development of artificial intelligence technology, the accuracy of image recognition is higher and higher, and the application range is wider and wider. The face recognition in the image recognition can be applied to a great number of scenes in life, and brings convenience to life. The face recognition can be applied to functions with higher requirements on security level, such as unlocking and payment, and the accuracy of the face recognition directly influences the security of the functions. At present, although the face recognition precision in the related art is high, the distinguishing precision of people with specific relationships such as relatives is poor.

Disclosure of Invention

The application provides an image identification method, an image identification device, an image identification equipment and a storage medium, which are used for solving the defects in the related art.

According to a first aspect of embodiments of the present application, there is provided an image recognition method, the method including:

acquiring an image to be identified;

responding to a specific relation between the image to be recognized and a first template, and acquiring a second template matched with the first template, wherein the first template is a reference image used for image recognition or a feature of the reference image, and the first template and the second template meet a preset similar relation;

and obtaining an identification result corresponding to the image to be identified based on the image to be identified, the first template and the second template.

In one embodiment, in case the image to be recognized is in the specific relationship with the first template, the method further comprises at least one of:

in response to there being no second template matching the first template, generating the second template based on the image to be recognized;

in response to receiving a trigger instruction indicating generation of a second template, generating the second template based on the image to be recognized;

and in response to the existing second template meeting an updating period or an updating condition, updating the second template based on the image to be identified.

In one embodiment, before the generating the second template based on the image to be recognized, the method further comprises:

acquiring a stored image set or a feature set, wherein each image in the image set is in the specific relationship with the first template, and each feature in the feature set is a feature of an image in the specific relationship with the first template;

the generating the second template based on the image to be recognized includes at least one of:

generating the second template based on the acquired image set and the image to be identified;

and generating the second template based on the acquired feature corresponding to each image in the image set or the acquired feature set and the features of the image to be identified.

In one embodiment, further comprising:

acquiring a first similarity between the image to be recognized or the characteristics of the image to be recognized and the first template;

and determining that the image to be recognized and the first template are in the specific relation in response to the first similarity being within a preset range.

In one embodiment, further comprising:

inputting an image combination or a feature combination into a specific relationship recognition network, determining an image to be recognized in the image combination or an image to be recognized to which a feature in the feature combination belongs in response to the output of the specific relationship recognition network being a specific relationship, and making the image combination and the first template have the specific relationship, wherein the image combination comprises the image to be recognized and the reference image, the feature combination comprises the feature of the image to be recognized and the feature of the reference image, and the specific relationship recognition network is pre-trained by using the labeled specific image.

In one embodiment, further comprising:

acquiring matching information between the attribute of the image to be identified and the attribute of the reference image;

and determining that the image to be recognized or the characteristic of the image to be recognized is in the specific relation with the first template in response to that the matching information of the attribute of the image to be recognized and the attribute of the reference image meets the preset requirement.

In one embodiment, the generating the second template based on the acquired feature corresponding to each image in the image set or the acquired feature set and the feature of the image to be recognized includes:

performing clustering processing of two categories on the features corresponding to each image in the image set or each feature in the feature set and the features of the image to be identified;

acquiring second similarity between the central features of the two categories and the features of the reference image respectively;

and determining the central feature corresponding to the highest second similarity in the second similarities or the image to which the central feature corresponding to the highest second similarity belongs as the second template.

In one embodiment, after the clustering processing of two categories of the features corresponding to each image in the image set or each feature in the feature set and the features of the image to be identified, the method further includes:

acquiring features of which the distances to the central features of the two categories are both greater than a distance threshold from the features corresponding to the images in the image set or the feature set and the features of the images to be identified, and determining the features as discrete features;

removing the discrete features.

In one embodiment, further comprising at least one of:

responsive to the image to be identified being in the particular relationship with the first template and a number of images in the set of images being less than a first number threshold, adding the image to be identified to the set of images;

in response to the image to be identified being in the particular relationship with the first template and the number of features in the feature set being less than a second number threshold, adding features of the image to be identified to the feature set;

the acquiring a stored set of images or features includes at least one of:

in response to the number of images in the stored set of images being greater than or equal to the first number threshold, retrieving the set of images;

in response to the number of features in the stored feature set being greater than or equal to the second number threshold, retrieving the feature set.

In one embodiment, the obtaining, based on the image to be recognized, the first template, and the second template, a recognition result corresponding to the image to be recognized includes:

acquiring the image to be recognized or the characteristics of the image to be recognized, and the third similarity of the image to be recognized and the first template and the fourth similarity of the image to be recognized and the second template;

in response to a third similarity with the first template being greater than or equal to a fourth similarity with the second template, determining that the recognition result is that the image to be recognized matches the reference image;

and in response to the third similarity with the first template being smaller than the fourth similarity with the second template, determining that the identification result is that the image to be identified does not match the reference image.

In one embodiment, further comprising:

and in response to the fact that the image to be recognized does not have the specific relation with the first template and/or no second template matched with the first template exists, obtaining a recognition result corresponding to the image to be recognized based on the image to be recognized and the first template.

In one embodiment, the reference image is a base library image used for image comparison of at least one of identity recognition, unlocking and payment, or an image corresponding to a base library feature used for image comparison.

In one embodiment, further comprising at least one of:

under the condition that the reference image is a bottom library image used for image comparison by the identity recognition function or an image corresponding to bottom library characteristics used for image comparison, responding to the recognition result that the image to be recognized is matched with the reference image, and outputting content for representing that the identity recognition result is identity authentication;

under the condition that the reference image is a bottom library image used for image comparison of the unlocking function or an image corresponding to bottom library characteristics used for image comparison, responding to the recognition result that the image to be recognized is matched with the reference image, and unlocking a screen and/or unlocking the operation authority of the designated function;

and under the condition that the reference image is a bottom library image used for image comparison of the payment function or an image corresponding to bottom library characteristics used for image comparison, responding to the recognition result that the image to be recognized is matched with the reference image, and paying to a target account.

According to a second aspect of embodiments of the present application, there is provided an image recognition apparatus, the apparatus comprising:

the first acquisition module is used for acquiring an image to be identified;

the second acquisition module is used for responding to the specific relation between the image to be recognized and a first template, acquiring a second template matched with the first template, wherein the first template is a reference image used for image recognition or the characteristic of the reference image, and the first template and the second template meet the preset similar relation;

and the first identification module is used for obtaining an identification result corresponding to the image to be identified based on the image to be identified, the first template and the second template.

In one embodiment, in the case that the image to be recognized is in the specific relationship with the first template, the apparatus further comprises at least one of:

a first generating module, configured to generate a second template based on the image to be recognized in response to an absence of the second template matching the first template;

the second generation module is used for responding to the receiving of a trigger instruction for generating a second template, and generating the second template based on the image to be identified;

and the third generation module is used for responding to the condition that the existing second template meets the updating period or the updating condition, and updating the second template based on the image to be identified.

In one embodiment, before the generating the second template based on the image to be recognized, the apparatus further comprises:

a third obtaining module, configured to obtain a stored image set or a feature set, where each image in the image set is in the specific relationship with the first template, and each feature in the feature set is a feature of an image in the specific relationship with the first template;

the first generation module, the second generation module, or the third generation module is specifically configured to, when generating the second template based on the image to be recognized, at least one of the following:

In one embodiment, the apparatus further comprises:

the fourth acquisition module is used for acquiring the image to be identified or the first similarity between the characteristic of the image to be identified and the first template;

and the first determining module is used for determining that the image to be recognized and the first template are in the specific relationship in response to the first similarity being within a preset range.

The apparatus also includes a second determining module to:

The apparatus also includes a third determining module to:

In an embodiment, the first generating module, the second generating module, or the third generating module is configured to, when generating the second template based on the acquired feature corresponding to each image in the image set or the acquired feature set, and the feature of the image to be recognized, specifically:

In one embodiment, after the first generation module, the second generation module, or the third generation module is configured to perform two categories of clustering on the feature corresponding to each image in the image set or each feature in the feature set, and the feature of the image to be identified, the method is further configured to:

removing the discrete features.

In one embodiment, the apparatus further comprises at least one of:

an image set construction module, configured to add the image to be identified to the image set in response to the image to be identified and the first template being in the specific relationship and the number of images in the image set being less than a first number threshold;

the characteristic set construction module is used for responding to the specific relation between the image to be recognized and the first template, and the quantity of the characteristics in the characteristic set is smaller than a second quantity threshold value, and adding the characteristics of the image to be recognized into the characteristic set;

the third obtaining module is configured to, when obtaining the stored image set or the feature set, specifically at least one of:

In one embodiment, the first identification module is specifically configured to:

acquiring the image to be recognized or the characteristics of the image to be recognized, the third similarity of the image to be recognized and the first template, and the fourth similarity of the image to be recognized and the second template;

In one embodiment, the system further comprises a second identification module for:

In one embodiment, the system further comprises an execution module for at least one of:

According to a third aspect of embodiments herein, there is provided an electronic device comprising a memory for storing computer instructions executable on a processor, the processor being configured to perform the image recognition method according to the first aspect when executing the computer instructions.

According to a fourth aspect of embodiments herein, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of the first aspect.

According to the embodiment, the image to be recognized is obtained, the second template matched with the first template is obtained in response to the specific relationship between the image to be recognized and the first template, the first template is a reference image used for image recognition or the characteristics of the reference image, the first template and the second template meet the preset similarity relationship, and finally the recognition result corresponding to the image to be recognized is obtained based on the image to be recognized, the first template and the second template. When the identified image to be identified is in a specific relation with the first template, the second template is combined to identify on the basis of the first template, so that the error identification of the specific image can be reduced, and the identification accuracy of the image is improved. For example, when face images with similar face features such as relatives are recognized, the face recognition accuracy can be improved, and the probability of mistakenly recognizing the face images of the relatives as the face images of the relatives is reduced.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

Fig. 1 is a flowchart illustrating an image recognition method according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of an image recognition apparatus according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of an electronic device shown in an embodiment of the present application.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

For most people, the current face recognition model can realize very high recognition accuracy, but for some special people, the face recognition algorithm has certain problems, for example, because the natural similarity between twins is high, the probability of misidentification is often high, and the phenomenon that the correct person in the twins cannot be recognized and matched is easy to occur in the recognition process.

In the existing human face recognition solution, most of comparison-based methods only use a fixed threshold value to determine the similarity of two human face pictures. The similarity score between twins is often high, and is easy to exceed a fixed threshold value, so that the phenomenon of false identification is caused. In the face of such a situation, the problem can be solved only by increasing the threshold, but the method can cause the reduction of the recognition throughput rate, and the throughput rate of the user himself is also reduced correspondingly, which seriously affects the use experience of the user.

In some special specialized models, the comparison performance of the twins picture is improved by adding more relatives and twins pictures in the training data. However, more face data and labeling cost are required, and the phenomenon of misrecognition caused by twins cannot be really and effectively avoided for a long time in practical use.

Based on this, in a first aspect, at least one embodiment of the present application provides an image recognition method, please refer to fig. 1, which illustrates a flow of the method, including steps S101 to S103.

The method may be performed by an electronic device such as a terminal or a server, where the terminal may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA) handheld device, a computing device, a vehicle-mounted device, a wearable device, and the like, and the method may be implemented by a processor calling a computer readable instruction stored in a memory. Alternatively, the method may be performed by a server, which may be a local server, a cloud server, or the like.

The image recognition method can be used for recognizing specific kinds of images, such as facial images. Therefore, a first template (i.e., a feature of the reference image or the reference image) such as a face image of the user or a feature of the face image is stored in advance in the main body for executing the image recognition method. The type of the image to be recognized is the same as that of the reference image, for example, when the reference image is a face image, the image to be recognized is also a face image. During identification, the identification can be completed by comparing the image to be identified or the characteristics of the image to be identified with the first template.

The image recognition method can be used for triggering specific functions, such as functions of identity recognition, unlocking, payment and the like, and the image recognition result can influence the execution of the functions. That is to say, the reference image is a base image used for image comparison with at least one of the functions of identification, unlocking and payment, or an image corresponding to a base characteristic used for image comparison.

In step S101, an image to be recognized is acquired.

In this embodiment, taking the execution subject as the terminal as an example, the image to be recognized may be collected by an image collecting element of the terminal executing the method, so that the image to be recognized collected by the image collecting element may be obtained when the image to be recognized is obtained. For example, if the terminal executing the method is a smart phone, the image to be recognized acquired by a camera of the smart phone can be acquired. It can be understood that there are many ways to acquire the image to be recognized, and the above example is not a limitation on the manner of acquisition.

In this embodiment, the image to be identified may be obtained according to the type of the image, that is, only the type of the image targeted by the image identification method may be obtained. For example, in the image recognition method, for a face image, the image to be recognized acquired in this step is the face image. The method comprises the steps of acquiring an image to be identified, acquiring a preview scene of an image acquisition element, and acquiring the image by the image acquisition element to serve as the image to be identified when an identification result shows that the image corresponding to the scene corresponds to the image type targeted by the image identification method. For example, the image recognition method is directed at a face image, a preview scene of an image acquisition element can be recognized, when a recognition result shows that a face exists in the scene, the image acquisition element is controlled to acquire an image, and the image is acquired as an image to be recognized.

In step S102, in response to that the image to be recognized and a first template are in a specific relationship, a second template matched with the first template is obtained, where the first template is a reference image used for image recognition or a feature of the reference image, and the first template and the second template satisfy a preset similarity relationship.

The image to be recognized having a specific relationship with the first template may refer to an image that is difficult to be recognized accurately, for example, an image having a high similarity with a reference image. The first template and the second template are of the same type, namely the first template is a reference image, the second template is also an image, the first template is the characteristic of the reference image, and the second template is also the image characteristic; the preset similarity relationship between the first template and the second template may be that the numerical values of the similarity have a certain relationship, or the dimensions of the similarity have a certain relationship. These specific images may be images of relatives, such as siblings like images of twins, images of parents and children, etc. The second template is the image to be recognized in a specific relation with the reference image or the characteristic of the image to be recognized in a specific relation with the reference image. The specific relationship is different, and the second template is of different kinds, for example, the specific relationship is a relative relationship, and the second template is a relative template, and for example, the specific image is a twins relationship, and the second template is a brother template, which can be represented by a Bro-template.

In addition, the second template may be pre-established and stored in the terminal executing the method, and the second template may be obtained according to the storage path of the second template in this step.

In step S103, an identification result corresponding to the image to be identified is obtained based on the image to be identified, the first template, and the second template.

In this embodiment, the second template may improve the recognition accuracy of the specific image, so that when the image to be recognized is the specific image, the second template is combined with the first template to perform recognition.

Optionally, the image to be recognized is recognized in the following manner: firstly, acquiring a third similarity between the image to be recognized or the characteristics of the image to be recognized and the first template and the second template respectively; next, in response to that a third similarity with the first template is greater than or equal to a third similarity with the second template, determining that the recognition result is that the image to be recognized is matched with the reference image; finally, in response to the third similarity with the first template being smaller than the third similarity with the second template, determining that the recognition result is that the image to be recognized does not match the specific image. The similarity may be determined based on euclidean and/or hamming distances between features. By comparing the image to be recognized or the third similarity between the characteristic of the image to be recognized and the first template and the second template, whether the image to be recognized is an image matched with the reference image can be accurately distinguished, for example, in a scene of face image recognition, if the specific relationship is a twins relationship, the face image to be recognized can be accurately distinguished as the face image of the person or the face image of a sibling brother and sister, so that the recognition error of the image can be avoided, for example, the face image of the relatives or the twins is recognized as the face image of the person.

It can be understood that, when the second template is combined on the basis of the first template to identify the image to be identified, the condition that the second template exists needs to be satisfied, because the operation of combining the second template cannot be realized without the second template.

Therefore, when the image to be recognized is not in a specific relationship with the first template, and/or when the second template matching with the first template does not exist, a recognition result corresponding to the image to be recognized is obtained based on the image to be recognized and the first template. That is, the similarity between the feature of the image to be recognized and the first template may be directly determined, and the similarity is compared with a preset similarity threshold, where the image to be recognized matches (e.g., passes) the reference image when the similarity between the feature of the image to be recognized and the first template is greater than or equal to the similarity threshold, and otherwise does not match (e.g., fails) the reference image.

In addition, when the image recognition method is a triggering step of some functions of the terminal, that is, the reference image is a base library image used for image comparison of some functions or an image corresponding to base library features used for image comparison, so that the function related to image recognition can be executed when the recognition result is that the reference image is matched with the reference image. For example, in face recognition, the above functions may be performed when face recognition passes. The functions related to image recognition may be identification, unlocking, payment, etc. When the function related to the image recognition is the identity recognition, namely the reference image is the image of the bottom library used for the image comparison of the identity recognition function, or the image corresponding to the characteristics of the bottom library used for the image comparison, in response to the recognition result being matched with the reference image, the content for representing that the identity recognition result is passing the identity authentication can be output; the method comprises the steps of prompting a user that a currently recognized image is an image of the user, and outputting the image in a display screen display mode or a loudspeaker voice broadcasting mode; when the function related to image recognition is unlocking, namely when the reference image is an image of a base library used for image comparison of the unlocking function or an image corresponding to the base library characteristic used for image comparison, the screen can be unlocked and/or the operation authority of the specified function can be unlocked in response to the recognition result of matching with the reference image; when the function related to image recognition is payment, namely the reference image is the image of the bottom library used for image comparison of the payment function or the image corresponding to the characteristics of the bottom library used for image comparison, the payment can be made to the target account in response to the recognition result of matching with the reference image.

According to the embodiment, the image to be recognized is obtained, the second template matched with the first template is obtained in response to the specific relationship between the image to be recognized and the first template, the first template is a reference image used for image recognition or the characteristics of the reference image, the first template and the second template meet the preset similarity relationship, and finally the recognition result corresponding to the image to be recognized is obtained based on the image to be recognized, the first template and the second template. When the image to be identified is identified, the second template is combined to identify on the basis of the first template, so that the probability of mistakenly identifying the specific image can be reduced, and the identification accuracy of the image is improved. For example, when the face image of the relative is identified, the face identification precision can be improved, and the probability of identifying the face image of the relative as the face image of the user by mistake is reduced.

In some embodiments of the present application, in the case that the image to be recognized and the first template are in the specific relationship, the generation of the second template may also be triggered in at least one of the following manners:

in a first mode, in response to the absence of a second template matching the first template, the second template is generated based on the image to be recognized. The absence of the second template means that the second template is not stored in the main body of the terminal or the like that performs the image recognition method, i.e., the second template is not generated before or the second template generated before is invalidated. The image recognition method improves the recognition accuracy by combining the second template, so that the template establishment instruction can be automatically triggered under the condition that the second template does not exist, namely the second template is automatically generated, so that the recognition accuracy is improved. It is understood that when the second template is stored in the main body such as a terminal that executes the image recognition method, the second template does not need to be generated, and thus the template establishment instruction is not generated. The method can generate the second template in time at the initial running stage of the method so as to improve the image recognition accuracy.

Alternatively, the condition for invalidating the second template may be preset for the terminal, for example, the duration of the second template being not invoked exceeds a time threshold, for example, the remaining amount of storage space of the terminal is lower than a space threshold, for example, the user invalidates the second template directly by an operation. And under the condition that the second template exists in the terminal, the terminal continuously acquires the non-called duration of the second template, the residual amount of the storage space of the terminal and the operation of the user for abandoning the second template, and when one of the duration and the residual amount of the storage space of the terminal reaches the preset condition, the second template is abandoned.

In a second mode, in response to receiving a trigger instruction for generating a second template, the second template is generated based on the image to be recognized. The trigger instruction may be generated based on an operation input by the user, that is, the operation of the user triggers the generation of the second template, for example, the user may operate a physical or virtual button of the terminal executing the image recognition method to generate a template establishing instruction; as another example, the method may provide a paternity mode, a twinned tire mode, etc. that generates a trigger instruction when a user turns these modes on. The method can meet the requirements of users, namely, the users have a scene of relatives identification or a scene of twins identification in the using process, or the users often have wrong identification in the using process, and can autonomously trigger the establishment of the second template, namely, autonomously improve the accuracy of image identification; and when the user does not have the above-mentioned need, the load and power consumption of the main body such as the terminal which executes the method can be reduced.

In a third mode, in response to the existing second template meeting an update period or an update condition, the second template is updated based on the image to be recognized. The updating of the second template can continuously improve the accuracy of image recognition. The update period may be determined automatically by the method or may be preset by the user. The update condition may be automatically determined by the method, or may be preset by the user, for example, the update condition is that the number of identification times reaches a preset number. The method can continuously improve the second template and overcome the inaccurate position of the second template, thereby continuously improving the image recognition precision.

The three modes can be operated independently or combined together. For example, the template establishing instruction can be generated in a first mode in the initial stage of the method operation, the second template can be updated in a third mode after the second template is established, and the user is allowed to manually input the triggering instruction in the whole process of the method operation.

In some embodiments of the present application, before the generating the second template based on the image to be recognized, a stored set of images or a set of features may be obtained, each image in the set of images being in the specific relationship with the first template, and each feature in the set of features being a feature of an image in the specific relationship with the first template. Further, when a second template is generated, the second template may be generated based on the acquired image set and the image to be recognized; the second template may also be generated based on the acquired feature corresponding to each image in the image set or the acquired feature set, and the feature of the image to be recognized.

The image set or the feature set is constructed in advance before the second template is generated, and in the case that the second template does not exist in a terminal for executing the method, images or image features which are in line with the image set or the feature set are screened from the images to be identified while the images to be identified are identified by using the first template, and the image set or the feature set is constructed by using the images or the image features. Optionally, in response to the image to be recognized and the first template being in the specific relationship, and the number of images in the image set being less than a first number threshold, adding the image to be recognized to the image set; in response to the image to be identified being in the particular relationship with the first template and the number of features in the feature set being less than a second number threshold, adding features of the image to be identified to the feature set. The first quantity threshold is the quantity requirement of the image set, the second quantity threshold is the quantity requirement of the feature set, and the first quantity threshold and the second quantity threshold may be the same or different; the first quantity threshold and the second quantity threshold may be determined according to the space occupied by the data storage and/or the requirement for the amount of data to generate the second template, e.g., the images in the set of images are of higher resolution so that the storage space occupied by each image in the set of images is greater than the storage space occupied by each feature in the set of features, so the first quantity threshold may be less than the second quantity threshold.

Since both the image combination and the feature set have respective quantity requirements, when acquiring the image set or the feature set, the image set may be acquired in response to the number of stored images in the image set being greater than or equal to the first quantity threshold; the feature set may also be obtained in response to a number of features in the stored feature set being greater than or equal to the second number threshold.

In this embodiment, the image or the image feature of the second template is generated, and has a certain number requirement, that is, when the image set or the feature set does not meet the number requirement, the image set or the feature set is continuously screened and expanded; after the images are combined or the feature set meets the quantity requirement, a second template can be generated by utilizing the images to be recognized and the images in the image set or the features of the images to be recognized and the features in the feature set. The number requirement may be greater than a number threshold, such as greater than 5. By requiring the number of images or features generating the second template, the accuracy of the second template can be improved, and the accuracy of image recognition can be further improved.

In some embodiments of the present application, it may be determined whether the image to be recognized is in a specific relationship with the first template after the image to be recognized is acquired, so as to generate a second template or perform image recognition by using the second template. The relationship of the image to be recognized to the first template may be determined in a variety of ways, which are not intended to be limiting, and the following illustratively describes three ways of determining whether a particular relationship is present.

Firstly, acquiring a first similarity between the image to be recognized or the characteristics of the image to be recognized and the first template; and then, in response to the first similarity being within a preset range, determining that the image to be recognized and the first template are in the specific relationship. In the related art, when performing image recognition, the recognition result may be determined using a first similarity of the image or the image feature, and the image having a specific relationship with the first template may be a specific image that cannot be accurately recognized using the first similarity. For example, when the first similarity of the feature of the image to be recognized and the first template is higher than or equal to the similarity threshold, the recognition result is determined to be matched with the reference image (for example, the recognition result is passed), and when the similarity is lower than the similarity threshold, the recognition result is determined to be not matched with the reference image, while the image in a specific relationship with the first template is not actually matched with the reference image, but the first similarity with the first template is often higher than the similarity threshold. Therefore, it is reasonable and accurate to select a specific image by using the similarity range, and a certain range may be determined as the similarity range, for example, 0.5-0.8 may be determined as the similarity range. The method further applies the first similarity on the basis of determining the identification result by using the first similarity, can improve the utilization rate of the first similarity, has higher efficiency, and has less load added to the original algorithm and less power consumption added to executing bodies such as the terminal and the like.

In a second mode, an image combination or a feature combination is input into a specific image recognition network, and in response to that the output of the specific relationship recognition network is a specific relationship, an image to be recognized in the image combination or an image to be recognized to which a feature in the feature combination belongs is determined, and the image combination and the first template are in a specific relationship, wherein the image combination comprises the image to be recognized and the reference image, the feature combination comprises the feature of the image to be recognized and the feature of the reference image, and the specific relationship recognition network is pre-trained by using a labeled specific image. The specific relationship identification network can be constructed based on a neural network, can be a classifier, can judge the relationship of input image combinations or feature combinations, and can judge whether a specific relationship exists or not. During pre-training, a certain number of image combinations or feature combinations with specific relationships and a certain number of image combinations or feature combinations without specific relationships can be taken, and all the image combinations or feature combinations are labeled, namely, the image combinations or feature combinations with specific relationships are labeled as the specific relationships, and the image combinations or feature combinations without specific relationships are labeled as the non-specific relationships; then inputting the marked image combination or feature combination into a specific relationship identification network so that the specific relationship identification network outputs a predicted value, namely whether a specific relationship exists or not, comparing the predicted value with a real value (namely marking information, whether the specific relationship exists or not), and adjusting network parameters of the specific relationship identification network by using a comparison result; and continuously iterating according to the training mode until the specific relation recognition network converges or the iteration number reaches the preset requirement. When an image to be recognized is recognized according to an original algorithm in the related technology, the image to be recognized and a reference image (or the image to be recognized and a first template) are input into a specific relationship recognition network, and whether the image to be recognized and the first template are in a specific relationship is judged according to an output result of a model. In the method, the specific relationship recognition network is trained in advance, so that the predicted specific relationship is more accurate, and the predicted result is only the specific relationship (for example, the similarity is within a certain range), so that the magnitude of training data required in the pre-training process is smaller, the magnitude of the training data is not as large as that of the training data for directly training the network model for distinguishing the relatives and the self, and the labeling work is simple and convenient.

In a third mode, firstly, acquiring matching information between the attribute of the image to be identified and the attribute of the reference image; and then, in response to that the matching information of the attribute of the image to be recognized and the attribute of the reference image meets the preset requirement, determining that the image to be recognized or the characteristic of the image to be recognized is in the specific relation with the first template. When the image is a face image, the person characteristics have a large difference in consideration of the person with different skin colors or regions, and therefore, the attributes of the image may include the skin color or the region of the person, and of course, may also include gender, the type of the face, the type of each five sense organs, and the like. Judging the specific relationship by the attributes is simple, and some images with non-specific relationships, such as non-specific relationships between a person with yellow skin color and a person with white skin color, are easily excluded.

The three modes can be operated independently or combined together. For example, the first manner and the second manner may be combined to operate, that is, a specific image is preliminarily screened out through the similarity range, and then an image pair or a feature pair composed of the screened specific image and the reference image is input into the specific relationship identification network to further determine whether the specific relationship exists. For another example, after determining whether a specific relationship exists, the third mode may be further combined, that is, after determining whether the specific relationship exists, the screened specific image attribute and the reference image attribute are used to exclude some non-specific relationship images, for example, if the face image in the specific image determined by the second mode is a man, and the face in the reference image is a woman, the image may be excluded from having the specific relationship with the first template.

In this embodiment, whether the image to be recognized and the first template have a specific relationship is determined through one or more of the above manners, that is, when the image to be recognized is recognized, the recognition work of the specific relationship is also synchronously performed, which is efficient, convenient and accurate in result.

In some embodiments of the present application, a second template may be generated based on the acquired feature corresponding to each image in the image set or the acquired feature set, and the feature of the image to be recognized, as follows.

Firstly, clustering two categories of features corresponding to each image in the image set or each feature in the feature set and the features of the image to be identified; next, obtaining second similarities of the central features of the two categories and the features of the reference image respectively; and finally, determining the central feature corresponding to the highest second similarity in the second similarities or the image to which the central feature corresponding to the highest second similarity belongs as the second template.

When the clustering process is carried out, one or more of a K-means clustering algorithm, a mean shift clustering algorithm, a density-based clustering algorithm, a maximum expectation clustering algorithm of a Gaussian mixture model, a coacervation hierarchical clustering algorithm and a graph group detection clustering algorithm can be adopted.

After the features of all the specific images are subjected to clustering processing of two categories, discrete features (or discrete points) can be removed, so that the distance between two category centers is larger as much as possible. Optionally, first, from features corresponding to each image in the image set or the feature set and features of the image to be recognized, features whose distances to central features of the two categories are both greater than a distance threshold are obtained, and are determined as discrete features, and then the discrete features are removed.

The second template determined by the method is accurate, and the method is convenient and reliable. The second template determined by the mode can be accurately distinguished from the first template, so that the distinguishing and the recognition of the specific image and the reference image are realized, and the probability of false recognition is reduced.

In this embodiment, the second template is determined in the above manner, the screened image set or feature set is fully utilized, a basis is provided for identifying an image in a specific high relationship with the first template, the accuracy of identifying an image in a specific high relationship with the first template is ensured, and the image in a specific high relationship with the first template is prevented from being mistakenly identified as a reference image.

According to a second aspect of the embodiments of the present application, there is provided an image recognition apparatus, referring to fig. 2, which shows a schematic structural diagram of the apparatus, the apparatus includes:

a first obtaining module 201, configured to obtain an image to be identified;

a second obtaining module 202, configured to obtain, in response to that the image to be recognized and a first template are in a specific relationship, a second template matched with the first template, where the first template is a reference image used for image recognition or a feature of the reference image, and the first template and the second template satisfy a preset similarity relationship;

the first identifying module 203 is configured to obtain an identifying result corresponding to the image to be identified based on the image to be identified, the first template, and the second template.

In some embodiments of the present application, in the case that the image to be recognized is in the specific relationship with the first template, the apparatus further comprises at least one of:

In some embodiments of the present application, before the generating the second template based on the image to be recognized, the apparatus further comprises:

In some embodiments of the present application, the apparatus further comprises:

The apparatus also includes a second determining module to:

The apparatus also includes a third determining module to:

In some embodiments of the present application, when the first generation module, the second generation module, or the third generation module is configured to generate the second template based on the acquired feature corresponding to each image in the image set or the acquired feature set, and the feature of the image to be identified, specifically:

In some embodiments of the present application, after the first generation module, the second generation module, or the third generation module is configured to perform two categories of clustering on the feature corresponding to each image in the image set or each feature in the feature set, and the feature of the image to be identified, the method is further configured to:

removing the discrete features.

In some embodiments of the present application, the apparatus further comprises at least one of:

In some embodiments of the present application, the first identification module is specifically configured to:

In some embodiments of the present application, further comprising a second identification module configured to:

In some embodiments of the present application, the reference image is an image of a base library used for image comparison of at least one of identity recognition, unlocking, and payment, or an image corresponding to a characteristic of the base library used for image comparison.

In some embodiments of the present application, further comprising an execution module for at least one of:

With regard to the apparatus in the above-mentioned embodiments, the specific manner in which each module performs the operation has been described in detail in the first aspect with respect to the embodiment of the method, and will not be elaborated here.

According to a third aspect of embodiments of the present application, there is provided an electronic device, as shown in fig. 3, the electronic device includes a memory 301 and a processor 302, the memory 301 is configured to store computer instructions executable on the processor 302, and the processor 302 is configured to perform the image recognition method according to the first aspect when executing the computer instructions. The processor 302 may be connected to the memory 301 through an internal bus 303, and the internal bus 303 may access the network through a network interface 304.

As will be appreciated by one skilled in the art, one or more embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, one or more embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the data processing apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to part of the description of the method embodiment.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

Embodiments of the subject matter and the functional operations described in this specification can be implemented in: digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on a tangible, non-transitory program carrier for execution by, or to control the operation of, data processing apparatus. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode and transmit information to suitable receiver apparatus for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

Computers suitable for executing computer programs include, for example, general and/or special purpose microprocessors, or any other type of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory and/or a random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer does not necessarily have such a device. Moreover, a computer may be embedded in another device, e.g., a mobile telephone, a Personal Digital Assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device such as a Universal Serial Bus (USB) flash drive, to name a few.

Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., an internal hard disk or a removable disk), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. In other instances, features described in connection with one embodiment may be implemented as discrete components or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. Further, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.

The above description is only for the purpose of illustrating the preferred embodiments of the one or more embodiments of the present disclosure, and is not intended to limit the scope of the one or more embodiments of the present disclosure, and any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the one or more embodiments of the present disclosure should be included in the scope of the one or more embodiments of the present disclosure.

In this application, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The term "plurality" means two or more unless expressly limited otherwise.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. An image recognition method, characterized in that the method comprises:

acquiring an image to be identified;

2. The method according to claim 1, wherein in case the image to be recognized is in the specific relationship with the first template, the method further comprises at least one of:

3. The method of claim 2, wherein prior to the generating the second template based on the image to be recognized, the method further comprises:

4. The image recognition method according to any one of claims 1 to 3, further comprising:

5. The image recognition method according to any one of claims 1 to 4, further comprising:

6. The image recognition method according to any one of claims 1 to 5, further comprising:

7. The image recognition method according to claim 3, wherein the generating the second template based on the features corresponding to each of the acquired images or the acquired feature set and the features of the image to be recognized comprises:

8. The image recognition method according to claim 7, wherein after the clustering the features corresponding to each image in the image set or each feature in the feature set and the features of the image to be recognized in two categories, the method further comprises:

removing the discrete features.

9. The image recognition method according to any one of claims 3, 7 and 8, further comprising at least one of:

the acquiring a stored set of images or features includes at least one of:

10. The image recognition method according to any one of claims 1 to 9, wherein obtaining a recognition result corresponding to the image to be recognized based on the image to be recognized, the first template, and the second template includes:

11. The image recognition method according to any one of claims 1 to 10, further comprising:

12. The image recognition method according to any one of claims 1 to 11, wherein the reference image is an image of a base library used for image comparison of at least one of identification, unlocking and payment, or an image corresponding to a characteristic of the base library used for image comparison.

13. The image recognition method of claim 12, further comprising at least one of:

14. An image recognition apparatus, characterized in that the apparatus comprises:

the first acquisition module is used for acquiring an image to be identified;

and the identification module is used for obtaining an identification result corresponding to the image to be identified based on the image to be identified, the first template and the second template.

15. An electronic device, comprising a memory for storing computer instructions executable on a processor, the processor being configured to base the image recognition method of any one of claims 1 to 13 when executing the computer instructions.

16. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the method of any one of claims 1 to 13.