CN115098846A

CN115098846A - Unlocking method, electronic device, storage medium and computer program product

Info

Publication number: CN115098846A
Application number: CN202210451359.5A
Authority: CN
Inventors: 赵博睿; 宋仁杰
Original assignee: Beijing Megvii Technology Co Ltd
Current assignee: TIANJIN JIHAO TECHNOLOGY CO LTD
Priority date: 2022-04-26
Filing date: 2022-04-26
Publication date: 2022-09-23

Abstract

The embodiment of the application provides an unlocking method, electronic equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a current unlocking image corresponding to the current unlocking operation; acquiring at least one previous unlocking image corresponding to at least one previous unlocking operation before the current unlocking operation; searching at least one previous unlocking image for a previous unlocking image with a first similarity larger than a first similarity threshold value; and determining whether to unlock the target base library based on previous unlocking results corresponding to the searched previous unlocking images or based on a current comparison result and previous unlocking results corresponding to one or more searched previous unlocking images, wherein the current comparison result is a comparison result obtained by comparing the current unlocking image with the base library images in the target base library. The accuracy of biological characteristic recognition is improved, and the unlocking success rate is improved.

Description

Unlocking method, electronic device, storage medium and computer program product

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an unlocking method, an electronic device, a storage medium, and a computer program product.

Background

At present, many electronic devices are protected by various unlocking technologies for security, and users need to pass identification verification to unlock the electronic devices. The existing unlocking technologies comprise various unlocking technologies based on biological characteristics, such as human face unlocking, fingerprint unlocking and the like.

Existing unlocking techniques have some limitations in use. For example, the acquisition area or the acquisition performance of the biometric features such as a human face or a fingerprint is limited, so that the accuracy of identifying the biometric features is low, and the success rate of unlocking is also reduced.

Disclosure of Invention

The present application has been made in view of the above problems. The application provides an unlocking method, an electronic device, a storage medium and a computer program product.

According to an aspect of the present application, there is provided an unlocking method, including: acquiring a current unlocking image corresponding to the current unlocking operation; acquiring at least one previous unlocking image corresponding to at least one previous unlocking operation before the current unlocking operation; searching at least one previous unlocking image for a previous unlocking image with a first similarity larger than a first similarity threshold value; and determining whether to unlock the target base library based on previous unlocking results corresponding to the searched previous unlocking images or based on a current comparison result and previous unlocking results corresponding to one or more searched previous unlocking images, wherein the current comparison result is a comparison result obtained by comparing the current unlocking image with the base library images in the target base library.

Exemplarily, the determining whether to unlock the target base library includes determining whether to unlock the target base library based on a previous unlocking result corresponding to each of the current unlocking image and the one or more previous unlocking images, where the determining includes: determining a current unlocking threshold value based on a previous unlocking result corresponding to each of the found one or more previous unlocking images; if the current comparison result has a second similarity larger than the current unlocking threshold, determining that the unlocking is successful; and if the current comparison result does not have the second similarity larger than the current unlocking threshold, determining that the unlocking is failed.

Exemplarily, the determining the current unlocking threshold value based on the previous unlocking result corresponding to each of the one or more found previous unlocking images includes: determining auxiliary unlocking information based on previous unlocking results corresponding to the found one or more previous unlocking images; a current unlock threshold is determined based on the secondary unlock information.

Illustratively, determining the current unlocking threshold based on the secondary unlocking information includes: under the condition that the auxiliary unlocking information indicates that the unlocking is not performed, the initial unlocking threshold value is increased to obtain a current unlocking threshold value; and/or, in the case that the auxiliary unlocking information indicates unlocking, turning down the initial unlocking threshold to obtain the current unlocking threshold.

For example, determining the auxiliary unlocking information based on the previous unlocking result corresponding to each of the one or more found previous unlocking images includes: and if the previous unlocking results with the first preset number or the first preset proportion exist in the previous unlocking results corresponding to the searched one or more previous unlocking images, determining the auxiliary unlocking information as unlocking, otherwise, determining the auxiliary unlocking information as not unlocking.

Illustratively, the previous unlocking result includes a second similarity between the corresponding previous unlocking image and the image of the target base library, and the determining the auxiliary unlocking information based on the previous unlocking result corresponding to each of the one or more found previous unlocking images includes: carrying out weighted average on the second similarity corresponding to the found one or more previous unlocking images to obtain a first average similarity; and if the first average similarity is larger than the second similarity threshold, determining the auxiliary unlocking information as unlocking, otherwise, determining the auxiliary unlocking information as not unlocking.

For example, in the weighted average, for any one of the one or more found previous unlocking images, the closer the execution time of the previous unlocking operation corresponding to the previous unlocking image is to the execution time of the current unlocking operation, the greater the weight of the second similarity corresponding to the previous unlocking image is.

Exemplarily, the previous unlocking result includes a previous comparison result, the current comparison result and the previous comparison result each include a second similarity between the corresponding unlocking image and each of the one or more bottom library images in the target bottom library, and the determining whether to unlock the target bottom library based on the current comparison result and the previous unlocking result corresponding to each of the one or more found previous unlocking images includes: carrying out weighted average on the found one or more previous unlocking images and the second similarity corresponding to the current unlocking image respectively to obtain a second average similarity; comparing the second average similarity to a current unlock threshold; if the second average similarity is larger than the current unlocking threshold, the unlocking is determined to be successful; and if the second average similarity is not larger than the current unlocking threshold, determining that the unlocking is failed.

For example, in the weighted average, the weight of the second similarity corresponding to the current unlock image is the largest, and for any one of the found one or more previous unlock images, the closer the execution time of the previous unlock operation corresponding to the previous unlock image is to the execution time of the current unlock operation, the greater the weight of the second similarity corresponding to the previous unlock image is.

Exemplarily, the current comparison result includes a second similarity between the current unlock image and each of one or more bottom library images in the target bottom library, and after searching for a previous unlock image, of which a first similarity with the current unlock image is greater than a first similarity threshold, from at least one previous unlock image, the method further includes: determining a current unlocking threshold as an initial unlocking threshold under the condition that one or more previous unlocking images are not found; if the current comparison result has a second similarity larger than the current unlocking threshold, determining that the unlocking is successful; and if the second similarity larger than the current unlocking threshold does not exist in the current comparison result, the unlocking failure is determined.

Illustratively, determining whether to unlock the mobile terminal based on the previous unlocking result corresponding to each of the found plurality of previous unlocking images includes one of the following items: if the previous unlocking results corresponding to the found previous unlocking images are successful in unlocking, determining that the unlocking is successful, otherwise determining that the unlocking is failed; if the previous unlocking result corresponding to the previous unlocking images with the second preset number or the second preset proportion exists in the plurality of searched previous unlocking images, the unlocking is determined to be successful, otherwise, the unlocking is determined to be failed; if the previous unlocking result corresponding to the continuous previous unlocking images with the third preset number or the third preset proportion exists in the plurality of searched previous unlocking images, the unlocking is determined to be successful, otherwise, the unlocking is determined to be failed; if the previous unlocking results corresponding to the found previous unlocking images are successful in unlocking, and the found previous unlocking images are previous unlocking images acquired in a first preset time period before the current moment, the unlocking is determined to be successful, otherwise, the unlocking is determined to be failed; if the previous unlocking result corresponding to the previous unlocking images with the fourth preset number or the fourth preset proportion exists in the plurality of searched previous unlocking images, the unlocking is successful, and the previous unlocking images with the fourth preset number or the fourth preset proportion are previous unlocking images acquired in a second preset time period before the current moment, the unlocking is determined to be successful, otherwise, the unlocking is determined to be failed; if the found multiple previous unlocking images have previous unlocking results corresponding to the continuous previous unlocking images with the fifth preset number or the fifth preset proportion, and the continuous previous unlocking images with the fifth preset number or the fifth preset proportion are previous unlocking images acquired in a third preset time period before the current moment, the unlocking is determined to be successful, otherwise, the unlocking is determined to be failed.

After acquiring the current unlocking image corresponding to the current unlocking operation, the method further comprises the following steps: respectively selecting representative images from a plurality of previous image sets or a plurality of bottom libraries which are in one-to-one correspondence with a plurality of different target objects to obtain representative image sets corresponding to the different target objects respectively; comparing the current unlocking image with each representative image set to determine a third similarity between the current unlocking image and each representative image set; selecting a previous image set and a bottom library corresponding to the representative image set with the maximum third similarity; determining an image of the selected set of previous images as at least one previously unlocked image; and determining the selected base library as a target base library.

Illustratively, at least one previous unlock image is stored in the storage unit, and the number of the previous unlock images stored in the storage unit is fixed.

Exemplarily, the method further comprises: determining an image quality of a previous unlock image stored in a storage unit; and deleting the target previous unlocking image and the corresponding previous unlocking result from the storage unit, and storing the current unlocking image and the current unlocking result corresponding to the current unlocking image in the storage unit, wherein the target previous unlocking image is the image with the lowest image quality in the previous unlocking images stored in the storage unit.

Illustratively, the at least one previous unlock image is a previous unlock image acquired within a fourth preset period before the current time.

According to another aspect of the present application, there is provided an electronic device comprising a processor and a memory, wherein the memory has stored therein computer program instructions for executing the above-mentioned unlocking method when the computer program instructions are executed by the processor.

According to another aspect of the present application, a storage medium is provided, on which program instructions are stored, which program instructions are adapted to perform the above-mentioned unlocking method when executed.

According to another aspect of the application, a computer program product is provided, comprising a computer program which is adapted to perform the above-mentioned unlocking method when run.

According to the unlocking method, the electronic device, the storage medium and the computer program product, unlocking judgment can be carried out based on information of a multi-frame unlocking image instead of information of a single-frame unlocking image, so that unlocking can depend on more image information. Therefore, the method is beneficial to improving the accuracy of the biometric identification, and further beneficial to improving the unlocking success rate.

Drawings

The above and other objects, features and advantages of the present application will become more apparent by describing in more detail embodiments of the present application with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings, like reference numbers generally represent like parts or steps.

Fig. 1 shows a schematic block diagram of an example electronic device for implementing an unlocking method and apparatus according to an embodiment of the present application;

FIG. 2 shows a schematic flow diagram of an unlocking method according to an embodiment of the present application;

FIG. 3 shows a flow diagram of an unlocking method according to an embodiment of the present application;

FIG. 4 shows a schematic block diagram of an unlocking device according to one embodiment of the present application;

FIG. 5 shows a schematic block diagram of an electronic device according to one embodiment of the present application.

Detailed Description

In recent years, technical research based on artificial intelligence, such as computer vision, deep learning, machine learning, image processing, and image recognition, has been actively developed. Artificial Intelligence (AI) is an emerging scientific technology for studying and developing theories, methods, techniques and application systems for simulating and extending human Intelligence. The artificial intelligence subject is a comprehensive subject and relates to various technical categories such as chips, big data, cloud computing, internet of things, distributed storage, deep learning, machine learning and neural networks. Computer vision is an important branch of artificial intelligence, particularly a machine is used for identifying the world, and computer vision technology generally comprises the technologies of face identification, unlocking, fingerprint identification and anti-counterfeiting verification, biological feature identification, face detection, pedestrian detection, target detection, pedestrian identification, image processing, image identification, image semantic understanding, image retrieval, character identification, video processing, video content identification, three-dimensional reconstruction, virtual reality, augmented reality, synchronous positioning and map construction (SLAM), computational photography, robot navigation and positioning and the like. With the research and progress of artificial intelligence technology, the technology is applied to many fields, such as security control, city management, traffic management, building management, park management, face passage, face attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile phone images, cloud services, smart homes, wearable equipment, unmanned driving, automatic driving, intelligent medical treatment, face payment, face unlocking, fingerprint unlocking, person certificate verification, smart screens, smart televisions, cameras, mobile internet, live webcasts, beauty treatment, medical beauty treatment, intelligent temperature measurement and the like.

In order to make the objects, technical solutions and advantages of the present application more apparent, exemplary embodiments according to the present application will be described in detail below with reference to the accompanying drawings. It should be apparent that the described embodiments are merely some embodiments of the present application and not all embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the application described in the application without inventive step, shall fall within the scope of protection of the application.

As described above, since the acquisition area or the acquisition performance of the biometric features such as the face or the fingerprint is limited, the accuracy of identifying the biometric features is low, and the success rate of unlocking is also reduced. The fingerprint unlock will be described as an example. The fingerprint unlocking comprises a fingerprint identification process, namely, whether two fingerprint images belong to the same finger or not is distinguished, and the fingerprint unlocking can be carried out only if the two fingerprint images belong to the same finger. Fingerprint unlocking technology is often applied to the field of intelligent terminals such as mobile phones. Nowadays, many manufacturers propose to place the fingerprint unlocking module at the side of the mobile phone, which brings great challenges to the fingerprint unlocking technology. Side fingerprint unblock is compared in traditional fingerprint unblock, and main difficult point lies in: 1. the image area is smaller; 2. the aspect ratio of the image is too large and the signal distribution is not uniform.

In order to at least partially solve the technical problem, embodiments of the present application provide an unlocking method, an electronic device, a storage medium, and a computer program product. According to the unlocking method, the unlocking information of a plurality of frames of unlocking images before and after are integrated, and the current unlocking operation is judged based on the integration information rather than the information of a single frame of unlocking image, so that the unlocking can depend on more image information, the accuracy of fingerprint, face and other biological feature identification is improved, and the unlocking success rate is improved. Experiments show that when the technology is applied to the field of fingerprint unlocking, the accuracy of fingerprint identification, particularly the accuracy of side fingerprint identification, can be obviously improved.

Although the problems of the prior art unlocking technology are further described above by taking fingerprint unlocking, especially side fingerprint unlocking as an example, it can be understood that the unlocking technology according to the embodiment of the present application can be applied to any field needing unlocking, including but not limited to various technical fields of unlocking based on human faces, fingerprints, irises or other features. When the unlocking method described herein is applied to the field of fingerprint unlocking, the unlocking method may be applied to contact fingerprint unlocking and may also be applied to non-contact (e.g., optical) fingerprint unlocking. When the unlocking method described herein is applied to the field of fingerprint unlocking, the unlocking method may be applied to front fingerprint unlocking as well as side fingerprint unlocking.

First, an example electronic device 100 for implementing an unlocking method and apparatus according to an embodiment of the present application is described with reference to fig. 1.

As shown in fig. 1, electronic device 100 includes one or more processors 102, one or more memory devices 104. Optionally, the electronic device 100 may also include an input device 106, an output device 108, and an image capture device 110, which may be interconnected via a bus system 112 and/or other form of connection mechanism (not shown). It should be noted that the components and structure of the electronic device 100 shown in fig. 1 are exemplary only, and not limiting, and the electronic device may have other components and structures as desired.

The processor 102 may be implemented in at least one hardware form of a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a microprocessor, the processor 102 may be one or a combination of several of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Application Specific Integrated Circuit (ASIC), or other forms of processing units having data processing capability and/or instruction execution capability, and may control other components in the electronic device 100 to perform desired functions.

The storage 104 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. On which one or more computer program instructions may be stored that may be executed by processor 102 to implement the client functionality (implemented by the processor) of the embodiments of the application described below and/or other desired functionality. Various applications and various data, such as various data used and/or generated by the applications, may also be stored in the computer-readable storage medium.

The input device 106 may be a device used by a user to input instructions and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.

The output device 108 may output various information (e.g., images and/or sounds) to an external (e.g., user), and may include one or more of a display, a speaker, and the like. Alternatively, the input device 106 and the output device 108 may be integrated and implemented using the same interactive device (e.g., a touch screen).

The image capture device 110 may capture images and store the captured images in the storage device 104 for use by other components. The image capture device 110 may be a separate camera or a camera in a mobile terminal, etc. It should be understood that the image capture device 110 is merely an example, and the electronic device 100 may not include the image capture device 110. In this case, other devices having image capturing capabilities may be used to capture an image and transmit the captured image to the electronic device 100.

For example, an example electronic device for implementing the unlocking method and apparatus according to the embodiments of the present application may be implemented on a device such as a personal computer or a remote server.

Next, an unlocking method according to an embodiment of the present application will be described with reference to fig. 2. FIG. 2 shows a schematic flow diagram of an unlocking method 200 according to one embodiment of the present application. As shown in fig. 2, the unlocking method 200 includes steps S210, S220, S230, and S240.

In step S210, a current unlock image corresponding to the current unlock operation is acquired.

The unlock image (including the current unlock image and the previous unlock image) described herein may be a still image or any video frame in a dynamic video. The unlock image may be an original image acquired by the image acquisition device, or may be an image acquired after preprocessing (such as digitizing, normalizing, smoothing, and the like) the original image. The image capturing device may be implemented using a conventional image sensor such as a camera or using other types of sensors other than a camera, such as a thermal fingerprint capturing sensor. Note that the preprocessing of the original image may include an operation of extracting a sub-image containing the target object from the original image acquired by the image acquisition device to obtain an unlock image. The target object may be any object including, but not limited to: a person or a part of a person's body such as a face, a fingerprint, a palm print, a lip print, an iris, etc. Illustratively, the unlock image may be a fingerprint image containing one or more fingerprints of a human body.

In step S220, at least one previous unlock image corresponding to at least one previous unlock operation before the current unlock operation is acquired.

Either one of the current unlocking operation and the previous unlocking operation may be an operation including steps S210 to S240. The current unlock operation is performed for the current unlock image, one previous unlock operation is performed for each previous unlock image.

The number of the at least one previous unlock image may be set to any suitable number as desired. For example, the at least one previous unlock image may be N previous unlock images stored in the storage unit. N may be a preset number, which is a positive integer greater than or equal to 1. For example, in a case where N or more than N previous unlocking operations have been performed before the current unlocking operation, N previous unlocking images among all the previous unlocking images may be determined as at least one previous unlocking image. In a case where N previous unlocking operations have not been performed before the current unlocking operation, all previous unlocking images may be determined as at least one previous unlocking image.

The at least one previous unlocking operation involved in step S220 may be adjacent to each other, i.e. consecutive unlocking operations, or non-consecutive unlocking operations. For example, each time an unlocking operation is performed based on a current unlocking image, after determining whether unlocking is possible, the current unlocking image may be stored in the storage unit, and simultaneously, an unlocking result corresponding to the current unlocking image may also be stored. In this way, when a new unlocking operation is performed next time, the previously stored unlocking image can be used as the previous unlocking image to assist in determining whether unlocking is currently possible. For another example, the unlock images may be filtered, and only the unlock images with better image quality may be stored, while the unlock images with poor quality may be discarded. The next time a new unlocking operation is performed, the auxiliary judgment is made based on the previously stored unlocking image of good quality. In this case, at least one previous unlocking operation may be non-consecutive.

In one example, a previous unlocking image corresponding to a previous unlocking operation adjacent to the current unlocking operation may be acquired in step S220. That is, only the previous unlock image corresponding to the previous unlock operation closest to the current unlock operation may be acquired at a time, and then the current unlock operation may be subjected to the auxiliary determination based on the previous unlock image.

In step S230, a previous unlock image having a first similarity greater than a first similarity threshold with the current unlock image is searched from at least one previous unlock image.

In a first example, for a current unlock operation, it may be determined whether to unlock currently based only on a previous unlock image. In this case, a plurality of previous unlock images having a first similarity greater than a first similarity threshold with the current unlock image may be searched through step S230. If the previous unlocking images cannot be found, whether unlocking is performed currently or current unlocking failure is determined directly based on the current comparison result. In a second example, for a current unlock operation, it may be determined whether the current unlock is unlocked in conjunction with a current unlock image and a previous unlock image. In this case, one or more previous unlock images having a first similarity greater than a first similarity threshold with the current unlock image may be searched through step S230. If the one or more previous unlocking images cannot be found, whether unlocking is currently performed or current unlocking failure is directly determined based on the current comparison result optionally.

The plurality of previous unlock images looked up in the first example and the one or more previous unlock images looked up in the second example described above may be partially the same, all the same, or all the different.

The number of the plurality of previous unlock images searched in the first example and the number of the one or more previous unlock images searched in the second example described above may be set to any suitable number as needed. In one example, the number of the plurality of previous unlock images looked up in the first example or the number of the one or more previous unlock images looked up in the second example is preset, fixed. For example, the plurality of previous unlock images looked up in the first example may be M1 previous unlock images selected from the above N previous unlock images, and the one or more previous unlock images looked up in the second example may be M2 previous unlock images selected from the above N previous unlock images. M1 is a positive integer greater than or equal to 2 and M1 is less than or equal to N. For example, M1 may be 2, 3, 4, etc. M2 is a positive integer greater than or equal to 1 and M2 is less than or equal to N. For example, M2 may be 1, 2, 3, etc. In this case, if there are M1 or more than M1 previous unlock images among the at least one previous unlock image, and the similarity between the current unlock image and the current unlock image (which is referred to herein as the first similarity for distinction) is greater than the first similarity threshold, M1 previous unlock images among them may be found as the plurality of previous unlock images to which the first example relates. If the first similarity between the M1 previous unlock images and the current unlock image is not greater than the first similarity threshold in the at least one previous unlock image, it may be considered that the required multiple previous unlock images cannot be found. The searching manner of one or more previous unlocking images related to the second example is similar to the searching manner of the plurality of previous unlocking images related to the first example, and the description thereof is omitted.

In another example, the number of the plurality of previous unlock images referred to by the first example or the one or more previous unlock images referred to by the second example may be determined according to the number of previous unlock images of the at least one previous unlock image having a first similarity with the current unlock image greater than a first similarity threshold, that is, how many previous unlock images correspond to the first similarity greater than the first similarity threshold, and how many previous unlock images referred to by the first example or the one or more previous unlock images referred to by the second example. Of course, the number of previous unlock images of the first example cannot be satisfied if there is at most only one previous unlock image of the at least one previous unlock image having a first similarity with the current unlock image that is greater than the first similarity threshold.

In this context, calculating the similarity between two images may be accomplished using any suitable existing or future similarity calculation method. For example, the features of the two images may be extracted separately by a neural network, and the similarity between the extracted features is calculated, and the similarity between the two images is represented by the similarity.

In step S240, it is determined whether to unlock based on previous unlocking results corresponding to the found previous unlocking images, or based on a current comparison result and previous unlocking results corresponding to the found one or more previous unlocking images, where the current comparison result is a comparison result obtained by comparing the current unlocking image with the library images in the target library.

The base library images in the target base library may be template images of authorized users of the electronic device. If the current unlocking image is successfully matched with the bottom library image, the current user can be considered as an authorized user, and then unlocking can be carried out.

When the unlocking method 200 is applied to the field of fingerprint unlocking, the target base library may be a base library corresponding to a single finger, that is, the target base library only contains a fingerprint image corresponding to the single finger. When the number of images in the bottom library is large, the images are classified and stored according to fingers, so that subsequent comparison is facilitated, and the comparison efficiency can be improved. Alternatively, the target base library may also be a base library corresponding to a plurality of fingers, that is, the target base library may simultaneously contain fingerprint images corresponding to a plurality of fingers.

The number of the base library images in the target base library may be one or more. In the case where the target chassis includes a single chassis image, the current unlock image may be directly compared therewith. In the case where the target base includes a plurality of base images, the current unlock image may be compared with the plurality of base images one by one. The way of comparison may include calculating a similarity (referred to as a second similarity) between the current unlock image and the base image. The current comparison result may include a second similarity between the current unlock image and each of the one or more base images.

The one or more previously unlocked images are images having a first similarity greater than a first similarity threshold with the currently unlocked image, that is, the previously unlocked images and the currently unlocked image can be regarded as a biometric feature of the same person, such as a face, a fingerprint, an iris, and the like of the person. Therefore, the previous unlocking results of the plurality of previous unlocking images may be combined with each other, or the previous unlocking results of the previous unlocking images may be combined with the current comparison result of the current unlocking image, i.e., the current unlocking result may be determined by combining the historical unlocking information.

By adopting the unlocking method, whether unlocking is performed currently can be determined according to the unlocking information of a plurality of previous unlocking images similar to the current unlocking image under the current unlocking operation; or, in addition to considering the comparison result between the current unlocking image and the bottom library image, combining the unlocking information of one or more previous unlocking images similar to the current unlocking image to comprehensively determine whether the current unlocking result is the unlocking result. That is, this scheme can make the unlocking determination based on the information of the multiple-frame unlocking image rather than the information of the single-frame unlocking image, so that the unlocking can rely on more image information. Therefore, the method is beneficial to improving the accuracy of the biometric identification, and further beneficial to improving the unlocking success rate. For example, the unlocking method according to the embodiment of the present application may be implemented in a device, an apparatus, or a system having a memory and a processor.

The unlocking method according to the embodiment of the application can be deployed at an image acquisition end, for example, at a personal terminal or a server end.

Alternatively, the unlocking method according to the embodiment of the present application may also be distributively deployed at the server side (or the cloud side) and the personal terminal. For example, an unlocking image can be collected at the client, the client transmits the collected unlocking image to the server (or the cloud), and the server (or the cloud) performs unlocking judgment.

In one example, in step S240, the current unlock image may be compared with a base library image in the target base library to obtain a current comparison result. Subsequently, whether to unlock may be determined based on the current comparison result and a previous unlocking result corresponding to each of the one or more previous unlocking images.

In this example, the comparison result of the current unlocking image and the bottom library image and the unlocking information of the previous unlocking image are used for joint judgment, so that the unlocking success rate can be better improved, and the unlocking performance in application scenes such as side fingerprint identification can be better improved.

According to an embodiment of the present application, the determining, based on the current comparison result and previous unlocking results corresponding to the found one or more previous unlocking images, whether to unlock the target base library may include: determining a current unlocking threshold value based on a previous unlocking result corresponding to each of the found one or more previous unlocking images; if the second similarity larger than the current unlocking threshold exists in the current comparison result, the unlocking is determined to be successful; and if the second similarity larger than the current unlocking threshold does not exist in the current comparison result, the unlocking failure is determined.

The unlocking results of the previous unlock image may be used to determine the current unlock threshold. The current unlocking threshold is a judgment threshold for judging whether the current unlocking image is matched with the bottom library image. For example and without limitation, if there is a second similarity between any previous unlock image and the current unlock image that is greater than the current unlock threshold, the unlock may be performed, and otherwise the unlock may not be performed.

Exemplarily, the determining the current unlocking threshold value based on the previous unlocking result corresponding to each of the one or more found previous unlocking images includes: and determining a current unlocking threshold value directly based on the previous unlocking result corresponding to the searched one or more previous unlocking images respectively.

Exemplarily, the determining the current unlocking threshold value based on the previous unlocking result corresponding to each of the one or more found previous unlocking images includes: determining auxiliary unlocking information based on previous unlocking results corresponding to the found one or more previous unlocking images respectively; a current unlock threshold is determined based on the secondary unlock information.

The previous unlocking results corresponding to the found one or more previous unlocking images can be integrated to determine auxiliary unlocking information, and then the current unlocking threshold value is determined based on the auxiliary unlocking information. The comprehensive mode can be set as required, for example, whether the auxiliary unlocking information is unlocked or not can be determined by counting the number or proportion of one or more previous unlocking results indicating successful unlocking or failed unlocking. Optionally, an average of similarities included in one or more previous unlocking results may also be calculated to determine the auxiliary unlocking information. These will be described below.

When the image is compared with the bottom library, whether the corresponding unlocking image is matched with the bottom library image, namely whether the unlocking image belongs to the bottom library or not is generally judged through an unlocking threshold value, and whether the unlocking can be carried out or not is judged. Therefore, the current unlocking threshold value is determined through the auxiliary unlocking information, the threshold value setting mode can conveniently integrate the unlocking information of the previous unlocking image into the judgment of the current unlocking operation, and the realization is simple.

According to an embodiment of the application, determining the current unlocking threshold based on the auxiliary unlocking information includes: under the condition that the auxiliary unlocking information indicates that the unlocking is not performed, the initial unlocking threshold value is increased to obtain a current unlocking threshold value; and/or, in the case that the auxiliary unlocking information indicates unlocking, turning down the initial unlocking threshold to obtain the current unlocking threshold.

The current unlock operation may have its own initial unlock threshold. For example, and without limitation, the initial unlock threshold may be a preset, fixed value.

Optionally, in a case that the auxiliary unlocking information indicates not to unlock, the initial unlocking threshold may also be directly determined as the current unlocking threshold (i.e., the initial unlocking threshold is not adjusted). Of course, alternatively, in the case that the auxiliary unlocking information indicates unlocking, the initial unlocking threshold may also be directly determined as the current unlocking threshold.

Note that for both cases where the auxiliary unlocking information indicates no unlocking and unlocking, the initial unlocking threshold will not be determined as the current unlocking threshold at the same time. That is, if the initial unlocking threshold is turned up to obtain the current unlocking threshold when the auxiliary unlocking information indicates not to unlock, the initial unlocking threshold may be turned down to obtain the current unlocking threshold or determined as the current unlocking threshold when the auxiliary unlocking information indicates to unlock. Or, if the initial unlocking threshold is turned down to obtain the current unlocking threshold under the condition that the auxiliary unlocking information indicates unlocking, the initial unlocking threshold may be turned up to obtain the current unlocking threshold or determined as the current unlocking threshold under the condition that the auxiliary unlocking information indicates not unlocking.

In the case that the auxiliary unlocking information of the previous unlocking image indicates unlocking, the threshold value of comparison with the bottom library is reduced, so that the current unlocking image is more likely to be considered to be matched with the bottom library, and can be unlocked. On the contrary, in the case that the auxiliary unlocking information of the previous unlocking image indicates that the unlocking is not performed, the threshold value compared with the bottom library is reduced, so that the current unlocking image is more likely to be regarded as unmatched with the bottom library and cannot be unlocked. Therefore, whether the current unlocking operation can be successfully unlocked or not can be determined more quickly and accurately by taking the unlocking information of the similar previous unlocking image as a reference.

According to the embodiment of the application, the determining of the auxiliary unlocking information based on the found previous unlocking result corresponding to each of the one or more previous unlocking images comprises: and if the previous unlocking results with the first preset number or the first preset proportion exist in the previous unlocking results corresponding to the found one or more previous unlocking images, determining the auxiliary unlocking information as unlocking, otherwise, determining the auxiliary unlocking information as not unlocking.

The first predetermined number and the first predetermined ratio can be set to any suitable values as needed, and the application is not limited thereto. For example, it may be set that, in the previous unlocking result, at least 50% of the unlocking results are successful in unlocking, and the auxiliary unlocking information may be determined to be unlocked. For another example, it may be set that, in the previous unlocking result, if there is at least one unlocking result that is successful in unlocking, the auxiliary unlocking information may be determined to be unlocked.

The unlocking result of each unlocking operation may include indication information, and the indication information may be used to indicate whether the current unlocking operation is successful. In this case, it is possible to know whether the result of each unlocking operation is successful or unsuccessful, and determine the auxiliary unlocking information based on the information of successful or unsuccessful unlocking.

According to the method and the device, only the number or the proportion of the previous unlocking results indicating the unlocking success or the unlocking failure needs to be counted, the calculation amount of the scheme is small, and the unlocking efficiency is improved.

According to an embodiment of the present application, the previous unlocking result includes a second similarity between the corresponding previous unlocking image and the library image in the target library, and determining the auxiliary unlocking information based on the previous unlocking result corresponding to each of the one or more found previous unlocking images includes: carrying out weighted average on the second similarity corresponding to the found one or more previous unlocking images to obtain a first average similarity; and if the first average similarity is larger than the second similarity threshold, determining the auxiliary unlocking information as unlocking, otherwise, determining the auxiliary unlocking information as not unlocking.

The unlocking result of each unlocking operation may further include a comparison result between the corresponding unlocking image and the bottom library image, and the comparison result may include a second similarity between the corresponding unlocking image and the bottom library image.

In the case where there is one base image in the target base, the second similarity obtained by calculation at the time of base comparison is one, and in the case where there are a plurality of base images in the target base, the second similarity obtained by calculation at the time of base comparison may be a plurality. For any unlocking image, the corresponding unlocking result or the second similarity recorded in the comparison result may include a second similarity between the unlocking image and each background library image in the target background library, or include a second similarity between the unlocking image and a similar background library image in the target background library, or include a second similarity between the unlocking image and a most similar background library image in the target background library. The similar base library image refers to a base library image with a second similarity greater than a corresponding unlocking threshold (for example, a current unlocking threshold corresponding to the current unlocking image), and the most similar base library image refers to a base library image with a highest second similarity with the corresponding unlocking image or a base library image with a highest second similarity with the corresponding unlocking image and a second similarity greater than the corresponding unlocking threshold. That is to say, the number of the second similarities recorded in the unlocking result or the comparison result corresponding to any unlocking image may be one or more.

Preferably, only the second similarity between the corresponding unlock image and the most similar base library image in the target base library may be recorded in the unlock result or the comparison result.

The second similarity corresponding to each of the one or more previous unlock images may be weighted-averaged to obtain the first average similarity. And compares the first average similarity with a similarity threshold (second similarity threshold) to determine whether the secondary unlocking information indicates unlocking or not unlocking.

The scheme for averaging the similarity integrates the unlocking information of the previous unlocking image more accurately, and is beneficial to improving the accuracy of unlocking judgment of the current unlocking image.

According to the embodiment of the application, in the weighted average, for any one of the found one or more previous unlocking images, the closer the execution time of the previous unlocking operation corresponding to the previous unlocking image is to the execution time of the current unlocking operation, the higher the weight of the second similarity corresponding to the previous unlocking image is.

If the acquisition and unlocking time of the previously unlocked image is too long, it may be less consistent with the current acquisition environment and the reliability and validity of the data may be degraded. Therefore, theoretically, the closer to the current unlock image, the more reliable the unlock information of the previous unlock image. The auxiliary unlocking based on the previous unlocking image with close time can be realized by setting the weight of the previous unlocking image closer to the current unlocking operation to be higher, so that the identification accuracy can be further improved.

According to an embodiment of the present application, the determining whether to unlock the target base library includes determining whether to unlock the target base library based on the previous unlocking result corresponding to the current comparison result and the found previous unlocking result corresponding to each of the one or more previous unlocking images, where the previous unlocking result includes a previous comparison result, and the determining includes: carrying out weighted average on the found one or more previous unlocking images and the second similarity corresponding to the current unlocking image respectively to obtain a second average similarity; comparing the second average similarity to a current unlock threshold; if the second average similarity is larger than the current unlocking threshold, the unlocking is determined to be successful; and if the second average similarity is not larger than the current unlocking threshold, determining that the unlocking is failed.

For example, the current unlocking result may include the current comparison result and the current indication information, and the previous unlocking result may include the previous comparison result and the previous indication information. The current comparison result and the previous comparison result each include a second similarity between the corresponding unlock image and each of the one or more base images in the target base. The current indication information and the previous indication information are each used to indicate whether the unlocking is successful.

Several cases of the second similarity included in the comparison result have been described above, and are not repeated here.

Illustratively, the indication information (including the current indication information and the previous indication information) may include first indication information for indicating success of unlocking and second indication information for indicating failure of unlocking. By way of example and not limitation, the first indication information and the second indication information may be represented by numbers 1 and 0, respectively.

For any unlocking image, the comparison result (i.e. the second similarity) between the unlocking image and the bottom library image and finally the information (i.e. the indication information) about whether the unlocking is successful can be obtained. When the unlocking result is stored, the second similarity and the indication information can be stored together, so that the information can be conveniently utilized for auxiliary unlocking in the follow-up process.

In this embodiment, the second similarities corresponding to the previous unlock image and the current unlock image may be averaged together. For example, assuming that for the current unlock image, 3 previous unlock images similar to the current unlock image are obtained by searching, the second similarities corresponding to the 3 previous unlock images and the current unlock image may be averaged to obtain the second average similarity.

After the second average similarity is calculated, the second average similarity is compared with a current unlocking threshold value, and whether unlocking is possible currently is determined based on the comparison result. Alternatively, the current unlock threshold herein may be the initial unlock threshold described herein.

In this embodiment, the second similarity obtained by comparing the previous unlock image with the base library is included in the similarity calculation of the current unlock image, and the integrated second similarity (i.e., the second average similarity) is used as the similarity result of the current unlock image. It can be understood that if the previous unlock image is compared and matched with the bottom library image, by synthesizing their corresponding second similarities, it is possible to pull the similarity result of the current unlock image high, so that the current unlock image is more likely to be considered as matched with the bottom library, and can be unlocked. Conversely, if the previous unlock image and the base image are not matched too much, by synthesizing their corresponding second similarities, it is possible to pull the result of the similarity of the current unlock image low, so that the current unlock image is more likely to be regarded as unmatched from the base, and cannot be unlocked.

Through the scheme, the purpose of determining the unlocking result of the current unlocking image by taking the unlocking information of the similar previous unlocking image as reference can be achieved.

According to the embodiment of the application, in weighted average, the weight of the second similarity corresponding to the current unlocking image is the largest, and for any one of the found one or more previous unlocking images, the closer the execution time of the previous unlocking operation corresponding to the previous unlocking image is to the execution time of the current unlocking operation, the greater the weight of the second similarity corresponding to the previous unlocking image is.

The present embodiment can be understood by referring to the above weight setting manner when calculating the first average similarity, and details are not described here.

According to the embodiment of the present application, the current comparison result includes a second similarity between the current unlocked image and each of the one or more bottom library images in the target bottom library, and after searching for a previous unlocked image from the at least one previous unlocked image, where a first similarity between the current unlocked image and the previous unlocked image is greater than a first similarity threshold (step S230), the method 200 may further include: determining a current unlocking threshold as an initial unlocking threshold under the condition that one or more previous unlocking images are not found; if the second similarity larger than the current unlocking threshold exists in the current comparison result, the unlocking is determined to be successful; and if the current comparison result does not have the second similarity larger than the current unlocking threshold, determining that the unlocking is failed.

If in step S230, a previous unlock image satisfying the requirement that the first similarity between the current unlock image and the previous unlock image is greater than the first similarity threshold is not found in the at least one previous unlock image, which indicates that there is not much correlation between the current unlock image and the previous unlock image, at this time, the initial unlock threshold may be selected to be kept and not adjusted, but the initial unlock threshold is directly used to determine a comparison result between the current unlock image and the library image, so as to determine whether unlocking is possible.

The above describes, with reference to the embodiment, a process of determining whether to unlock based on a previous unlocking result corresponding to each of the current comparison result and the found one or more previous unlocking images.

In one example, in step S240, it may be determined whether to unlock based on previous unlocking results corresponding to each of a plurality of previous unlocking images. At this time, the number of the previous unlocking images is multiple, and whether unlocking is performed or not can be determined directly based on previous unlocking results corresponding to the multiple previous unlocking images without comparing the current unlocking image with the bottom library image in the target bottom library.

In this example, because the current unlocking image does not need to be compared with the bottom library image, the workload of the unlocking system can be greatly saved, and the unlocking rate can be effectively increased.

A process of determining whether to unlock based on previous unlocking results corresponding to each of the found multiple previous unlocking images is described below with reference to an embodiment.

According to the embodiment of the application, whether unlocking is performed is determined based on the previous unlocking result corresponding to each of the found previous unlocking images, wherein the previous unlocking result comprises one of the following items: if the previous unlocking results corresponding to the found previous unlocking images are successful in unlocking, determining that the unlocking is successful, otherwise determining that the unlocking is failed; if the previous unlocking result corresponding to the previous unlocking images with the second preset number or the second preset proportion exists in the plurality of found previous unlocking images is successful in unlocking, the unlocking is determined to be successful, and if not, the unlocking is determined to be failed; if the previous unlocking result corresponding to the continuous previous unlocking images with the third preset number or the third preset proportion exists in the plurality of found previous unlocking images is successful in unlocking, the unlocking is determined to be successful, and otherwise, the unlocking is determined to be failed; if the previous unlocking results corresponding to the found previous unlocking images are successful in unlocking and the found previous unlocking images are previous unlocking images acquired in a first preset time period before the current moment, determining that the unlocking is successful, otherwise, determining that the unlocking is failed; if the previous unlocking result corresponding to the previous unlocking images with the fourth preset number or the fourth preset proportion exists in the plurality of searched previous unlocking images, the unlocking is successful, and the previous unlocking images with the fourth preset number or the fourth preset proportion are previous unlocking images acquired in a second preset time period before the current moment, the unlocking is determined to be successful, otherwise, the unlocking is determined to be failed; if the found multiple previous unlocking images have previous unlocking results corresponding to the continuous previous unlocking images with the fifth preset number or the fifth preset proportion, and the continuous previous unlocking images with the fifth preset number or the fifth preset proportion are previous unlocking images acquired in a third preset time period before the current moment, the unlocking is determined to be successful, otherwise, the unlocking is determined to be failed.

The preset numbers, the preset ratios or the preset time periods can be set to any suitable values according to the needs, which is not limited in the present application.

In the case of determining whether to unlock based on only the previous unlocking result, whether to unlock may be determined based on various schemes. For example, it may be defined that all of the plurality of previous unlock images are successfully unlocked, and the unlocking may be successful only this time. For another example, it may be defined that the unlocking is successful only if the unlocking is successful for a predetermined number or a preset proportion of previous unlocking images. For another example, it may be defined that the unlocking is successful only when the unlocking is successful in a predetermined number or a preset proportion of consecutive previous unlocking images. According to the above-described embodiment, whether or not unlocking is currently performed is determined based on the previous unlocking result corresponding to each of the plurality of previous unlocking images. According to the scheme, the auxiliary unlocking judgment can be performed by using the sequence information of a plurality of front and back frames, the judgment performed by using the information of a single unlocking image is avoided, and the unlocking accuracy and the unlocking success rate can be obviously improved.

In addition, for the above examples, the time period in which the previous unlocking images participating in the determination are located may be further defined, for example, all the previous unlocking images within the past several minutes, or a preset number or a preset proportion of the previous unlocking images, or a preset number or a preset proportion of consecutive previous unlocking images are successfully unlocked, and the unlocking at this time may be considered to be successful. When the time interval between the current unlocking image and the previous unlocking image is short, if the similarity between the current unlocking image and a plurality of or a plurality of continuous previous unlocking images is greater than the threshold value, the continuous operation of the same user can be considered at the moment, and whether unlocking is performed or not can be directly judged according to a plurality of previous unlocking results under the condition. Therefore, the current unlocking image does not need to be compared with the bottom library image, the unlocking speed is favorably improved, and the user experience is further improved.

According to the embodiment of the present application, after acquiring the current unlocking image corresponding to the current unlocking operation (step S210), the method 200 may further include: respectively selecting representative images from a plurality of previous image sets or a plurality of bottom libraries which are in one-to-one correspondence with a plurality of different target objects to obtain a representative image set corresponding to each of the plurality of different target objects; comparing the current unlocking image with each representative image set to determine a third similarity between the current unlocking image and each representative image set; selecting a previous image set and a bottom library corresponding to the representative image set with the maximum third similarity; determining an image of the selected set of previous images as at least one previously unlocked image; and determining the selected base library as a target base library.

The target object may be any biometric object including, but not limited to: face, fingerprint, palm print, lip print, iris, etc. The different target objects may be target objects under different parts of the same person, such as different position fingers of the same person (e.g., index finger, middle finger, etc.). The different target objects may also be target objects under the same part of different persons, e.g. the same position fingers (e.g. both index fingers) of different persons. Of course, different target objects may also be target objects under different parts of different persons.

For example, a base library may be respectively established for 10 fingers of a person, and each time a current unlock image is acquired, a finger corresponding to the current unlock image is determined by the base library comparison and the like, and the current unlock image is correspondingly stored in a storage location corresponding to the finger, so as to form a previous image set corresponding to the finger.

When the current unlocking image is acquired, which finger the current unlocking image belongs to can be judged first, and this can be realized by comparing the current unlocking image with the representative image set corresponding to each finger. The set of representative images corresponding to any finger may be from the base library or the set of previous images corresponding to that finger.

The current unlock image is most similar to the representative image set for which finger, and it is considered to belong to which finger. After determining which finger the current unlock image belongs to, steps S220 to S240 may be performed based only on the previous image set and the base library to which the finger corresponds. According to the scheme, comparison of all previous unlocking images and the bottom library image is not required for each finger, and the calculation amount can be greatly reduced.

According to the embodiment of the present application, after acquiring the current unlocking image corresponding to the current unlocking operation (step S210), the method 200 may further include: for each target object in the plurality of different target objects, taking an image in the previous image set corresponding to the target object as at least one previous unlock image, and taking the base library corresponding to the target object as a target base library, and performing steps S220 to S240.

For example, the current unlock image may be compared with the previous unlock image and the bottom library image corresponding to each finger one by one for a plurality of fingers to determine whether unlocking is possible. In the case of a small number of images to be compared, the scheme does not need to establish a plurality of libraries, and the storage requirement is low.

According to an embodiment of the present application, at least one previous unlock image is stored in the storage unit, and the number of the previous unlock images stored in the storage unit is fixed.

The storage unit stores a fixed number of previous unlock images, and when the stored previous unlock images reach the fixed number, whether to update the stored previous unlock images with the new unlock images may be selected each time a new unlock image is acquired.

According to the embodiment of the application, the unlocking method further comprises the following steps: calculating an image quality of a previous unlock image stored in the storage unit; and deleting the target previous unlocking image and the corresponding previous unlocking result from the storage unit, and storing the current unlocking image and the current unlocking result corresponding to the current unlocking image in the storage unit, wherein the target previous unlocking image is the image with the lowest image quality in the previous unlocking images stored in the storage unit.

Optionally, the step of deleting the target previous unlock image and the corresponding previous unlock result from the storage unit, and storing the current unlock image and the current unlock result corresponding to the current unlock image in the storage unit is performed in a case where the image quality of the target previous unlock image is lower than that of the current unlock image or is directly performed.

The previous unlock image of the worst quality in the storage unit may be replaced with the current unlock image. Of course, the replaced previous unlock image is an image whose similarity with the current unlock image is greater than the first similarity threshold (i.e., both belong to the same target object). In one example, the replacement is performed when the current unlock image is better in quality than the worst quality previous unlock image. In another example, the current unlock image may be directly substituted for the worst quality previous unlock image, regardless of the quality of the current unlock image. The acquisition time of the current unlocking image is closer, so that the current most real biological characteristic state of the user can be represented. Therefore, the current unlocking image is used for replacing the previous unlocking image, the unlocking image stored in the storage unit and the corresponding unlocking result are conveniently and timely updated, and the effectiveness and the accuracy of unlocking are improved.

Image quality may be determined using any suitable quality assessment criteria, for example, image quality may be measured using one or more of the following criteria: a degree of image blur, a degree of occlusion of a target object in an image, a pose of a target object in an image, a size of a target object in an image, and so on. The overall image quality score can be calculated by combining the above criteria, and the image quality can be evaluated based on the size of the image quality score.

Through the mode, the image which is fresh or good in quality and the unlocking result thereof can be kept stored in the storage unit, so that the difficulty of subsequent unlocking is reduced, and the accuracy of subsequent unlocking is further improved.

According to an embodiment of the present application, the at least one previous unlock image is a previous unlock image acquired within a fourth preset time period before the current time.

The fourth preset time period may be set to any suitable value as needed, and the application is not limited thereto. At least one previous unlock image is set as a previous unlock image within a certain period before the current time. If the images are different unlock images acquired within a relatively short time in the past, the probability that the images are the same target object is considered to be high, or the states of the biometrics characteristics of the user are relatively close. Therefore, the image data in the time period is used for unlocking, so that the unlocking accuracy or efficiency is improved.

For example, a previous unlocking image and a corresponding previous unlocking result thereof, which are more than a fifth preset time period from the current time, may be deleted from the storage unit, which is beneficial to saving storage resources of the terminal. The fifth predetermined period of time may be set to any suitable value as needed, and the application is not limited thereto.

According to an embodiment of the present application, step S230 may include: respectively aligning the current unlock image with each of at least one previous unlock image; determining a particular number of previously unlocked images that are most similar based on the alignment result; calculating a first similarity between each of a certain number of previous unlocked images and the current unlocked image, and selecting a plurality of previous unlocked images with the first similarity greater than a first similarity threshold as the searched plurality of previous unlocked images or selecting one or more previous unlocked images with the first similarity greater than the first similarity threshold as the searched one or more previous unlocked images.

The particular number may be any suitable number, which may be set as desired. For example, the specific number may be one.

For each of the at least one previous unlock image, the previous unlock image may be image-aligned, i.e., image-registered, with the current unlock image. The alignment or registration result between the previous unlock image and the current unlock image may reflect the similarity between the previous unlock image and the current unlock image, but the alignment or registration result is a relatively rough similarity estimation and does not represent the exact similarity between the previous unlock image and the current unlock image. Subsequently, a previous unlock image that is most similar to the current unlock image may be found based on the alignment or registration result. At this time, the exact similarity between the most similar previous unlock image and the current unlock image is unknown. Subsequently, the similarity calculation may be performed only on the most similar previous unlocked image and the current unlocked image, and the previous unlocked image with the first similarity larger than the first similarity threshold value is selected. According to the scheme, accurate similarity between all the previous unlocking images and the current unlocking image does not need to be calculated, so that the overall calculation amount is small.

According to the embodiment of the application, the searching for the previous unlocking image with the first similarity larger than the first similarity threshold value with the current unlocking image from the at least one previous unlocking image comprises: calculating a first similarity between the current unlock image and each of the at least one previous unlock image; selecting a plurality of previous unlocked images with the first similarity larger than a first similarity threshold from the at least one previous unlocked image as the searched plurality of previous unlocked images or selecting one or more previous unlocked images with the first similarity larger than the first similarity threshold as the searched one or more previous unlocked images.

For each of the at least one previously unlocked image, a similarity between the previously unlocked image and the currently unlocked image may be directly calculated. Then, a previous unlock image may be selected from among the first similarity directly based thereon. The scheme needs to calculate the accurate similarity between all the previous unlocking images and the current unlocking image, and the overall calculation amount is large, but the accuracy is high.

Fig. 3 shows a flow diagram of an unlocking method according to an embodiment of the present application. As shown in fig. 3, an exemplary flow of the unlocking method of the present application may be as follows: firstly, storing former K (K is any positive integer greater than or equal to 1) frame unlocking information in an unlocking sequence, wherein the former K frame unlocking information comprises an original unlocking image and an unlocking result (unlocking/unlocking not or unlocking success/unlocking failure) corresponding to the unlocking image; then, in each unlocking comparison, in addition to the comparison with the bottom library image, a most similar image can be picked out from the previous unlocking images stored in the K frames, and the current unlocking image is compared with the picked-out image (shown as the nth frame in fig. 3); finally, 1, if the comparison between the current unlock image and the previous unlock image is successful, that is, the similarity between the current unlock image and the previous unlock image is greater than the first similarity threshold, it may be considered that there is information coupling between the current unlock image and the previous unlock image, and an auxiliary judgment may be performed according to the unlock result of the previous unlock image, for example: if the unlocking result of the previous unlocking image at the time indicates unlocking, the current unlocking threshold value is reduced; otherwise, the current unlocking threshold value is increased; 2. if the comparison between the current unlock image and the previous unlock image is not successful (fig. 3 does not show this situation), it indicates that there is not too strong correlation between the two images, and whether to unlock or not can be determined only according to the comparison result between the current unlock image and the bottom library image.

The advantages of the unlocking method of the present application are described below in conjunction with table 1. Table 1 shows the recognition effect after applying the unlocking method 200 to a side fingerprint unlocking data set.

TABLE 1 recognition effects after applying the unlocking method 200 to a side fingerprint unlocking dataset

	FA	FRR @ conventional	FRR @ rotation
				Existing methods	23/193896	5.22％	12.45％
The method of the present application	7/193896	5.12％	12.42％

In table 1, FA represents the number of fingers judged to be the same from different fingers when fingerprints are compared, and FRR represents the ratio of fingers judged to be different from the same finger. "FRR @ conventional" denotes FRR when conventional fingerprint images are aligned. "FRR @ rotation" represents the FRR of the rotated fingerprint image when aligned. As can be seen from table 1, the unlocking method provided in the embodiment of the present application can greatly reduce the number of FAs while maintaining the FRR unchanged. It will be appreciated that if FA remains unchanged, FRR will be substantially increased. FA corresponds to security of unlocking and FRR corresponds to success rate of unlocking. Therefore, on the premise of keeping the same safety, the unlocking method has higher unlocking success rate.

According to another aspect of the present application, an unlocking device is provided. Fig. 4 shows a schematic block diagram of an unlocking device 400 according to an embodiment of the present application.

As shown in fig. 4, the unlocking device 400 according to the embodiment of the present application includes a first obtaining module 410, a second obtaining module 420, a searching module 430 and a determining module 440. The various modules may perform the various steps/functions of the unlocking method described above in connection with fig. 2-3, respectively. Only the main functions of the respective components of the unlocking apparatus 400 will be described below, and the details that have been described above will be omitted.

The first obtaining module 410 is configured to obtain a current unlocking image corresponding to a current unlocking operation. The first obtaining module 410 may be implemented by the processor 102 in the electronic device shown in fig. 1 executing program instructions stored in the storage 104.

The second obtaining module 420 is configured to obtain at least one previous unlocking image corresponding to at least one previous unlocking operation before the current unlocking operation. The second obtaining module 420 may be implemented by the processor 102 in the electronic device shown in fig. 1 executing program instructions stored in the storage 104.

The searching module 430 is configured to search for a previous unlock image having a first similarity greater than a first similarity threshold with a current unlock image from at least one previous unlock image. The lookup module 430 may be implemented by the processor 102 in the electronic device shown in fig. 1 executing program instructions stored in the storage 104.

The determining module 440 is configured to determine whether to unlock the target chassis based on a previous unlocking result corresponding to each of the found multiple previous unlocking images or based on a previous unlocking result corresponding to the current comparison result and the found one or more previous unlocking images, where the current comparison result is a comparison result obtained by comparing the current unlocking image with a chassis image in the target chassis. The determination module 440 may be implemented by the processor 102 in the electronic device shown in fig. 1 executing program instructions stored in the storage 104.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

FIG. 5 shows a schematic block diagram of an electronic device 500 according to an embodiment of the present application. The electronic device 500 includes a storage (i.e., memory) 510 and a processor 520.

The storage means 510 stores computer program instructions for implementing the respective steps in the unlocking method according to an embodiment of the present application.

The processor 520 is configured to execute computer program instructions stored in the storage device 510 to perform the corresponding steps of the unlocking method according to the embodiment of the present application.

In one embodiment, the computer program instructions, when executed by the processor 520, are operable to perform the steps of: acquiring a current unlocking image corresponding to the current unlocking operation; acquiring at least one previous unlocking image corresponding to at least one previous unlocking operation before the current unlocking operation; searching at least one previous unlocking image for a previous unlocking image with a first similarity larger than a first similarity threshold value; and determining whether to unlock the target base library based on previous unlocking results corresponding to the searched previous unlocking images or based on a current comparison result and previous unlocking results corresponding to one or more searched previous unlocking images, wherein the current comparison result is a comparison result obtained by comparing the current unlocking image with the base library images in the target base library.

Exemplarily, the electronic device 500 may further include an image capture device 530. The image capturing device 530 is used for capturing an unlock image. The image capture device 530 is optional and the electronic device 500 may not include the image capture device 530. In this case, the unlocking image may be captured by another image capturing device and transmitted to the electronic apparatus 500.

In addition, according to an embodiment of the present application, there is also provided a storage medium, on which program instructions are stored, and when the program instructions are executed by a computer or a processor, the program instructions are used to execute corresponding steps of the unlocking method according to the embodiment of the present application, and are used to implement corresponding modules in the unlocking device according to the embodiment of the present application. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), a portable compact disc read only memory (CD-ROM), a USB memory, or any combination of the above storage media.

In one embodiment, the program instructions, when executed by a computer or a processor, may cause the computer or the processor to implement the respective functional modules of the unlocking apparatus according to the embodiment of the present application and/or may perform the unlocking method according to the embodiment of the present application.

In one embodiment, the program instructions are operable when executed to perform the steps of: acquiring a current unlocking image corresponding to the current unlocking operation; acquiring at least one previous unlocking image corresponding to at least one previous unlocking operation before the current unlocking operation; searching at least one previous unlocking image for a previous unlocking image with a first similarity larger than a first similarity threshold value; and determining whether to unlock the target base library based on previous unlocking results corresponding to the searched previous unlocking images or based on a current comparison result and previous unlocking results corresponding to one or more searched previous unlocking images, wherein the current comparison result is a comparison result obtained by comparing the current unlocking image with the base library images in the target base library.

Furthermore, according to an embodiment of the present application, a computer program product is also provided, which includes a computer program, and the computer program is configured to execute the unlocking method 200.

The modules in the electronic device according to embodiments of the present application may be implemented by a processor of the electronic device implementing unlocking according to embodiments of the present application running computer program instructions stored in the memory, or may be implemented when computer instructions stored in a computer-readable storage medium of a computer program product according to embodiments of the present application are run by a computer.

Although the illustrative embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the above-described illustrative embodiments are only exemplary, and are not intended to limit the scope of the present application thereto. Various changes and modifications may be effected therein by one of ordinary skill in the pertinent art without departing from the scope or spirit of the present application. All such changes and modifications are intended to be included within the scope of the present application as claimed in the appended claims.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, a division of a unit is only one type of division of logical functions, and in actual implementation, there may be other divisions, for example, a plurality of units or components may be combined or may be integrated into another device, or some features may be omitted, or not executed.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the description of exemplary embodiments of the present application, various features of the present application are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the application and aiding in the understanding of one or more of the various application aspects. However, the method of this application should not be construed to reflect the intent: this application is intended to cover such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this application.

It will be understood by those skilled in the art that all of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where such features are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that although some embodiments herein include some features included in other embodiments, not others, combinations of features of different embodiments are meant to be within the scope of the application and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

The various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some of the blocks in an unlocking apparatus according to an embodiment of the present application. The present application may also be embodied as apparatus programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present application may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

The above description is only for the specific embodiments of the present application or descriptions thereof, and the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of the changes or substitutions within the technical scope of the present application, and shall cover the protection scope of the present application. The protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An unlocking method, comprising:

acquiring a current unlocking image corresponding to the current unlocking operation;

acquiring at least one previous unlocking image corresponding to at least one previous unlocking operation before the current unlocking operation;

searching for a previous unlocking image with a first similarity larger than a first similarity threshold value with the current unlocking image from the at least one previous unlocking image;

and determining whether to unlock the target base library based on a previous unlocking result corresponding to each of the found previous unlocking images or based on a current comparison result and a previous unlocking result corresponding to each of the found one or more previous unlocking images, wherein the current comparison result is a comparison result obtained by comparing the current unlocking image with a base library image in the target base library.

2. The method of claim 1, wherein the current comparison result comprises a second similarity between the current unlock image and each of one or more base images in the target base library,

the determining whether to unlock based on the current comparison result and previous unlocking results corresponding to the found one or more previous unlocking images includes:

determining a current unlocking threshold value based on a previous unlocking result corresponding to each of the found one or more previous unlocking images;

if the current comparison result has a second similarity which is larger than the current unlocking threshold, determining that the unlocking is successful;

and if the current comparison result does not have the second similarity which is larger than the current unlocking threshold, determining that the unlocking is failed.

3. The method as claimed in claim 2, wherein said determining a current unlocking threshold value based on previous unlocking results corresponding to the found one or more previous unlocking images comprises:

determining auxiliary unlocking information based on previous unlocking results corresponding to the found one or more previous unlocking images respectively;

determining the current unlocking threshold based on the secondary unlocking information.

4. The method of claim 3, wherein the determining the current unlocking threshold based on the secondary unlocking information comprises:

under the condition that the auxiliary unlocking information indicates that unlocking is not performed, an initial unlocking threshold value is increased to obtain the current unlocking threshold value; and/or the presence of a gas in the gas,

and under the condition that the auxiliary unlocking information indicates unlocking, turning down the initial unlocking threshold value to obtain the current unlocking threshold value.

5. The method according to claim 3 or 4, wherein the determining of the auxiliary unlocking information based on the previous unlocking result corresponding to each of the one or more found previous unlocking images comprises:

if the previous unlocking results with the first preset number or the first preset proportion exist in the previous unlocking results corresponding to the searched one or more previous unlocking images, the auxiliary unlocking information is determined to be unlocked, and otherwise, the auxiliary unlocking information is determined not to be unlocked.

6. The method of claim 3 or 4, wherein the previous unlocking result comprises a second similarity between a corresponding previous unlocking image and a bottom library image in the target bottom library, and the determining auxiliary unlocking information based on the previous unlocking result corresponding to each of the one or more found previous unlocking images comprises:

carrying out weighted average on the second similarity corresponding to the found one or more previous unlocking images to obtain a first average similarity;

and if the first average similarity is larger than a second similarity threshold, determining the auxiliary unlocking information as unlocking, otherwise, determining the auxiliary unlocking information as not unlocking.

7. The method according to claim 6, wherein in the weighted average, for any one of the one or more found previous unlock images, the closer the execution time of the previous unlock operation corresponding to the previous unlock image is to the execution time of the current unlock operation, the more the weight of the second similarity corresponding to the previous unlock image is.

8. The method of claim 1, wherein the previous unlock result comprises a previous comparison result, the current and previous comparison results each comprise a second similarity between the corresponding unlock image and each of the one or more base images in the target base,

performing weighted average on the found one or more previous unlocking images and the second similarity corresponding to the current unlocking image respectively to obtain a second average similarity;

comparing the second average similarity to a current unlock threshold;

if the second average similarity is larger than the current unlocking threshold, determining that the unlocking is successful;

and if the second average similarity is not larger than the current unlocking threshold, determining that the unlocking is failed.

9. The method according to claim 8, wherein, in the weighted average, a weight of the second similarity corresponding to the current unlock image is the largest, and for any one of the one or more found previous unlock images, a weight of the second similarity corresponding to the previous unlock image is the larger the closer the execution time of the previous unlock operation corresponding to the previous unlock image is to the execution time of the current unlock operation.

10. The method of any one of claims 1 to 4, wherein the current alignment result comprises a second similarity between the current unlock image and each of one or more bottom library images in the target bottom library,

after the searching for the previous unlock image from the at least one previous unlock image having a first similarity with the current unlock image greater than a first similarity threshold, the method further comprises:

determining a current unlocking threshold value as an initial unlocking threshold value under the condition that one or more previous unlocking images are not found;

if a second similarity larger than the current unlocking threshold exists in the current comparison result, the unlocking is determined to be successful;

11. The method of claim 1, wherein the determining whether to unlock based on the previous unlocking result corresponding to each of the found plurality of previous unlocking images comprises one of:

if the previous unlocking results corresponding to the found previous unlocking images are successful in unlocking, determining that the unlocking is successful, otherwise determining that the unlocking is failed;

if the previous unlocking result corresponding to the previous unlocking images with the second preset number or the second preset proportion exists in the plurality of searched previous unlocking images, the unlocking is determined to be successful, otherwise, the unlocking is determined to be failed;

if the previous unlocking result corresponding to the continuous previous unlocking images with the third preset number or the third preset proportion exists in the plurality of searched previous unlocking images, the unlocking is determined to be successful, otherwise, the unlocking is determined to be failed;

if the previous unlocking results corresponding to the found previous unlocking images are successful in unlocking, and the found previous unlocking images are previous unlocking images acquired in a first preset time period before the current moment, the unlocking is determined to be successful, otherwise, the unlocking is determined to be failed;

if the previous unlocking result corresponding to the previous unlocking images with the fourth preset number or the fourth preset proportion exists in the plurality of searched previous unlocking images and is successful in unlocking, and the previous unlocking images with the fourth preset number or the fourth preset proportion are previous unlocking images acquired in a second preset time period before the current moment, determining that the unlocking is successful, otherwise, determining that the unlocking is failed;

if the found previous unlocking images have previous unlocking results corresponding to the fifth preset number or fifth preset proportion of continuous previous unlocking images, and the fifth preset number or fifth preset proportion of continuous previous unlocking images are previous unlocking images acquired in a third preset time period before the current moment, determining that unlocking is successful, otherwise, determining that unlocking is failed.

12. The method of any of claims 1 to 11, wherein after the obtaining a current unlock image corresponding to a current unlock operation, the method further comprises:

respectively selecting representative images from a plurality of previous image sets or a plurality of bottom libraries which are in one-to-one correspondence with a plurality of different target objects to obtain representative image sets which respectively correspond to the plurality of different target objects;

comparing the current unlocking image with each representative image set to determine a third similarity between the current unlocking image and each representative image set;

selecting a previous image set and a bottom library corresponding to the representative image set with the maximum third similarity;

determining an image of the selected set of previous images as the at least one previously unlocked image;

and determining the selected base library as the target base library.

13. The method of any one of claims 1 to 12, wherein the at least one previous unlock image is stored in a storage unit, the number of previous unlock images stored in the storage unit being fixed.

14. The method of claim 13, wherein the method further comprises:

determining an image quality of a previously unlocked image stored in the storage unit;

deleting a target previous unlocking image and a corresponding previous unlocking result from the storage unit, and storing the current unlocking image and the current unlocking result corresponding to the current unlocking image in the storage unit, wherein the target previous unlocking image is an image with the lowest image quality in the previous unlocking images stored in the storage unit.

15. The method of any one of claims 1 to 14, wherein the at least one previous unlock image is a previous unlock image acquired within a fourth preset time period before the current time.

16. An electronic device comprising a processor and a memory, wherein the memory has stored therein computer program instructions for executing the unlocking method according to any one of claims 1 to 15 when the computer program instructions are executed by the processor.

17. A storage medium on which program instructions are stored, characterized in that the program instructions are adapted to perform the unlocking method according to any one of claims 1 to 15 when executed.

18. A computer program product comprising a computer program, characterized in that the computer program is adapted to perform the unlocking method according to any one of claims 1 to 15 when running.