WO2022022365A1

WO2022022365A1 - Image matching method applied to fingerprint recognition and related device

Info

Publication number: WO2022022365A1
Application number: PCT/CN2021/107689
Authority: WO
Inventors: 雷文超; 常冰; 龙水平; 石腾; 赵晓娜; 黄庆超
Original assignee: 华为技术有限公司
Priority date: 2020-07-31
Filing date: 2021-07-21
Publication date: 2022-02-03
Also published as: CN114078258A

Abstract

An image matching method and a related device. The method comprises: determining, according to morphological feature information of an object to be detected, attribute information of an image to be recognized used for recognizing said object; screening template images according to the attribute information of the image to be recognized, attribute information of the screened template images and the attribute information of the image to be recognized meeting an attribute correlation condition; and querying, in the screened template images, an image matched with the image to be recognized. The method can improve efficiency of image recognition.

Description

An image matching method and related device applied to fingerprint recognition

This application claims the priority of the Chinese patent application filed on July 31, 2020 with the application number 202010762013.8 and the application name "An Image Matching Method and Related Device Applied in Fingerprint Recognition", the entire contents of which are approved by Reference is incorporated in this application.

technical field

The present application relates to the technical field of image recognition, and in particular, to an image matching method and related device applied to fingerprint recognition.

Background technique

With the continuous development of computer technology, image recognition technology has been widely used in various scenes of people's lives. For example, in the field of identity recognition technology and fingerprint recognition technology, recognition technology based on image matching is used.

Image matching can include a template image acquisition stage and an image matching stage. Before image matching, some template images are pre-collected, and after the images to be recognized are collected, the pre-collected template images are queried whether there is an image that matches the image to be recognized. , the matched image may be the same as the to-be-recognized image or an image with a higher similarity. When the images are successfully matched, the object information corresponding to the matched images is used as the result of image recognition.

When the number of pre-collected template images is large, the efficiency of querying matching images is low.

SUMMARY OF THE INVENTION

The present application provides an image matching method and related device applied to fingerprint identification, which can improve the efficiency of image matching.

In a first aspect, an embodiment of the present application provides an image matching method applied to fingerprint recognition, the method comprising:

According to the morphological feature information of the object to be detected, the attribute information of the image to be recognized is determined, wherein the image to be recognized is used to recognize the object to be detected;

Screening at least one template image according to the attribute information of the to-be-recognized image; the attribute information of the filtered template image and the attribute information of the to-be-recognized image satisfy the attribute correlation condition;

In the filtered template image, the image matching the to-be-recognized image is queried.

The morphological feature information may include static and dynamic morphological feature information of the object to be detected.

In an optional implementation manner, the morphological feature information of the object to be detected may include at least one of the following information: contour information, position information, and kinematic information of the object to be detected;

The attribute information of the to-be-recognized image includes at least one of the following information: the object type of the to-be-detected object, the habitual posture information of the to-be-detected object, and the outline of the to-be-recognized image and the to-be-detected object Global location information between messages.

In an optional implementation manner, before determining the attribute information of the to-be-recognized image according to the morphological feature information of the to-be-detected object, the method includes:

obtaining the to-be-recognized image;

acquiring the touch signal of the object to be detected;

According to the touch signal of the object to be detected, determine the morphological feature information of the object to be detected;

Wherein, the time difference between the time when the image to be recognized is obtained and the time when the touch signal of the object to be detected is obtained is less than a time deviation threshold.

In an optional implementation manner, the touch signal of the object to be detected is collected by a touch device;

Before determining the morphological feature information of the to-be-detected object according to the touch-control information of the to-be-detected object, the method further includes: acquiring three-dimensional position information of the touch-control device through a gesture sensor;

The determining the attribute information of the image to be recognized according to the morphological feature information of the object to be detected includes: determining the attribute information of the image to be recognized according to the morphological feature information of the object to be detected and the three-dimensional position information of the touch control device information.

In an optional implementation manner, the object to be detected is a fingerprint part to be recognized; the image to be recognized is a fingerprint image of the fingerprint part to be recognized collected by a fingerprint sensor.

In an optional implementation manner, the object type of the object to be detected includes at least one of the following: left and right hand information, and finger type.

In an optional implementation manner, the global position information between the to-be-identified image and the contour information of the to-be-detected object includes at least one of the following: a region located on the left side of the contour centerline, and a region located on the right side of the contour centerline Region, the region position information with the contour information as the reference system.

In an optional implementation manner, the habitual posture information of the object to be detected includes at least one of the following: global direction information of the object to be detected, the speed of the object to be detected, and the information in the image to be recognized. the moving direction of the object to be detected, the movement track of the object to be detected in the image to be recognized;

and / or,

The global position information between the to-be-recognized image and the outline information of the to-be-detected object includes: when the to-be-recognized image is a partial image of the to-be-detected object, the to-be-recognized image is located in the to-be-detected image The corresponding area location identifier in the outline information of the object.

In an optional implementation manner, the attribute correlation condition includes at least one of the following:

When the attribute information of the to-be-recognized image includes the object type corresponding to the to-be-recognized image, the object type corresponding to the filtered template image is the same as the object type corresponding to the to-be-recognized image;

When the attribute information of the to-be-recognized image includes gesture information corresponding to the to-be-recognized image, the habitual gesture information corresponding to the filtered template image is the same as the habitual gesture information corresponding to the to-be-recognized image;

When the attribute information of the to-be-recognized image includes the global location information corresponding to the to-be-recognized image, the global location information corresponding to the filtered template image and the global location information corresponding to the to-be-recognized image satisfy the location proximity condition;

The location proximity condition includes: the area indicated by the global location information corresponding to the filtered template image contains the global location information corresponding to the to-be-recognized image, or the global location corresponding to the filtered template image There is an intersection between the area represented by the information and the global location information corresponding to the image to be recognized, or the area represented by the global location information corresponding to the filtered template image and the global location information corresponding to the image to be recognized. The distance between the regions is less than the adjacent region distance threshold.

In an optional implementation manner, the number of attribute information of the image to be recognized is at least two;

The screening of at least one template image according to the attribute information of the image to be recognized includes: screening the at least one template image according to at least two attribute information of the to-be-recognized image and the weight of each attribute information .

In an optional implementation manner, when the number of filtered template images is 0, or, in the filtered template images, no target template image matching the to-be-recognized image is queried, the Methods also include:

In the at least one template image except for the other template images in the filtered template image, query the target template image that matches the to-be-recognized image.

According to the ranking of the historical matching success rate corresponding to the at least one template image, in the at least one template image, the image matching the to-be-recognized image is queried.

In an optional embodiment, the method further includes:

When the target template image matching the to-be-recognized image is queried, obtain the template entity identifier and/or use authority corresponding to the target template image;

When no target template image matching the to-be-recognized image is found in the at least one template image, it is determined that the to-be-detected object corresponding to the to-be-matched image does not have the right to use.

In an optional implementation manner, before the at least one template image is screened according to the attribute information of the to-be-recognized image, the method includes:

obtaining the morphological feature information of the at least one template image and the template object corresponding to each template image; each template image is in one-to-one correspondence with each template object;

The attribute information of each template image is determined according to the morphological feature information of the template object corresponding to each template image.

In an optional implementation manner, the object to be detected is the fingerprint part to be recognized; the morphological feature information of the object to be detected is the outline information of the fingerprint part to be recognized; the image to be recognized is the fingerprint part to be recognized. Identify the partial fingerprint image of the fingerprint part;

The determining the attribute information of the to-be-recognized image according to the morphological feature information of the to-be-detected object includes: determining, according to the outline information of the to-be-recognized fingerprint part, the outline information of the partial fingerprint image in the to-be-recognized fingerprint part The region location identifier in ; the region location identifier is used as the attribute information of the local fingerprint image.

In an optional implementation manner, the acquiring the at least one template image and the morphological feature information of the template object corresponding to each template image includes: acquiring P-1 first template images of the first template object, and , the outline information of the first template object corresponding to the P-1 first template images;

The determining the attribute information of each template image according to the morphological feature information of the template object corresponding to each template image includes: determining each first template image according to the outline information of the first template object corresponding to each first template image respectively. The region location identifier in the corresponding contour information;

The method further includes: judging whether the set of area location identifiers corresponding to the P-1 first template images satisfies the coverage condition; if not, acquiring the Pth first template image of the first template object;

Wherein, the covering condition includes: covering at least one local area of the first template object, or covering the edge covering condition of the edge area of the first template object, or satisfying the outline of covering the first template object Global coverage condition of information; P is an integer greater than or equal to 2.

In a second aspect, the present application also provides an image matching method applied to fingerprint identification, the method comprising:

obtaining at least one first template image of the template object;

Obtain the outline information of the template object, wherein, the outline information of the template object is used to determine the region position identifier of the at least one first template image in the outline information of the template object;

When the set of area location identifiers corresponding to the at least one first template image does not satisfy the coverage condition, collecting a second template image of the template object;

Wherein, the at least one first template image and the second template image are used to query the image matching the image to be recognized.

In an optional embodiment, the coverage conditions include at least one of the following:

covering at least one local area of the template object;

Covering the edge area of the template object exceeds a preset coverage ratio;

The outline information covering the template object exceeds a preset coverage ratio.

In this embodiment of the present application, the coverage condition may further include a preset number threshold of template images. For example, the coverage condition may include that the edge area of the template object covered exceeds a preset coverage ratio and the number of template images is greater than a preset number threshold, Alternatively, the coverage condition may include covering the global area of the template object exceeding a preset coverage ratio and the number of template images is greater than a preset number threshold.

In an optional implementation manner, the acquiring the outline information of the template object includes: acquiring the outline information of the template object through a touch screen of an electronic device;

The acquiring at least one first template image of the template object includes: acquiring the at least one first template image through a fingerprint sensor;

The fingerprint sensor is located on a side of the touch screen close to the body of the electronic device.

In an optional implementation manner, before the acquisition of the second template image of the template object, the method further includes:

determining a supplementary mining area according to the set of area location identifiers corresponding to the at least one first template image and the coverage condition;

Prompt information is output, where the prompt information is used to prompt the input of a second template image of the template object including the supplementary acquisition area.

In a third aspect, the present application provides an image matching device applied to fingerprint identification, comprising:

a determination module, configured to determine attribute information of an image to be recognized according to the morphological feature information of the object to be detected, wherein the image to be recognized is used to identify the object to be detected;

a matching module, configured to screen at least one template image according to the attribute information of the to-be-recognized image, wherein the attribute information of the filtered template image and the attribute information of the to-be-recognized image satisfy an attribute correlation condition; and, In the filtered template image, the image matching the to-be-recognized image is queried.

In an optional embodiment, the device further includes:

an acquisition module, configured to acquire the image to be recognized and the touch signal of the object to be detected; and, according to the touch signal of the object to be detected, determine the morphological feature information of the object to be detected; wherein , the time difference between the time when the image to be recognized is obtained and the time when the touch signal of the object to be detected is obtained is less than a time deviation threshold.

In an optional implementation manner, the matching module is specifically configured to filter the at least one template image according to at least two types of attribute information of the to-be-recognized image and the weight of each attribute information.

In an optional implementation manner, the matching module is further configured to: when the number of the filtered template images is 0, or, in the filtered template images, no match with the to-be-recognized image is queried. When the target template image is selected, in the at least one template image other than the filtered template image, the image matching the to-be-identified image is queried.

In an optional implementation manner, the matching module is further configured to: when the number of the filtered template images is 0, or, in the filtered template images, no match with the to-be-recognized image is queried. When the target template image is the target template image, sort according to the historical matching success rate corresponding to the at least one template image, and query the image matching the to-be-recognized image in the at least one template image.

In an optional implementation manner, the matching module is further configured to acquire the template entity identifier corresponding to the target template image when a target template image matching the to-be-identified image is queried and/or use and, when the target template image matching the to-be-recognized image is not queried, it is determined that the to-be-detected object corresponding to the to-be-matched image does not have use rights.

In an optional implementation manner, the acquiring module is further configured to acquire the at least one template image and each template before the at least one template image is screened according to the attribute information of the to-be-recognized image Morphological feature information of the template object corresponding to the image; each template image corresponds to each template object one-to-one;

The determining module is further configured to determine attribute information of each template image according to the morphological feature information of the template object corresponding to each template image.

In an optional implementation manner, the acquiring module is specifically configured to acquire P-1 first template images of the first template object, and the first template images corresponding to the P-1 first template images The outline information of a template object;

The determining module is specifically configured to determine the region position identifier of each first template image in the corresponding contour information according to the contour information of the first template object corresponding to each first template image;

The determining module is further configured to determine whether the set of region location identifiers corresponding to the P-1 first template images satisfies the coverage condition; if not, instructing the obtaining module to obtain the P-th image of the first template object. a first template image; wherein, the coverage condition includes: an edge coverage condition covering the edge region of the first template object, or, satisfying a global coverage condition covering the outline information of the first template object; P is greater than or an integer equal to 2.

In a fourth aspect, the present application provides an image matching device applied to fingerprint identification, comprising:

an acquiring module, configured to acquire at least one first template image of a template object; and, acquiring contour information of the template object, wherein the contour information of the template object is used to determine that the at least one first template image is in the The region location identifier in the outline information of the template object;

a determination module, configured to determine whether the set of area location identifiers corresponding to the at least one first template image satisfies the coverage condition, and instruct the acquisition module to acquire the second template image of the template object when the coverage condition is not met; wherein , the at least one first template image and the second template image are used to query the image matching the image to be recognized.

covering at least one local area of the template object;

Covering the edge area of the template object exceeds a preset coverage ratio;

In an optional implementation manner, the determining module is further configured to determine the supplementary mining area according to the set of area location identifiers corresponding to the at least one first template image and the coverage condition;

The apparatus further includes: an output module for outputting prompt information, the prompt information being used for prompting the input of a second template image of the template object including the supplementary mining area.

In a fifth aspect, the application provides an electronic device, comprising:

an image acquisition device for acquiring an image to be recognized, wherein the image to be recognized is used to recognize an object to be detected;

a morphological feature collecting device, used for collecting morphological feature information of the object to be detected;

a processor, configured to determine attribute information of the to-be-recognized image according to the morphological feature information of the to-be-detected object; and, according to the attribute information of the to-be-recognized image, screen at least one template image, wherein the filtered template An attribute correlation condition is satisfied between the attribute information of the image and the attribute information of the to-be-recognized image; and, in the filtered template image, an image matching the to-be-recognized image is queried.

In an optional implementation manner, the morphological feature collecting device is a touch device.

In an optional implementation manner, the morphological feature collecting device is a touch screen, the image collecting device is a fingerprint sensor, and the fingerprint sensor is disposed on a side of the touch screen close to the body of the electronic device.

In a sixth aspect, the application provides an electronic device, comprising:

an image acquisition device, configured to acquire at least one first template image of a template object; and, when the set of region location identifiers corresponding to the at least one first template image does not satisfy the coverage condition, acquire a second template of the template object an image; wherein, the at least one first template image and the second template image are used to query images that match the image to be recognized;

The touch device is configured to acquire contour information of the template object, wherein the contour information of the template object is used to determine a region position identifier of the at least one first template image in the contour information of the template object.

In an optional implementation manner, the image acquisition device is a fingerprint sensor; the touch control device is a touch screen in an electronic device; the fingerprint sensor is disposed on a side of the touch screen close to the body of the electronic device .

In an optional implementation manner, the touch control device is further configured to output prompt information when the set of area location identifiers corresponding to the at least one first template image does not satisfy the coverage condition, and the prompt information uses upon prompting to input a second template image of the template object through the image acquisition device.

In yet another aspect, the present application provides an electronic device, comprising: a processor and a memory, the memory for storing instructions, the processor for executing the instructions to cause the electronic device to perform the first to second aspects The method of any of the aspects.

In yet another aspect, the present application provides a computer storage medium, comprising a computer program, which, when executed on a computer, causes the computer to perform the method of any one of the first to second aspects.

Description of drawings

In order to illustrate the technical solutions in the present application or the prior art more clearly, the following briefly introduces the accompanying drawings that are used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are the For some embodiments of the application, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1A is a schematic diagram of a processing process of an image matching method provided by an embodiment of the present application;

1B is a schematic diagram 1 of an application scenario of an image matching method provided by an embodiment of the present application;

1C is a second schematic diagram of an application scenario of the image matching method provided by the embodiment of the present application;

1D is a schematic diagram of the global position information of the to-be-recognized image in the outline information of the to-be-detected object in the image matching method provided by the embodiment of the application;

2A is a schematic flowchart of an input stage of an image matching method provided by an embodiment of the present application;

2B is a schematic diagram of determining morphological feature information according to a touch signal in an embodiment of the present application;

2C is a schematic diagram of inferring left and right hand types in an embodiment of the present application;

2D is a schematic diagram of inferring a finger type in an embodiment of the present application;

2E is a schematic diagram of inferring global position information and global direction information in an embodiment of the present application;

3A is a schematic flowchart of a recognition stage of an image matching method provided by an embodiment of the present application;

FIG. 3B is a schematic diagram of an approximate area inferred in an embodiment of the present application;

3C is a schematic diagram of template matching in an embodiment of the present application;

4 is a second schematic flowchart of an image matching method provided by an embodiment of the present application;

5A is a schematic diagram of the position of a template image of an edge region in an image matching method provided by an embodiment of the present application;

5B is a schematic diagram 3 of an application scenario of the image matching method provided by the embodiment of the present application;

FIG. 6 is a schematic diagram of a processing flow of a fingerprint entry stage in an image matching method provided by an embodiment of the present application.

FIG. 7 is a schematic diagram of a processing flow of a fingerprint identification stage in an image matching method provided by an embodiment of the present application.

FIG. 8 is a schematic structural diagram 1 of an image matching apparatus provided by an embodiment of the present application;

FIG. 9 is a second schematic structural diagram of an image matching apparatus provided by an embodiment of the present application;

FIG. 10 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

detailed description

The terms used in the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

The image matching method provided in the embodiment of the present application can be applied to various object recognition scenarios, such as a fingerprint recognition scenario, a face recognition scenario, and other user identity recognition scenarios. Exemplarily, the image to be recognized may be a fingerprint image.

In this embodiment of the present application, the image matching method may involve: an input stage and a recognition stage. The processing in the entry stage includes: entering the template image in the image database, that is, establishing the image database. The processing in the identification stage includes: collecting the image to be identified from the object to be detected, and querying the template image in the image database for the target template image that matches the image to be identified. In this embodiment of the present application, the image matching result may be used to determine one or more types of object information, such as entity identification or usage authority corresponding to the object to be detected from which the image to be recognized is collected.

FIG. 1A is a schematic diagram of a processing process of an image matching method provided by an embodiment of the present application.

As shown in FIG. 1A , in the image matching method provided by the embodiment of the present application, in the input stage, when the template image is collected for the template object, the morphological feature information of the template object may also be collected, and then, according to the morphological feature information of the template object One or more attribute information corresponding to the template image is determined, and then the template image and the corresponding attribute information are stored in the image database. In the recognition stage, when the image to be recognized is collected from the object to be detected, the morphological feature information of the object to be detected can also be collected, and then one or more attribute information corresponding to the image to be recognized is determined according to the morphological feature information of the object to be detected, and then , the attribute information of the image to be recognized can be used to screen many template images in the image database, and the template image matching the image to be recognized can be queried in the filtered template images, which can improve the efficiency of image matching.

For example, the morphological feature information of the template object may include at least one kind of information used to represent the shape, size, position, direction, speed, etc. of the template object, for example, the outline information of the template object, the relative information of the template object relative to the electronic device Position information or position information in the geodetic coordinate system, kinematic information, etc. The attribute information corresponding to the template image may be determined according to the morphological feature information of the corresponding template object, for example, the object type of the template object, the global position information of the template image in the outline information of the template object, the current posture of the template object, the movement track Wait. For example, FIG. 1D is a schematic diagram of global position information of the to-be-identified image in the outline information of the to-be-detected object in the image matching method provided by the embodiment of the present application. As shown in FIG. 1D , the position of the partial fingerprint image to be identified in the contour information of the corresponding finger to be detected. In the embodiments of the present application, the morphological feature information and attribute information and the processing steps of determining the attribute information according to the morphological feature information will be described in detail in combination with practical applications, which will not be repeated here.

The actual application scenarios of the image matching method provided by the embodiments of the present application are exemplarily described below.

In this embodiment of the present application, the image matching method may be deployed in an electronic device having an image capturing device. The electronic device may be a terminal with an image acquisition device, such as a mobile phone, a tablet computer, a handheld identification device, a wearable device, or the like. The image capture device may be a camera or a sensor. Taking a fingerprint recognition scenario as an example, the image acquisition device can be a fingerprint sensor, and the fingerprint sensor can be used to collect template images in the input stage, and to collect images to be recognized in the recognition stage.

In this embodiment of the present application, a touch control device may also be provided on the electronic device. The touch device can be used to acquire the morphological feature information of the template object in the input stage, and obtain the morphological feature information of the object to be detected in the recognition stage.

FIG. 1B is a schematic diagram 1 of an application scenario of the image matching method provided by the embodiment of the present application. As shown in FIG. 1B , the touch control device may be a touch screen on an electronic device.

In one example, the fingerprint sensor may be disposed below the touch screen of the electronic device (as shown in FIG. 1B ). FIG. 1C is a second schematic diagram of an application scenario of the image matching method provided by the embodiment of the present application. Taking the entry stage as an example, the template object may be the fingerprint part of the user's finger. When the user's finger presses on the collection area of the fingerprint sensor (as shown in FIG. 1C ), the fingerprint sensor can collect the fingerprint image through the touch screen. In another example, the fingerprint sensor may also be provided in other areas on the surface of the electronic device that are not covered by the touch screen.

Taking the template object and the object to be detected as the fingerprint part of the user's finger as an example, the touch screen can obtain the coordinate set of the contact point when the current finger touches the touch screen, and generate a touch signal, and each frame of the touch signal can include the coordinate set of the contact point. Determined touch image. In this embodiment of the present application, each frame of the touch signal may further include information such as a touch pressure (touch pressure, TP) value on the contact point. In this embodiment of the present application, the touch screen may generate multiple frames of touch signals during the current period when a finger touches the touch screen. The multi-frame touch signals may form a touch signal sequence, and the touch signal sequence may be referred to as a touch sequence for short.

In other embodiments of the present application, the touch screen may also generate a touch signal when it is detected that the current distance between the finger and the touch screen is less than a preset sensing distance. Taking the sensing distance of 1 cm as an example, when the current finger enters the sensing space 1 cm above the touch screen, the touch screen can generate one or more frames of touch signals. Coordinate set, each frame of touch signal may include touch image determined by the coordinate set of these points and information such as the distance between the user's finger and these points. When the user's finger continues to stay in the sensing space, the touch screen can generate multiple frames of touch signals.

It should be noted that the morphological feature information may also be collected by other morphological feature collecting devices such as a camera or a sensor.

In this embodiment of the present application, an attitude sensor (not shown in the figure) may also be provided on the electronic device, and the attitude sensor may be used to acquire three-dimensional position information of the electronic device in the earth coordinate system. Illustratively, the attitude sensor may be a gyroscope or a gravity accelerometer. In practical applications, the data collected by the touch device and the attitude sensor can be combined to determine the attribute information of the template object. The processing steps for determining the attribute information of the template object in combination with the data collected by the touch device and the gesture sensor will be described in detail in the following embodiments.

The image database involved in the embodiments of this application may be deployed on an electronic device, or may be deployed independently. This embodiment of the present application does not limit this.

The technical solutions of the input stage and the recognition stage in the image matching method provided in the embodiment of the present application may also be implemented by different electronic devices, and the electronic devices used in any stage may include the above-mentioned image acquisition device, touch device, attitude sensor, etc. . The image matching method provided by the embodiments of the present application is exemplarily described below.

Example 1

This embodiment of the present application provides an optional implementation of the input stage of the image matching method. FIG. 2A is a schematic flowchart of an input stage of an image matching method provided by an embodiment of the present application.

As shown in Figure 2A, the steps of the entry phase may include:

S101, obtain a template image.

In this embodiment of the present application, a template image may be collected from a template object by an image collection device. For example, when the template object is a fingerprint part, the fingerprint image to be recognized can be collected by the fingerprint sensor of the fingerprint part to be recognized.

S102: Acquire a touch signal of the template object.

In the embodiment of the present application, the touch signal of the object to be detected may be collected by the touch device. In an example, the touch control device may be a touch screen on an electronic device. The relative positions of the touch device and the image capture device may be predetermined, as shown in FIG. 1B .

It should be noted that step S101 and step S102 may be performed simultaneously, that is, the touch signal of the template object and the template image may be simultaneously collected when the template object is pressed on the touch screen. In practical applications, steps S101 and S102 can be set to be completed in a relatively short period of time. For example, the time difference between the time of acquiring the image to be recognized and the time of acquiring the touch signal of the object to be detected can be set to be less than the time deviation threshold.

In this embodiment of the present application, after detecting that the template object contacts the touch screen, one frame or multiple frames of touch signals of the template object may be acquired. FIG. 2B is a schematic diagram of determining morphological feature information according to a touch signal in an embodiment of the present application. Taking the morphological feature information to be determined as the finger contour information as an example, starting from the touch signal generated by the finger pressing on the touch screen, a group of continuous touch signals within a period of time are obtained as the touch sequence, for example, as shown in FIG. 2B , 20 frames of touch signals can be acquired as a touch sequence, wherein each frame of touch signals includes a touch image determined by the coordinate set of the contact points when the current frame of touch signals is collected (as shown in the upper part of FIG. 2B ). Show).

S103: Determine morphological feature information of the template object according to the touch signal of the template object.

In the embodiment of the present application, for example, the morphological feature information of the template object may be determined according to one or more frames of touch signals of the template object.

In this embodiment of the present application, the morphological feature information of the template object may be contour information, position information, kinematic information, and the like of the template object. It should be noted that the position of the image capture device that captures the template image and the relative position of the touch screen that captures the touch signal are determined. Therefore, the morphological feature information of the template image can be the contour information with the position of the image capture device as a reference. , location information, kinematics information. The contour information may be a set of projection points of the template object in a plane (such as the screen where the touch device is located), or a set of points in a three-dimensional coordinate system; the position information may be the relative position of the template object relative to the image acquisition device , and the kinematic information can be the moving direction, moving speed, and moving trajectory of the template object.

Taking the morphological feature information to be determined as the contour information of the finger as an example, as shown in FIG. 2B , edge detection can be performed on the preprocessed contour map such as normalization of each frame of touch signal, to obtain each frame of touch. A set of edge points representing the finger contour corresponding to the control signal, in which the Sobel algorithm can be used for edge detection; then, the edge detection results of the multi-frame touch signals are synthesized, and the detected finger contour is aggregated to obtain the current input. Profile information of fingers. For example, first, determine the center point of each frame of touch signals, wherein any ray starting from the center point will pass through at least one edge point in the finger outline of the current frame, and then, respectively, from the touch images of each frame Select a reference ray from the center point as the starting point, the slope of the reference ray in each frame is the same, compare the coordinates of the edge points that the reference ray passes through in the finger outline of each frame, and select the point farthest from the corresponding center point as the final point. Then, rotate the reference ray according to the preset rotation interval angle, compare the coordinates of the edge points that the rotated ray passes through in the touch image of each frame, and select the point farthest from the corresponding center point as the final point After that, rotate the reference ray several times to obtain the corresponding edge coordinate points until the reference ray rotates once, and finally, the set of all edge coordinate points is determined as the outline information of the currently pressed finger. In the embodiments of the present application, edge detection methods such as Laplacian operator and Canny operator, or computer vision algorithms such as machine learning methods may be used to perform contour detection on the touch image frame data. Preprocessing can include methods such as normalization, graphing, and time series analysis. For example, the value of the coordinate point is limited within the interval [0, 255], or the touch signal sequence is filtered by time using the Kalman filtering method.

In other embodiments of the present application, when the touch screen supports acquiring the touch signal generated by the template object entering the sensing control, the touch signal may also include relative position information between the template object and the touch screen. The position is fixed. Therefore, the relative position information of the template object relative to the touch screen obtained according to the touch signal can be converted into the position information of the object to be detected relative to the image acquisition device. For example, the touch signal can contain a set of three-dimensional coordinates. The touch image determined by the point set in the system. In addition, when the acquired touch signal includes multiple frames of touch signals, the kinematic information of the template object such as the movement speed and movement trajectory of the template object can also be determined according to the change rule of the relative position information in the multi-frame touch signals.

The foregoing method for determining the morphological feature information of the template object according to the touch signal is merely an example, which is not limited in this embodiment of the present application.

In other embodiments of the present application, before determining the morphological feature information of the object to be detected according to the touch information of the object to be detected, the method may further include: acquiring the three-dimensional position information of the touch control device through a gesture sensor; correspondingly, step S103 is based on The morphological feature information of the object to be detected and the determination of the attribute information of the image to be recognized may include: determining the attribute information of the image to be recognized according to the morphological feature information of the object to be detected and the three-dimensional position information of the touch device. In other embodiments of the present application, the use of the attitude sensor to correct the attribute information of the image to be recognized will be described in detail.

S104: Determine attribute information of the template image according to the morphological feature information of the template object.

In this embodiment of the present application, the attribute information of the template image may be the object type of the template object, the habitual posture information of the template object, the global position information of the template image in the outline information of the template object, and the like.

In this embodiment of the present application, the object type may be any of the following. In an example, the object type of the template object may include at least one of the following: fingerprint part, palm print part, left and right hand information, and finger type. In another example, the object type of the template object may also include other types of objects that can be identified through the outline information, such as ears, lips, head, and soles of feet. In other embodiments of the present application, the object type of the template object may also be entered by the user.

In this embodiment of the present application, the habitual gesture information may include at least one of a movement track type, a movement speed type, global direction information, and movement direction information.

Among them, the movement trajectory type can be used to indicate that the user's finger moves in a straight line or along an arc when approaching the touch screen. The speed level of the speed at which the finger approaches the touch screen when the finger is pressed, and the press stay duration type may indicate the length level of the duration of the user's finger staying on the touch screen when pressing. The global orientation information may indicate the orientation of the tip of the user's finger when pressing the touch screen, for example, the tip is inclined to the left. The movement direction information may be the sliding direction of the user's finger when it approaches the touch screen until it contacts the touch screen.

In this embodiment of the present application, the global position information of the template image in the contour information of the template object may include at least one of the following: a distance from a reference point in the contour information of the template object, with the contour information of the template object as a reference The regional location identifier of the system. The area location identifier may be a central area, an edge area, an upper half area, a lower half area, an area identified by the coordinate value of the area center, and the like.

In this embodiment of the present application, determining the attribute information of the template image according to the morphological feature information of the template object may include one or more of the following implementations.

In an optional implementation manner of determining the attribute information, step S104 may include: determining the object type of the template object according to the outline information of the template object; and using the object type of the template object as the attribute information of the template image. Wherein, in an example, the object type of the template object may include at least one of the following: fingerprint part, palm print part, left and right hand information, and finger type. In another example, the object type of the template object may also include other types of objects that can be identified through the outline information, such as ears, lips, head, and soles of feet. In yet another example, the object type of the template object may also be other objects that can be identified through the outline information. For example, when the shape of the object to be detected represented by the outline information is close to an ellipse and the maximum size is less than 6 square centimeters, the object can be considered as a fingerprint part.

In this embodiment of the present application, there are multiple ways to infer the finger object type. Data analysis methods or methods based on machine learning can be used, combined with attitude sensor data, to determine the type of the currently entered finger according to the distribution characteristics of touch signals. In an embodiment of inferring the object type, a finger type recognition system can be constructed, wherein the feature extraction of the finger type can use a machine learning-based feature extraction network (eg, Res-Net). In another embodiment of inferring the type of the finger, a classification network (such as classic frameworks such as SVM and CNN) can be constructed directly based on big data to determine the type of the finger. In addition, the finger object type can also be inferred by theoretical deduction and statistical analysis directly based on the finger touch signal and gesture sensor data. Here, the inference of the finger object type is exemplified by theoretical derivation and statistical analysis.

In an example, the finger object type may include left and right hand types, and FIG. 2C is a schematic diagram of inferring left and right hand types in an embodiment of the present application.

As shown in Figure 2C, first, taking the vertical center line of the screen as the reference vertical axis, and the bottom line of the screen as the horizontal axis, extract the edge contours of the multi-frame touch signals and obtain a union, and use curve fitting and other methods to calculate the edges after the union. The contour is smoothed, and the line connecting the two points with the largest contour curvature is the long axis to represent the finger pressing direction, as shown in the slanted dotted line in the left figure of Figure 2C. Then, in the pressing direction on the right side of the vertical axis of the screen, the included angle θ1 between the dotted line whose abscissa is greater than 0 and the positive direction of the vertical axis of the screen is calculated. If the angle between the pressing direction and the positive direction of the vertical axis of the screen is an obtuse angle, it can be inferred that the left and right hand type of the current finger is the right hand, as shown in the left figure in Figure 2C, if it is an acute angle, it can be inferred that the left and right hand type of the current finger is the left hand. As shown in the right panel in Figure 2C.

In addition, according to the attitude sensor data of the electronic device itself, the three-dimensional attitude information of the touch panel can be estimated at this time, wherein the attitude sensor data can be used to estimate that the vertical axis and the horizontal axis of the extension plane of the touch panel are relative in the geodetic coordinate system. The inclination angle of the ground surface and the orientation of the extension plane, etc. For example, the inclination angle θ2 of the horizontal axis of the touch panel with respect to the ground surface can be calculated. After that, θ1 is corrected in combination with the horizontal axis inclination angle θ2 of the touch panel. For example, if θ1-θ2>0, θ1 can be corrected to θ1-θ2, and if θ1-θ2<0, θ1 can be corrected to 180-θ2+θ1. Finally, the left and right hand types of the fingers are inferred from the corrected angle. For example, when θ2=0, as shown in the left figure in Fig. 2C, and the correction value is an obtuse angle, it is presumed that the left and right hand types of the current finger are right-handed. The left and right hand type of the current finger is left hand.

In another example, the finger object types may include finger types such as thumb, index finger, middle finger, etc. FIG. 2D is a schematic diagram of inferring finger types in this embodiment of the present application.

As shown in FIG. 2D , the left figure is a schematic time sequence diagram of the touch pressure (touch pressure, TP) value of the left thumb, and the right figure is a schematic time sequence diagram of the TP value of the right index finger or middle finger. In the embodiment of the present application, according to the pre-collected time sequence diagrams of TP values corresponding to fingers of various finger types, the distribution rules of TP values of various finger types can be obtained. Then, the finger type of the current finger can be inferred according to the distribution law of the TP value of the finger type.

For example, before determining the finger type of the current finger, first collect touch signal sequences of various finger types. Exemplarily, a touch sequence composed of 20 frames of touch signals may be taken, wherein each touch signal includes the TP value on the pixels touched by the current finger in the 8*8 area. Then, draw the timing diagram of each pixel in the 8x8 area, and a total of 64 timing lines are obtained. Among them, each pixel corresponds to a curve, which represents the change of the TP value on a certain pixel over time over time. law. After that, the proportion of the time series lines whose TP value is greater than the preset TP value threshold in a certain time interval in the total number of time series lines is counted. Exemplarily, from the left graph and the right graph in Figure 2D, it can be concluded that in the time period [0.05, 0.10], in the time sequence line of the thumb shown in the left graph, the number of time sequence lines with TP values higher than 175 , significantly more than the index and middle finger timing lines shown in the right figure, as shown by the intersecting area of the dotted box in Figure 2D.

Based on this, in the input stage, the touch signal sequence of the current finger can be collected, and the timing diagram of the TP value can be drawn. If the proportion is higher than the preset classification proportion threshold, it can be inferred that the finger type of the current finger is the thumb, otherwise, it can be inferred that the finger type of the current finger is the index finger or the middle finger. Taking the classification ratio threshold of 5/64 as an example, when the ratio is greater than the classification ratio threshold, it can be inferred that the current finger is the thumb; if the ratio is less than or equal to the classification ratio threshold, it can be inferred that the current finger is the index finger or the middle finger. It should be noted that, in the steps of the identification stage in this embodiment of the present application, a similar method may also be used to determine the finger type of the current finger collected in the identification stage.

In yet another example, the finger object types may include left and right hand types and finger types, eg, right thumb, left index finger, left middle finger. The finger object type of the current finger can be inferred in conjunction with the previous examples. For example, according to FIG. 2C and FIG. 2D , it can be jointly inferred that finger 1 is the thumb of the right hand, and finger 2 is the index finger or middle finger of the left hand. In another optional implementation manner of determining the attribute information, step S104 may include: determining the global position information of the template image in the contour information of the template object according to the contour information of the template object; Attribute information as a template image. Wherein, the global position information of the template image in the contour information of the template object may include at least one of the following: a distance from a reference point in the contour information of the template object, a region position identifier with the contour information of the template object as a reference system . Exemplarily, the area may be a rectangle, the center of the area may be a coordinate point in a reference system, and the length and width of the rectangle may be preset length and width values.

FIG. 2E is a schematic diagram of inferring global position information and global direction information in an embodiment of the present application. As shown in FIG. 2E , according to the position of the touch pixel extracted from the touch signal, the position of the finger outline of the current finger on the screen can be determined. Then, since the position of the collection area of the fingerprint collector is fixed during hardware design, the global position information of the template image can be determined by coordinate transformation from the relative position of the template image in the fingerprint contour.

For example, referring to FIG. 2E , the long and short axes of the smoothed general outline of the finger (ellipse in FIG. 2E ) are determined as the finger X axis and the finger Y axis of the finger coordinate system, wherein the The major axis can be determined according to the line connecting the two points with the largest curvature of the contour. Then, the coordinates (x _i , y _i ) of the center of the template image are taken as the global position information of the template image. Then, the angle θ _i between the diagonal line of the template image and the X-axis of the finger is taken as the global direction information. Wherein, as shown in FIG. 2E , the diagonal line of the template image is a dotted line passing through the center of the template image.

In another optional implementation manner of determining the attribute information, step S104 may include: determining the habitual posture information of the template object according to at least one morphological feature information among contour information, position information, and kinematic information of the template object; The customary pose information of the template object is used as the attribute information of the template image. The customary posture information of the template object includes at least one of the following: global direction information of the template object, speed or speed level of the template object, movement direction of the template object, and movement track of the template object. For example, the global orientation information of the template object includes at least one of the following: the top end is inclined to the left, and the top end is inclined to the right.

For example, the template object is a fingerprint, and the morphological feature information of the template object may be the kinematic information of the template object, for example, the moving speed is lower than 0.01 m/s, and the global direction information is that the top is tilted to the left. The moving speed and global direction information of the template object can be used as the attribute information of the template image. S105, add the template image and the attribute information of the template image to the image database.

The image database may store at least one template image and attribute information of each template image. In addition, the image database may also store object information such as object identifiers or entity identifiers or usage rights corresponding to template objects corresponding to each template image.

Table 1 is a schematic diagram of the data stored in the image database.

Table 1

The usage authority may be one or more of unlocking the screen, booting up, logging in to the system, logging in to an application (Application, APP) with account information corresponding to the entity identification, or other usage authority for the application or data corresponding to the entity identification, etc. For usage rights, see examples in Table 1, which are not limited in this application.

Using the technical method of the embodiments of the present application to construct an image database can provide template images and attribute information corresponding to the template images, and then, according to the attribute information of the template images, the template images that satisfy the attribute information similarity condition with the attribute information of the image to be recognized can be screened , query the target template image matching the to-be-recognized image in the filtered template image, which can improve the message of image matching.

Embodiment 2

The implementation of the identification phase is exemplified below.

This embodiment of the present application provides an optional implementation of the identification stage of the image matching method.

FIG. 3A is a schematic flowchart of a recognition stage of an image matching method provided by an embodiment of the present application. As shown in Figure 3A, the steps of the identification phase may include:

S201, an image to be recognized is acquired.

The image to be recognized is an image of the object to be detected. Taking the object to be detected as the fingerprint part of the user's finger as an example, the image to be recognized may be a fingerprint image. The manner of acquiring the image to be recognized is similar to the manner of acquiring the template image, and reference may be made to the relevant description in step S101.

S202: Obtain morphological feature information of the object to be detected.

The manner of acquiring the morphological feature information of the object to be detected is similar to the manner of acquiring the morphological feature information of the template object, and reference may be made to the relevant description in step S102.

S203: Determine attribute information of the image to be recognized according to the morphological feature information of the object to be detected.

The implementation of this step is similar to that of step S103, and reference may be made to the relevant description in S103.

S204: Screen at least one template image according to the attribute information of the image to be recognized.

The attribute correlation condition is satisfied between the attribute information of the filtered template image and the attribute information of the image to be recognized.

In this embodiment of the present application, the attribute information of each template image may be obtained by using any of the methods mentioned in the first embodiment. In this embodiment of the present application, other implementations may also be used to input the attribute information of the template image. For example, while collecting template images, the electronic device may prompt the user to select or input attribute information corresponding to each template image in the input device of the electronic device. In an example, the user can be prompted to input the fingerprint of the right thumb in the graphical user interface of the electronic device, and the collected fingerprint image and the corresponding attribute information "right thumb" are added to the image database. In another example, the user may also be prompted to enter a template image whose global position information is the edge region, and the collected fingerprint image and the corresponding attribute information "edge region" are added to the image database. This application does not limit this.

In this embodiment of the present application, the number of attribute information of the image to be recognized may be one or more. When there are at least two types of attribute information of the image to be recognized, the above step S204 may include: screening at least one template image according to the at least two types of attribute information of the image to be recognized and the weight of each attribute information. For example, a template image with higher similarity in attribute information can be selected.

Exemplarily, the similarity of attribute information can be calculated according to the following formula:

Similarity percentage of attribute information = weight 1 * similarity percentage of two images - weight 2 * distance of global position information of two images - weight 3 * difference of global orientation information of two images

It should be noted that in some scenarios, weight 2 and weight 3 can also be set to 0

In this embodiment of the present application, for different attribute information, the attribute correlation condition in step 204 may include one of the following implementations or a combination of multiple implementations.

In an optional implementation manner of the attribute correlation condition, when the attribute information of the to-be-recognized image includes the object type corresponding to the to-be-recognized image, the object type corresponding to the filtered template image may be the same as the object type corresponding to the to-be-recognized image same. For example, the morphological feature information of the object to be detected may be the outline information of the fingerprint part to be recognized, and according to the outline information of the object to be detected, it can be determined that the object type of the object to be detected is the finger of the left hand, and then the object type is used as the image to be recognized. attribute information, first filter the template image whose object type is the finger of the left hand in the image database.

In another optional implementation of the attribute correlation condition, when the attribute information of the to-be-recognized image includes the gesture information corresponding to the to-be-recognized image, the habitual gesture information corresponding to the filtered template image may be the same as that of the to-be-recognized image. The habitual posture information is the same. For example, for example, the object to be detected is a fingerprint, and the morphological feature information of the object to be detected can be the kinematic information of the object to be detected, for example, the moving speed is lower than 0.01 m/s, and the global direction information is the top left. The moving speed and global direction information of the object to be detected can be used as the attribute information of the image to be recognized, and the corresponding moving speed is less than 0.01 m/s and the global direction information is the template image with the top left facing in the image database, and then in the image database. The target template image that matches the query in the filtered template. In another optional implementation manner of the attribute correlation condition, when the attribute information of the image to be recognized includes global location information corresponding to the image to be recognized, the global location information corresponding to the filtered template image is the same as the image to be recognized. The global location information satisfies the location proximity condition.

The location proximity condition may include: the location indicated by the global location information corresponding to the filtered template image includes the location indicated by the global location information corresponding to the image to be recognized, or, the location indicated by the global location information corresponding to the filtered template image There is an intersection between the position represented by the global position information corresponding to the image to be recognized, or the position represented by the global position information corresponding to the filtered template image and the position represented by the global position information corresponding to the image to be recognized. The distance is less than the proximity distance threshold.

For example, the attribute information of the image to be recognized includes object type and global position information, the object type of the image to be recognized is the index finger of the left hand, and the region position of the image to be recognized in the outline information of the object to be detected is identified as region 1, then the A group of template images are screened out from at least one template image, the object type of the screened template image is the index finger of the left hand, and the region position of the template image in the contour information of the corresponding template object is identified as region 2, wherein region 2 contains the region 1, or, there is an intersection between area 2 and area 1, or the distance between area 2 and area 1 is less than the adjacent area distance threshold.

In an example of practical application, if the object to be detected is the fingerprint part to be recognized, the to-be-recognized image collected by the image acquisition device is the fingerprint image of the user's finger pressed in the fingerprint collection area, which may be called a partial image or a partial topology. The morphological feature information of the object to be detected can be the contour information of the object to be detected, which can be called a global image or global topology; according to the contour information of the object to be detected and the position of the fingerprint collection area, the global position information of the local image in the global image can be determined . 1D, the global image may be an ellipse corresponding to the thumb of the right hand, and the local image may be located on the left side of the long axis of the ellipse, that is, the left side of the Y axis in the figure. After that, the global position information corresponding to the image to be recognized in the contour information of the object to be detected is used as the attribute information of the image to be recognized, and the numerous template images in the image database are screened. Exemplarily, the global position information may be screened to be a template image located on the left side of the long axis, and a target template image matching the image to be recognized may be queried from the screened template images.

In yet another embodiment in practical applications, as shown in FIG. 1A , the attribute information of the image to be recognized may include global position information of the image to be recognized in the complete outline of the object to be detected. In an example, the image to be recognized The attribute value corresponding to the attribute information may be "edge area", then a template image with the corresponding attribute information as "edge area" may be filtered out of the template image, and the selected template image may be searched for matches with the image to be recognized. target template image.

In another embodiment in practical applications, when the location proximity condition is that the distance between the location represented by the global location information corresponding to the filtered template image and the location represented by the global location information corresponding to the image to be recognized is smaller than the adjacent area distance threshold. The approximate region to which the image to be recognized belongs can be inferred based on the global location information of the image to be recognized. After that, the template image corresponding to the global location information is located in the inferred approximate region, and the template image that matches the image to be recognized is queried in the filtered template image. Target template image. The approximate region to which it belongs may be referred to as an inference region or an inference template region.

FIG. 3B is a schematic diagram of an approximate area inferred in an embodiment of the present application. As shown in FIG. 3B , for example, the coordinates of the center of the fingerprint image to be recognized can be taken as the center of the estimated approximate area, and the area within a preset radius around the center is determined as the approximate area, as shown by the dotted line in FIG. 3B the circular area. For example, the Euclidean distance between the center position of the template image in the template library and the center position of the current fingerprint to be recognized is calculated, and when the distance is less than the specified radius, it can be determined that the template image is located in the approximate area to which it belongs. Based on this judgment principle, the two template images adjacent to the image to be recognized in Fig. 3B belong to the inference area, while the template image whose center is (x _k , y _k ) is not in the inference area. In this embodiment of the present application, the approximate area may also be an area of another fixed shape centered on the global position of the fingerprint image to be recognized in the fingerprint contour. The fixed shape may be, for example, a regular hexagon, an ellipse, a rectangle, or the like.

S205 , in the filtered template images, query a target template image that matches the image to be recognized.

The target template image matching the image to be recognized may be a target template image whose image similarity with the image to be recognized reaches a similarity threshold.

In this embodiment of the present application, in step S205, in the filtered template images, query the target template image that matches the image to be recognized, which may include: comparing the similarity between the filtered template image and the image to be recognized, and comparing the similarity Template images greater than the similarity threshold are used as matching target template images. In an example, when the number of the filtered template images is multiple, before the similarity comparison is performed, the filtered template images may also be sorted according to the historical matching success rate of the filtered template images from high to low. , and then compare the similarity between the filtered template image and the to-be-recognized image one by one in sequence.

In the embodiment of the present application, one or more attribute information of the image to be recognized is used to screen the template images in the image database, and in the filtered template images, the target template image that matches the image to be recognized is queried. It can provide the efficiency of image matching.

In this embodiment of the present application, the number of filtered template images obtained in step S204 is 0, or, in step S205, when no target template image matching the image to be recognized is found in the filtered template images, in step S205 After step S205, step S206 may also be included.

S206 , in at least one template image other than the filtered template image, search for a target template image that matches the image to be recognized.

In an example, the above-mentioned other template images may be sorted according to the historical matching success rate from high to low, and then the similarity between each other template image and the image to be recognized is compared in order.

When step S206 is adopted, when no matching target template image is found in the filtered template images, the matching target template image can be continuously searched in other template images in the image database to improve the success rate of image recognition.

For example, in practical applications, FIG. 3C is a schematic diagram of template matching in an embodiment of the present application. As shown in branch 1 in FIG. 3C , first, the template triples in the global template library that belong to the inferred finger type and are located in the approximate area of the inference, exemplarily, the global position information in FIG. 3C is (x _j , y _j ), the template triplet is matched with the to-be-recognized image extracted by the current pressing. The matching order within this range may be determined according to historical statistical sorting information, and the matching formula may be calculated according to the attribute information similarity calculation formula given in S204 in the foregoing embodiment. If the similarity percentage is greater than the similarity matching success threshold, the matching is successful in the inferred finger type and the inferred area, the historical statistical ranking information is updated, and the user is prompted in the output device that fingerprint recognition is successful; If the matching in the template image fails, there are two strategies for the follow-up process.

In practical applications, there are various ways of matching strategies for the above screening and the matching process after screening. Table 2 is a set of schematics for matching strategies.

Table 2

The inferred finger type refers to the left and right hand type and finger type of the object to be recognized determined according to the morphological feature information of the object to be recognized, and the inferred template area is determined according to the global position information of the image to be recognized in the outline information of the object to be detected. approximate area. The first strategy or the second strategy can be used after matching the template image of the inferred finger type and the inferred template region first, but after the matching fails.

In the example of the first strategy, you can try to match the template image of the inferred template area under the non-inferred finger type, and if it still fails, enter the second strategy or try to match the template image of the non-inferred template area under the non-inferred finger type. template image to match. If the match is successful at this stage, update the historical statistical sorting information and enter the output device to prompt the user that the fingerprint identification is successful. The matching order within this range can still be determined according to historical statistical sorting information.

In the example of the second strategy, you can try to match the template image of the non-inferred template area under the inferred finger type, and if it still fails, enter the first strategy or try to match the non-inferred template area under the non-inferred finger type. template image to match. If the match is successful at this stage, update the historical statistical sorting information and enter the output device to prompt the user that the fingerprint identification is successful. The matching order within this range can still be determined according to historical statistical sorting information. It should be noted that if the first strategy is executed first, and then the second strategy is executed, if the recognition still fails after the second strategy is executed, it will directly enter the template image of the non-inferred template area under the trial and non-inferred finger types. to match.

In this embodiment of the present application, after step S205 or S206, the steps in the matching and identifying stage may further include steps S207 and S208.

S207, when the target template image matching the image to be recognized is queried, obtain the template entity identifier and/or the usage authority corresponding to the target template image.

S208 , when the target template image matching the image to be recognized is not queried, it is determined that the object to be detected corresponding to the image to be recognized does not have the right to use.

The template entity identifier may be user identity information, and the usage authority may be unlocking the screen, booting, logging in to the system, logging in to the APP, and the like.

In this embodiment of the present application, in order to further improve the success rate of image matching. In the input stage, it is also possible to ensure that enough template images are collected for the template object as much as possible according to the global position information corresponding to the template image. For example, when collecting a template image, a template image covering the edge portion of the template object may be collected, or by collecting multiple template images for the template object, it is ensured that the multiple template images corresponding to the template object can cover the contour information of the template object. Detailed descriptions will be made in other embodiments of the present application.

In the image matching method provided by the embodiment of the present application, the attribute information of the image to be recognized is determined by the morphological feature information of the object to be detected, and at least one template image is screened according to the attribute information of the image to be recognized, and then, after the screening The target template image matching the image to be recognized is queried in the template image, which can improve the efficiency of image matching.

Embodiment 3

The embodiment of the present application also provides an optional implementation of the input stage of the image matching method. Among them, in the input stage, the template image and the morphological feature information of the template object are collected, and the attribute information of the template image is determined according to the morphological feature information of the template object. Detect the morphological feature information of the object.

For example, in the identification stage, one method is to obtain both the image to be identified and the morphological feature information of the object to be detected, and then, according to the attribute information of the image to be identified determined by the morphological feature information of the object to be detected, at least one Template images to filter. In another way, only the to-be-recognized image may be acquired. When only the to-be-recognized image is acquired, the template image may not be screened, and the target template image matching the to-be-recognized image may be directly queried in the complete set of template images.

It should be noted that, in the input stage, when the template image and the attribute information corresponding to the template image are added to the image database, template images with various values of the attribute information can be collected for each attribute information. In this way, as many template images as possible can be collected, thereby improving the success rate of image recognition.

The following takes the morphological feature information as the contour information as an example for illustrative description.

FIG. 4 is a second schematic flowchart of an image matching method provided by an embodiment of the present application. As shown in FIG. 4 , the steps in this embodiment of the present application may include:

S401. Acquire at least one first template image of a template object.

Wherein, at least one first template image of the template object can be collected by the image collection device

S402, obtain the outline information of the template object.

Wherein, the outline information of the template object may be acquired through the touch device, and the outline information of the template object is used to determine the region position identifier of the at least one first template image in the outline information of the template object.

In the embodiment of the present application, for example, the template object may be the first template object, for example, the fingerprint part of the index finger of the right hand of the user 1 . For the manner of collecting the template image, the manner of acquiring the outline information of the template object, and the manner of determining the region position identifier, reference may be made to the descriptions in other embodiments of the present application.

S403: Determine, according to the outline information of the template object, a region position identifier of each first template image in the outline information of the template object.

In this embodiment of the present application, reference may be made to descriptions in other embodiments of the present application for the manner of determining the area location identifier.

S404: Collect a second template image of the template object when the set of region location identifiers corresponding to at least one first template image does not satisfy the coverage condition.

Wherein, at least one of the first template image and the second template image is used to query the target template image matching the image to be recognized.

In the embodiment of the present application, the above coverage conditions may include: covering at least one local area of the template object; covering the edge area of the template object exceeding the edge coverage condition of the preset coverage ratio, or, covering the outline information of the template object exceeding the preset coverage A global override condition for the scale. In an example, the preset coverage ratio may be 50%, 90%, or the like.

FIG. 5A is a schematic diagram of the position of a template image of an edge region in an image matching method provided by an embodiment of the present application. As shown in FIG. 5A , for example, it can be checked whether the acquired template image covers the edge area of the template object, or whether it completely covers the entire area of the template object. When not enough template images are collected, the user may be prompted to continue collecting template images of the uncovered area. In this way, the success rate of image matching can be improved.

In some other embodiments, before step S404, the steps in this embodiment of the present application may further include:

S405: Determine the supplementary mining area according to the set of area location identifiers and coverage conditions corresponding to the at least one first template image.

Wherein, the supplementary mining area may be an area where at least one first template image does not satisfy the coverage condition. For example, if the coverage condition is to cover the center area and the first edge area, and at least one first template image only covers the center area, the supplementary mining area may be the first edge area.

S406: Output prompt information, where the prompt information is used to prompt the input of a second template image of the template object including the supplementary mining area.

In practical applications, for example, the image acquisition device can be a fingerprint sensor; the touch device is a touch screen in an electronic device; the fingerprint sensor is disposed on the side of the touch screen close to the body of the electronic device, as shown in FIG. 1C . The output prompt information can be prompted through a touch screen, or through a device such as a sound.

FIG. 5B is a schematic diagram 3 of an application scenario of the image matching method provided by the embodiment of the present application. In practical applications, as shown in FIG. 5B , the user enters a fingerprint on the fingerprint entry interface of the fingerprint entry system. If one or more fingerprints of the finger have been collected, but the collected fingerprint image does not cover the left edge of the fingerprint outline, a prompt message "The collected fingerprint does not cover the left edge of the fingerprint outline, please You rotate your finger to continue enrolling your fingerprints".

It should be noted that steps S405 and S406 are not steps that must be performed in this embodiment of the present application.

Before step S403, the steps in this embodiment of the present application may further include: collecting the three-dimensional attitude information of the electronic device through the attitude sensor, and then correcting the position of each first template object in the template object according to the outline information of the template object obtained by using the touch device. The region location identifier in the contour information of . Please refer to the detailed description in other embodiments of the present application.

In practical applications, for example, the steps in the template image entry stage may include: collecting the P-1 th template image and the P-th template object corresponding to the P-1 th template image for the first template object -1 contour information; wherein, P can be an integer greater than or equal to 2. The region position identifier of the P-1 th template image in the P-1 th outline information is determined according to the P-1 th outline information. If the set of the 1st to P-1th area location identifiers corresponding to the 1st to P-1th template images of the first template object fails to satisfy the global coverage condition. The P th template image and the P th contour information of the first template object corresponding to the P th template image may continue to be collected from the first template object. If the combination of the 1st to Pth region location identifiers corresponding to the 1st to Pth template images still fails to satisfy the global coverage condition, the P+1th template image and The P+1th contour information of the first template object corresponding to the P+1 template images. When the set of the region location identifiers corresponding to the collected template image of the first template object reaches the global coverage condition, the input can be ended. The image database obtained in this way may include at least P template images of the first template object. It should be noted that the time difference between the moment of acquiring the corresponding first template image and the moment of acquiring the outline information of the first template object is less than a preset time deviation threshold; exemplarily, the corresponding The first template image and the contour information of the first template object are obtained.

By using the image matching method provided by the embodiment of the present application, it is possible to collect as much template image as possible in the template image input stage, for example, to collect a template image including an edge area, or to collect a plurality of template images to cover the outline of the template object information, that is, to build a global fingerprint template library, so as to avoid the problem of recognition failure caused when the image to be recognized is located at the edge of the object to be detected or at some special position.

Embodiment 4

In practical applications, for example, a template triplet corresponding to each template image may be stored in the image database. Exemplarily, the template triplet may include a template image, global position information, and the global orientation in the above-mentioned user gesture information information. The global position information may be referred to as template position for short, and the global direction information may be referred to as template direction for short.

In this embodiment of the present application, in the input stage, steps S101 to S105 may be performed multiple times to input multiple template images and attribute information of the template images. Wherein, when a plurality of template images and corresponding attribute information are collected for an object identification, a condition for stopping inputting may also be set. The following is an exemplary description of the processing flow of the input stage.

The technical solutions provided by the embodiments of the present application can be applied to the field of fingerprint identification technology. The fingerprint identification technology is a biometric identification technology, and the biometric identification can capture the biometrics as a digital signal through a sensor device. The image matching method provided by the embodiments of the present application can be used to provide a fingerprint identification system. The fingerprint recognition system can be a pattern recognition system including fingerprint image acquisition, preprocessing, feature extraction, comparison and other modules, and can be used in places that require personnel identity confirmation, such as mobile phone unlocking, notebook unlocking, mobile payment, bank internal processing, access control system and attendance system. Referring to FIG. 1C , when a fingerprint identification system is deployed on a smart terminal device, a fingerprint reader can be set under the screen, and the reader can collect information and match and identify the user's fingerprint through the screen. The fingerprint reader can be combined with optical recognition, capacitive recognition and ultrasonic recognition. The under-screen fingerprint recognition technology mainly uses under-screen optical fingerprint recognition and under-screen ultrasonic recognition.

Similar to the methods shown in the foregoing embodiments, the image matching method provided by the embodiments of the present application may include: a fingerprint entry stage and a fingerprint identification stage.

Among them, in the fingerprint entry stage, a global fingerprint template library can be constructed for the user's finger. The global fingerprint template library may be referred to as a global template library for short, and the fingerprint template image may be referred to as a fingerprint template for short. The global template library can contain attribute information of each fingerprint template. Exemplarily, the attribute information of each fingerprint template may include the global position information and global orientation information of each fingerprint template in the overall fingerprint of the user's finger, the global fingerprint profile corresponding to the template library, and the corresponding finger type, such as: left and right hands, thumb , index and middle fingers, etc.

The global fingerprint template library can include fingerprint images and attribute information corresponding to fingerprint images, a set of fingerprint images and attribute information corresponding to fingerprint images can be called a fingerprint template, and each fingerprint template can include fingerprint images, global location information, and global orientation information. , which can also be called fingerprint template triples.

It should be noted that, using the captured touch signal sequence and attitude sensor data, on the one hand, the profile and type information of the currently entered finger can be estimated. On the other hand, according to the contour information, the hardware position information of the fingerprint image collector relative to the touch screen, and the pixel information of the screen touch dot matrix, the global position information and global orientation of the captured fingerprint image in the overall fingerprint of the finger can be estimated. information.

As shown in Figure 6, in practical applications, the processing flow of the input stage may include:

The first step is to prompt the user to enter fingerprints in the designated area.

Among them, if it is at the beginning of entry, the global fingerprint template library and its corresponding fingerprint profile and type can be set as empty. If the fingerprint template of the finger is already stored in the global fingerprint template library, then according to the existing information in the template library in the device storage area, determine the fingerprint area that still needs to be entered by the user, and prompt the user on the display device in the form of image display Enter the fingerprint of the designated area of the finger. Refer to the illustration of the graphical user interface shown in FIG. 5B .

The second step is to capture fingerprint images and touch signals.

Among them, the fingerprint authentication system can obtain fingerprint image, touch signal sequence and attitude sensor data respectively after receiving the touch signal. For the implementation of this step, reference may be made to the relevant descriptions in steps S101 and S102.

The third step is to extract the template image, and determine the outline and type of the finger.

Among them, the outline and type of the currently entered finger are estimated by using the multi-frame touch signals obtained in the second step and the data of the attitude sensor and other information. For the implementation of this step, please refer to the related descriptions of determining the morphological feature information according to the touch signal in step S103 and S104 according to the attribute information of the morphological feature information

The fourth step is to determine template triples, including template image, template position, and template direction.

Wherein, according to the touch signal corresponding to the extracted template image, the global position information and orientation information of the fingerprint template in the finger contour are located. For example, after deducing the profile of the entered finger, each entered fingerprint template can calculate the position of the collected fingerprint template on the finger according to the pixel position information of the touch dot matrix on the screen and the absolute position information of the fingerprint sensor collection area on the screen. Global position information and global orientation information in the contour. Referring to FIG. 1D , for example, a reference coordinate system can be determined based on the contour of the fingerprint portion, and then the region position identification, global orientation information, etc. of the collected fingerprint template in the finger contour information can be identified in the reference coordinate system. For the implementation of this step, reference may be made to the detailed description of determining the global position information of the template image in the outline information of the template object in step S104.

The fifth step is to judge whether the current template triplet already exists in the global template library, if so, execute the first step, and if not, execute the sixth step.

The similarity of attribute information between the extracted template image and the template image in the global template library is compared to determine whether the extracted template image exists in the global template library. Exemplarily, the similarity of attribute information can be calculated according to the following formula:

It should be noted that, in some scenarios, weight 2 and weight 3 can also be set to 0.

The sixth step is to add the current template triplet to the global template library.

Among them, the triplet of fingerprint template image information, global position information, and global orientation information is stored in the global fingerprint template database, and the finger number and type are updated and the global finger contour is enriched.

For example, the fingerprint image information, global position information and global orientation information extracted from the current entry are combined into triples, and the similarity is compared with the templates existing in the existing fingerprint global template database. If the similarity is greater than a certain threshold, it is determined that the currently extracted template triplet is already in the global template library. At this time, it can be judged whether the fingerprint location entered by the user is the entry area required by the first step according to the extracted global location information of the template, and if it does not meet the requirements of the entry area, request the user to enter the required area again, and go to the first step . If the similarity is less than or equal to a certain threshold, judge whether the currently extracted template triplet does not exist in the constructed global fingerprint template library, if not, add it to the global fingerprint template library, and use the currently extracted template triplet Finger profile and type update the coverage degree information and type of the finger profile corresponding to the current global fingerprint template library. In one example, the fingerprint template image information, global position information, and global orientation information can be stored as triples in the global fingerprint template library, and the finger number and type can be updated and the coverage degree information of the global finger contour can be enriched. It should be noted that when this entry is the first entry, the finger outline and type corresponding to the current global fingerprint template library can be updated using the finger outline and type extracted from the current entry. If it is the first entry, since the global template library is empty during the first entry, the comparison step of the fifth step can be skipped.

The seventh step is to judge whether to end the input, if not, go to the first step.

Wherein, it can be judged whether the global template library is complete, or whether the number of entries has reached the upper limit of the number of entries, or whether the entry is to be terminated by a combined judgment method.

For example, it can be determined whether the number of entries meets the upper limit of the number of entries. If the upper limit of the number of entries is met, the user is prompted to complete the fingerprint entry. If the upper limit of the number of entries is not reached, it is determined whether the template information in some areas is missing more according to the global location information in the saved fingerprint template library. For example, it can be determined whether the coverage percentage of the coverage contour information is lower than a preset coverage ratio threshold. If there are many missing areas, it is determined that the global fingerprint template library is still incomplete, and a prompt message is output to prompt the user to enter the fingerprints in the missing area. If the template information in the partial area is not missing or is missing less, it can be further determined whether the template information in the global finger contour meets the preset number of templates threshold. If the number of templates does not reach the threshold of the number of templates, the global fingerprint template library is still incomplete, and the user can be further prompted to enter an area with a small number of templates. If the number of templates reaches the threshold of the number of templates, it can be determined that the global fingerprint template library has been constructed, and the number of the currently entered finger is bound to its corresponding global fingerprint profile and type, and the user is prompted to complete the fingerprint entry. Exemplarily, the fingerprint contour can be divided into several areas, for example, a central area and an edge area, and the number of templates in each area is not less than one.

In an optional implementation manner, for example, the main process of the fingerprint identification stage includes: first, infer the attribute information of the current fingerprint to be identified. Exemplarily, the attribute information of the fingerprint may include the type of the finger to which the fingerprint belongs and the approximate area in the global fingerprint template database. Then, the global position information and global orientation information in the triplet of the fingerprint template are combined with the inferred contour to estimate the approximate area of the current fingerprint to be recognized in the global template library. Then, in the fingerprint matching stage, the template image of the inferred finger type and the template image in the approximate region of the inferred template are preferentially matched. With the help of finger type information saved in the global template library and the global position information of each template image, after inferring the type of the finger to which the fingerprint to be recognized belongs and the approximate area to which the global position information belongs, the fingers in the global template library can be filtered out. For template triples whose types match and whose global location information is located in the approximate area of inference, at least one template triple filtered out is preferentially matched with the triple of the current fingerprint to be identified, which can achieve accurate matching and improve fingerprint recognition. speed.

With the technical solutions provided in the embodiments of the present application, if the currently pressed fingerprint belongs to the fingerprint of the finger in the template library, by this method, the fingerprint matching process can be completed in a short time, the fingerprint recognition process under the screen is accelerated, and the user experience is improved.

FIG. 7 is a schematic diagram of processing logic in a fingerprint identification stage in an image matching method provided by an embodiment of the present application.

As shown in Figure 7, in practical applications, the processing flow of the identification stage may include:

The first step is to obtain the fingerprint image and touch signal to be recognized.

Among them, this step is similar to the second step of the fingerprint entry stage. After receiving the touch signal, the fingerprint authentication system determines whether it is a finger press, and if it is a finger press, obtains the fingerprint image, touch signal sequence and additional attitude sensor data generated by the current pressing process.

In the second step, the fingerprint image to be recognized is extracted, and the current finger outline and type are determined.

Among them, it is similar to the third step of the fingerprint entry stage. According to the multi-frame touch signal and gesture sensor data, the outline and type of the currently recorded finger are estimated. The global position information and global orientation information of the fingerprint image to be recognized in the finger contour are located according to the touch signal.

The third step is to infer the triplet of the fingerprint image to be identified and the approximate area to which it belongs.

Among them, for this step, please refer to the step.

The fourth step is to preferentially match the template images of the inferred finger and the inferred region.

In the fifth step, it is judged whether the template image of the inferred finger type or the inferred region has not been matched successfully.

Among them, when the template image of the inferred finger type or the inferred region fails to match successfully, it may be that the inferred finger type or the inferred approximate region is inaccurate, resulting in the unsuccessful matching. Template image.

Step 6: Match other template images of the global template library according to historical statistical sorting information.

The seventh step is to judge whether the matching is successful among other template images in the global template library, if not, confirm that the authentication fails, and if so, confirm that the authentication is successful.

Wherein, after the authentication is successful, the fingerprint identification system can update the historical statistical ranking information to prompt the user that the fingerprint identification is successful, and during the authentication and identification, it can prompt the user that the fingerprint identification fails.

For example, in the global fingerprint template library, each fingerprint template corresponds to a corresponding finger type, global position information, global orientation information and other information. First, according to the current pressing finger type inferred in the second step, the priority matching range is limited to the template image whose finger type is the inferred finger in the global template library. Secondly, by calculating a certain distance metric between the global position information in the template triplet saved in the global template library and the position information of the triplet of the current fingerprint to be identified, it is judged whether the template image in the global template library is located in the third step. The approximate area to which the inferred current fingerprint to be recognized belongs. In one example, if the distance metric value is smaller than the radius specified in the detailed description of the third step, it is determined that the template image is located within the approximate area. Furthermore, when performing template matching, the template images belonging to the approximate region are preferentially matched.

It should be noted that, the inferred finger type and the inferred area can be preferentially matched here, and the matching process of the following two situations can be adopted:

In one example, the fourth step is performed first to firstly match the template image of the inferred area of the inferred finger type, secondly to match the template image of the non-inferred area of the inferred finger type, and then to match the template image of the inferred area of the non-inferred finger type, and finally transfer to The fifth step is to match non-inferred template regions of non-inferred finger types according to historical statistical ranking information. If a match succeeds at any of these steps, the match is terminated.

In another example, the fourth step is firstly performed to firstly match the template image of the inferred area of the inferred finger type, secondly match the template image of the inferred area of the non-inferred finger type, then match the non-inferred template area of the inferred finger type, and finally transfer to The fifth step is to match the non-inferred template region of the non-inferred finger type according to the historical statistical sorting information; if the matching is successful in any step, the matching is terminated.

The method of matching based on the inferred finger type with high confidence and the template image of the inferred area is adopted. Since most fingerprint recognition processes can be successfully authenticated in the fourth step, the subsequent process of matching other template images can be avoided as much as possible. It can effectively accelerate the recognition speed of trusted fingerprints, improve the speed of fingerprint authentication, and reduce power consumption to a certain extent.

Based on the above-mentioned technical solutions in the fingerprint entry stage and the fingerprint matching identification stage, the image matching method provided by the embodiments of the present application can solve various technical problems in the fingerprint identification scene.

On the one hand, in the fingerprint entry stage, establishing a global fingerprint template library of fingers can avoid the problem of recognition failure or low recognition efficiency due to incomplete templates. The fingerprint template library is mainly used to judge whether the pressed fingerprint exists in the template library when the user uses fingerprint authentication, or, the similarity with the template image in the template library is high enough, that is, the above-mentioned embodiment mentioned in the image similarity degree. matching process.

In the fingerprint entry stage, a sufficient fingerprint template library is saved for trusted users, which can improve the user's unlocking accuracy in the later fingerprint identification stage, and help to quickly locate the fingerprint image related to the captured fingerprint image in the later fingerprint identification stage. Template image. For example, relative to the information in the template library, it only contains the partial image information (or called partial topology) extracted from each fingerprint image entry, and does not include the fingerprint image (or called partial fingerprint) in the entry of finger fingerprints (or called partial fingerprint). In the case of absolute position information (or referred to as global topology) in the global fingerprint), the method provided by the embodiment of the present application can quickly perform the screening and matching of template images, thereby improving the matching efficiency. For another example, the technical solutions provided by the embodiments of the present application can help the fingerprint system to determine whether the currently constructed template library is sufficient to enable the user to unlock effectively, for example, whether the template library contains the user's edge fingerprint.

On the other hand, in the fingerprint identification stage, by screening template images, the problem of low matching rate due to too many template images can be solved. In particular, by quickly locating the position of the captured fingerprint image in the global fingerprint template library, the number of template images that need to be judged for matching can be reduced, thereby improving the matching efficiency, and at the same time, based on the global template library success rate. Among them, for example, in fingerprint authentication, the authentication speed and accuracy mainly affect the user's experience. The technical solutions provided by the embodiments of the present application rapidly and accurately locate the position of the fingerprint image in the fingerprint template database for rapid matching, thereby accelerating the fingerprint recognition speed. In addition, with the help of the fingerprint template library under the global topology, the accuracy of fingerprint identification can be maximized as much as possible, and the user's edge fingerprints can be prevented from being rejected by the fingerprint identification system.

In addition, the solution provided by the embodiments of the present application to accelerate the fingerprint recognition under the screen based on the touch signal of the finger pressing the screen can improve the fingerprint recognition speed and unlocking accuracy under the screen without adding a hardware module specially used for image processing. In addition, since multiple attribute information is matched during identification, for example, successful template matching needs to ensure that the similarity of fingerprint image information, global position information and global orientation information are all higher than a certain threshold, that is, the triple information is used for matching. Therefore, The technical solutions provided by the embodiments of the present application can also improve the security of fingerprint recognition under the screen without affecting the user experience.

Embodiment 5

The image matching method provided in the embodiment of the present application can be deployed in an electronic device having an image acquisition device.

In an example, the electronic device may include the following components: an input unit, a processor unit, a communication unit, a storage unit, an output unit, and a power supply and other electronic devices.

Wherein, the input unit includes a fingerprint image collector, an attitude sensor and a touch panel. The fingerprint image collector includes a sensor capable of extracting fingerprint information, but is not limited to a capacitive fingerprint sensor, an optical fingerprint sensor or an ultrasonic fingerprint sensor. The attitude sensor is a high-performance three-dimensional motion attitude measurement system based on MEMS technology. It contains motion sensors such as a three-axis gyroscope, a three-axis accelerometer, and a three-axis electronic compass. The sensor is mainly used to capture the gesture information of the touch panel. In the actual fingerprint recognition system under the screen, this device is an optional device. A touch panel, also known as a touch screen or a touch screen, collects the actions of a user touching or approaching it. For example, the user uses any suitable object or accessory, such as a finger, a stylus, or an operation action on the touch panel or a position close to the touch panel, and drives the corresponding connection device according to a preset program. Optionally, the touch panel may include two parts, a touch detection device and a touch controller. The touch detection device detects the user's touch operation, converts the detected touch operation into an electrical signal, and transmits the electrical signal to the touch controller; the touch controller receives the electrical signal from the touch detection device and converts it into an electrical signal. Contact coordinates are sent to the processing unit. The touch controller can also receive and execute commands from the processing unit. In addition, the touch panel can be realized by various types such as resistive type, capacitive type, infrared (Infrared), and surface acoustic wave.

The processor unit is the control center of the electronic equipment. It uses various interfaces and lines to connect various parts of the entire electronic equipment. It runs or executes the software programs and/or modules stored in the storage unit, and calls the data stored in the storage unit. , to perform various functions of the electronic device and/or process data. The processor unit may be composed of an integrated circuit (Integrated Circuit, IC for short), for example, may be composed of a single packaged IC, or may be composed of a plurality of packaged ICs with the same function or different functions. For example, the processor unit may only include a central processing unit (Central Processing Unit, CPU for short), or may be a GPU, a digital signal processor (Digital Signal Processor, DSP for short), and a control chip (such as a baseband) in the communication unit chip) combination. In this implementation, the processor unit is mainly used to run the fingerprint entry and identification system.

The communication unit is used for establishing a communication channel, enabling the electronic device to connect to the remote server through the communication channel, and download media data from the remote server.

The output unit includes, but is not limited to, an image output unit and a sound output unit. The video output unit is used to output text, pictures and/or videos. The image output unit may include a display panel, such as a display configured in the form of an LCD (Liquid Crystal Display, liquid crystal display), an OLED (Organic Light-Emitting Diode, organic light-emitting diode), a field emission display (field emission display, FED for short), etc. panel. Alternatively, the image output unit may include a reflective display, such as an electrophoretic display, or a display utilizing Interferometric Modulation of Light. The image output unit may include a single display or multiple displays of different sizes. In a specific embodiment of the present invention, the touch panel used in the above-mentioned input unit can also be used as a display panel of the output unit at the same time. For example, when the touch panel detects a touch or an approaching gesture operation on it, it is transmitted to the processing unit to determine the type of the touch event, and then the processing unit provides corresponding visual output on the display panel according to the type of the touch event.

The storage unit can be used to store software programs and modules, and the processing unit executes various functional applications of the electronic device and implements data processing by running the software programs and modules stored in the storage unit.

Power supplies are used to power different parts of an electronic device to keep it running. Exemplarily, the power source can be a built-in battery, such as a common lithium-ion battery, a nickel-metal hydride battery, etc., and also includes an external power source that directly supplies power to the electronic device, such as an AC adapter.

Embodiment 6

Embodiments of the present application further provide an image matching apparatus. FIG. 8 is a first structural schematic diagram of an image matching apparatus provided by an embodiment of the present application. As shown in FIG. 8, the present application provides an image matching apparatus 1000, including:

A determination module 1001, configured to determine attribute information of an image to be identified according to morphological feature information of the object to be detected, wherein the image to be identified is used to identify the object to be detected;

A matching module 1002, configured to screen at least one template image according to the attribute information of the image to be recognized, wherein the attribute information of the filtered template image and the attribute information of the image to be recognized satisfy the attribute correlation condition; and, in In the filtered template images, the images matching the images to be recognized are queried.

In an optional embodiment, the apparatus 1000 may further include: an acquisition module 1003, configured to acquire the image to be recognized, and acquire the touch signal of the object to be detected; and, according to the touch signal of the object to be detected, determine Morphological feature information of the object to be detected; wherein the time difference between the moment when the image to be recognized is acquired and the moment when the touch signal of the object to be detected is acquired is less than a time deviation threshold.

In an optional implementation manner, the matching module 1002 is specifically configured to screen at least one template image according to at least two types of attribute information of the image to be recognized and the weight of each attribute information.

In an optional implementation manner, the matching module 1002 is further configured to: when the number of the filtered template images is 0, or, in the filtered template images, no target template matching the to-be-recognized image is queried. When creating an image, among other template images in at least one template image except the filtered template images, an image matching the to-be-recognized image is queried.

In an optional implementation manner, the matching module 1002 is further configured to: when the number of the filtered template images is 0, or, in the filtered template images, no target template matching the to-be-recognized image is queried. When the image is selected, it is sorted according to the historical matching success rate corresponding to the at least one template image, and in the at least one template image, the image matching the image to be recognized is queried.

In an optional implementation manner, the matching module 1002 is further configured to obtain the template entity identifier and/or use authority corresponding to the target template image when the target template image matching the to-be-recognized image is queried; and, in When no target template image matching the to-be-identified image is found, it is determined that the to-be-detected object corresponding to the to-be-matched image does not have the right to use.

In an optional implementation manner, the acquiring module 1003 is further configured to acquire at least one template image and the form of the template object corresponding to each template image before screening the at least one template image according to the attribute information of the image to be recognized feature information; each template image is in one-to-one correspondence with each template object; the determining module 1001 is further configured to determine attribute information of each template image according to the morphological feature information of the template object corresponding to each template image.

In an optional implementation manner, the acquiring module 1003 is specifically configured to acquire P-1 first template images of the first template object, and the outline of the first template object corresponding to the P-1 first template images The determining module 1001 is specifically configured to determine the region position identifier of each first template image in the corresponding contour information according to the contour information of the first template object corresponding to each first template image; the determining module 1001 is further configured to Judging whether the set of area location identifiers corresponding to the P-1 first template images satisfies the coverage condition; if not, instruct the acquisition module 1003 to obtain the Pth first template image of the first template object; wherein, the coverage conditions include: coverage The edge coverage condition of the edge region of the first template object, or, satisfies the global coverage condition of covering the contour information of the first template object; P is an integer greater than or equal to 2.

In an optional implementation manner, the determining module 1001 is further configured to determine the supplementary mining area according to the set of area location identifiers corresponding to the at least one first template image and the coverage conditions; the apparatus further includes: an output module 1004, configured to Output prompt information, where the prompt information is used to prompt the input of the template image of the first template object including the supplementary mining area.

The image matching apparatus provided in the embodiments of the present application can be used to execute the methods in the foregoing embodiments. For details of other technical solutions and technical effects of the embodiments of the present application, reference may be made to the descriptions in other embodiments of the present application.

Embodiments of the present application further provide an image matching apparatus. FIG. 9 is a first structural schematic diagram of an image matching apparatus provided by an embodiment of the present application. As shown in FIG. 9, the present application provides an image matching apparatus 1100, including:

The obtaining module 1103 is configured to obtain at least one first template image of the template object; and, obtain contour information of the template object, wherein the contour information of the template object is used to determine the position of the at least one first template image in the contour information of the template object. area location identifier;

A determination module 1101, configured to determine whether the set of area location identifiers corresponding to at least one first template image satisfies the coverage condition, and when the coverage condition is not met, instruct the acquisition module to acquire the second template image of the template object; wherein at least one first template image The template image and the second template image are used to query the image matching the image to be recognized.

In an optional implementation manner, the apparatus 1100 may further include a matching module 1102, configured to query at least one of the first template image and the second template image for an image that matches the image to be recognized.

In an optional implementation manner, the coverage condition includes at least one of the following: covering at least one partial area of the template object; covering the edge area of the template object exceeds a preset coverage ratio; covering the outline information of the template object exceeds a preset coverage ratio .

In an optional implementation manner, the determining module 1101 is further configured to determine the supplementary mining area according to the set of location identifiers corresponding to the at least one first template image and the coverage conditions; the apparatus further includes: an output module 1104, configured to Output prompt information, where the prompt information is used to prompt input of the second template image of the template object including the supplementary mining area.

FIG. 10 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As shown in FIG. 10, an embodiment of the present application provides an electronic device 1200, including: a processor 1210, a memory 1220, and an input and output device 1230; wherein:

The processor 1210 may be configured to execute the execution instructions to cause the electronic device to execute any of the image matching methods in the foregoing embodiments.

The memory 1220 can be used to store program instructions and data involved in any of the image matching methods in the foregoing embodiments.

The input and output devices 1230 may include data input and output devices and data input and output devices. The input and output device 1230 may be used to acquire or output various information involved in performing the above-mentioned image matching method.

For example, the input and output device 1230 may include: an image acquisition device and a morphological feature acquisition device.

Wherein, the morphological feature collection device may be a touch device, and exemplarily, it may be a touch screen. The image collection device may be a fingerprint sensor, a fingerprint collector, or the like for collecting fingerprint images. In one example, the fingerprint sensor is located on the side of the touch screen close to the body of the electronic device. In addition, the touch screen can also be used to output prompt information. See Figure 1C.

In a first optional implementation of the electronic device:

an image acquisition device, which can be used to acquire an image to be recognized of the object to be detected;

A morphological feature collection device, which can be used to collect morphological feature information of the object to be detected;

The processor is used to determine the attribute information of the image to be recognized according to the morphological feature information of the object to be detected; and, according to the attribute information of the image to be recognized, screen at least one template image, wherein the attribute information of the filtered template image is The attribute correlation condition is satisfied with the attribute information of the image to be recognized; and, in the filtered template image, the image matching the image to be recognized is queried.

In a second optional embodiment of the electronic device:

An image acquisition device, which can be used to acquire at least one first template image of the template object; and, when the set of area position identifiers corresponding to the at least one first template image does not satisfy the coverage condition, acquire the second template image of the template object; wherein, at least one of the first template image and the second template image is used to query the image matching the image to be recognized;

The touch device can be used to obtain contour information of the template object, wherein the contour information of the template object is used to determine a region position identifier of the at least one first template image in the contour information of the template object.

In an optional implementation manner, the touch control device is further configured to output prompt information when the set of area location identifiers corresponding to the at least one first template image does not satisfy the coverage condition, and the prompt information is used to prompt the image acquisition device to pass the Enter the second template image of the template object.

The electronic device provided in this embodiment of the present application can be used to execute the image matching method described in any one of the foregoing embodiments.

For details of other technical solutions and technical effects of the embodiments of the present application, reference may be made to the descriptions in other embodiments of the present application.

In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the present application result in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device. Computer instructions may be stored on or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server, or data center over a wire (e.g. coaxial cable, optical fiber, digital subscriber line) or wireless (eg infrared, wireless, microwave, etc.) means to another website site, computer, server or data center. A computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like that includes an integration of one or more available media. Useful media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk), among others.

Claims

An image matching method applied to fingerprint identification, characterized in that the method comprises:

According to the morphological feature information of the object to be detected, the attribute information of the image to be recognized is determined, wherein the image to be recognized is used to recognize the object to be detected;

Screening at least one template image according to the attribute information of the to-be-recognized image; the attribute information of the filtered template image and the attribute information of the to-be-recognized image satisfy the attribute correlation condition;

In the filtered template image, the image matching the to-be-recognized image is queried.
The method according to claim 1, wherein before the determining the attribute information of the to-be-recognized image according to the morphological feature information of the to-be-detected object, the method comprises:

obtaining the to-be-recognized image;

acquiring the touch signal of the object to be detected;

According to the touch signal of the object to be detected, determine the morphological feature information of the object to be detected;

Wherein, the time difference between the time when the image to be recognized is obtained and the time when the touch signal of the object to be detected is obtained is less than a time deviation threshold.
The method according to claim 2, wherein the touch signal of the object to be detected is collected by a touch device;

Before determining the morphological feature information of the to-be-detected object according to the touch-control information of the to-be-detected object, the method further includes: acquiring three-dimensional position information of the touch-control device in the geodetic coordinate system;

The determining the attribute information of the image to be recognized according to the morphological feature information of the object to be detected includes: determining the morphological feature information of the object to be detected and the three-dimensional position information of the touch control device to determine the image to be recognized. Identify the attribute information of the image.
The method according to any one of claims 1-3, wherein the morphological feature information of the object to be detected includes at least one of the following information: contour information, position information, and kinematic information of the object to be detected;

The attribute information of the to-be-recognized image includes at least one of the following information: the object type of the to-be-detected object, the global position information of the to-be-recognized image in the outline information of the to-be-detected object, and the Detect the habitual pose information of the object.
The method according to any one of claims 1-4, wherein the object to be detected is a fingerprint part to be recognized; the image to be recognized is a fingerprint image of the fingerprint part to be recognized collected by a fingerprint sensor.
The method according to any one of claims 1-5, wherein the determining the attribute information of the to-be-recognized image according to the morphological feature information of the to-be-detected object comprises: according to the outline of the to-be-detected object information, determine the object type of the object to be detected; take the object type of the object to be detected as the attribute information of the image to be recognized;

Wherein, the object type of the object to be detected includes at least one of the following: left and right hand information, and finger type.
The method according to any one of claims 1-6, wherein the determining the attribute information of the to-be-recognized image according to the morphological feature information of the to-be-detected object comprises: according to the outline of the to-be-detected object information, determine the global position information of the image to be recognized in the contour information of the object to be detected; take the global position information corresponding to the image to be recognized as the attribute information of the image to be recognized;

Wherein, the global position information of the image to be recognized in the contour information of the object to be detected includes at least one of the following: a distance from a reference point in the contour information of the object to be detected The outline information of the object is the regional position identification of the reference system.
The method according to any one of claims 1-7, wherein the determining the attribute information of the to-be-recognized image according to the morphological feature information of the to-be-detected object comprises: according to the outline of the to-be-detected object at least one morphological feature information among information, position information, and kinematics information, to determine the habitual posture information of the object to be detected; use the habitual posture information of the object to be detected as the attribute information of the image to be recognized;

Wherein, the habitual posture information of the object to be detected includes at least one of the following: global direction information of the object to be detected, the speed of the object to be detected, the moving direction of the object to be detected, the object to be detected movement trajectory.
The method according to any one of claims 1-5, wherein the attribute correlation condition includes at least one of the following:

When the attribute information of the to-be-recognized image includes the object type corresponding to the to-be-recognized image, the object type corresponding to the filtered template image is the same as the object type corresponding to the to-be-recognized image;

When the attribute information of the to-be-recognized image includes gesture information corresponding to the to-be-recognized image, the habitual gesture information corresponding to the filtered template image is the same as the habitual gesture information corresponding to the to-be-recognized image;

When the attribute information of the to-be-recognized image includes the global location information corresponding to the to-be-recognized image, the global location information corresponding to the filtered template image and the global location information corresponding to the to-be-recognized image satisfy the location proximity condition;

The location proximity condition includes: the location indicated by the global location information corresponding to the filtered template image includes the location indicated by the global location information corresponding to the to-be-recognized image, or the filtered template image There is an intersection between the position indicated by the corresponding global position information and the position indicated by the global position information corresponding to the to-be-recognized image, or, the position indicated by the global position information corresponding to the filtered template image and the to-be-identified image. The distance between the positions represented by the global position information corresponding to the image is smaller than the adjacent area distance threshold.
The method according to any one of claims 1-9, wherein the quantity of the attribute information of the to-be-recognized image is at least two;

The screening of at least one template image according to the attribute information of the image to be recognized includes: screening the at least one template image according to at least two kinds of attribute information of the image to be recognized and the weight of each attribute information .
The method according to any one of claims 1-10, wherein the number of the filtered template images is 0, or, in the filtered template images, no image matching the to-be-recognized image is queried. When the target template image is used, the method further includes:

Among other template images in the at least one template image except the filtered template images, an image matching the to-be-recognized image is queried.
The method according to any one of claims 1-10, wherein the number of the filtered template images is 0, or, in the filtered template images, no image matching the to-be-recognized image is queried. When the target template image is used, the method further includes:

According to the ranking of the historical matching success rate corresponding to the at least one template image, in the at least one template image, the image matching the to-be-recognized image is queried.
The method according to any one of claims 1-12, wherein the method further comprises:

When the target template image matching the to-be-recognized image is queried, obtain the template entity identifier and/or usage authority corresponding to the target template image;

When no target template image matching the to-be-identified image is found, it is determined that the to-be-detected object corresponding to the to-be-matched image does not have the right to use.
The method according to any one of claims 1-13, wherein before the screening of at least one template image according to the attribute information of the to-be-recognized image, the method comprises:

obtaining the morphological feature information of the at least one template image and the template object corresponding to each template image; each template image is in one-to-one correspondence with each template object;

The attribute information of each template image is determined according to the morphological feature information of the template object corresponding to each template image.
The method of claim 14, wherein:

The acquiring the at least one template image and the morphological feature information of the template object corresponding to each template image includes: acquiring P-1 first template images of the first template object, and, the P-1 first templates Outline information of the first template object corresponding to the image;

The determining the attribute information of each template image according to the morphological feature information of the template object corresponding to each template image includes: determining each first template image according to the outline information of the first template object corresponding to each first template image. The region location identification in the corresponding contour information;

The method further includes: judging whether the set of area location identifiers corresponding to the P-1 first template images satisfies the coverage condition; if not, acquiring the Pth first template image of the first template object;

The coverage conditions include: edge coverage conditions covering the edge region of the first template object, or satisfying a global coverage condition covering contour information of the first template object; P is an integer greater than or equal to 2.
An image matching method applied to fingerprint identification, comprising:

obtaining at least one first template image of the template object;

acquiring contour information of the template object, wherein the contour information of the template object is used to determine a region position identifier of the at least one first template image in the contour information of the template object;

when the set of area location identifiers corresponding to the at least one first template image does not satisfy the coverage condition, collecting a second template image of the template object;

Wherein, the at least one first template image and the second template image are used to query the image matching the image to be recognized.
The method according to claim 16, wherein the coverage condition comprises at least one of the following:

covering at least one local area of the template object;

Covering the edge area of the template object exceeds a preset coverage ratio;

The outline information covering the template object exceeds a preset coverage ratio.
The method according to claim 16 or 17, wherein the acquiring the outline information of the template object comprises: acquiring the outline information of the template object through a touch screen of an electronic device;

The acquiring at least one first template image of the template object includes: acquiring the at least one first template image through a fingerprint sensor;

The fingerprint sensor is located on a side of the touch screen close to the body of the electronic device.
The method according to claim 17 or 18, wherein before the acquiring the second template image of the template object, the method further comprises:

determining a supplementary mining area according to the set of area location identifiers corresponding to the at least one first template image and the coverage condition;

Prompt information is output, where the prompt information is used to prompt the input of a second template image of the template object including the supplementary acquisition area.
An electronic device, characterized in that it comprises: a processor and a memory, the memory is used for storing instructions, and the processor is used for executing the instructions, so that the electronic device executes any one of claims 1-19 method.
A computer storage medium, characterized by comprising a computer program, which, when executed on an electronic device, causes the electronic device to execute the method of any one of claims 1-19.