JP2020126302A - Image processing apparatus, image processing method and image processing program - Google Patents

Abstract

To provide an image processing apparatus, an image processing method and an image processing program capable of improving authentication accuracy.SOLUTION: An image processing apparatus includes a saturation region detection unit that detects a saturation region of a pixel from a biological image obtained by photographing a biological object moving while contacting a touch panel, and a correction unit that corrects a scale of the biological image or a biological feature extracted from the biological image according to a detection result by the saturation region detection unit.SELECTED DRAWING: Figure 6

Description

The present invention relates to an image processing device, an image processing method, and an image processing program.

Development of biometric authentication that can be used in small mobile terminals is underway. Most mobile terminals are used by holding the terminal with a hand and operating a touch panel or the like provided in the terminal. By using such a structure peculiar to the mobile terminal and tracing the operation guide displayed on the touch panel, it is possible to read the biometric characteristics required for verification while keeping the posture of the biological body stable.

On the other hand, if the registered biometric image and the matching biometric image differ in distance from the image capturing device at the time of image capturing, it is required to match the reduced scale of the images. To deal with this, the distance from the imaging device to the living body is estimated from the relationship between the movement amount of the living body on the touch panel and the movement amount on the image, and scale matching is performed on both images according to the estimated distance. Therefore, a technique for ensuring the authentication accuracy is disclosed (for example, see Patent Document 1).

JP, 2017-136136, A

However, when the photographing device and the living body are close to each other, the intensity of the illumination light becomes strong, and white spots occur on the image, which makes it difficult to detect the distance. As a result, the authentication accuracy decreases.

In one aspect, an object of the present invention is to provide an image processing device, an image processing method, and an image processing program capable of image processing capable of improving authentication accuracy.

In one aspect, the image processing device provides a saturated area detection unit that detects a saturated area of a pixel from a biometric image obtained by capturing an image of a living body that moves while touching a touch panel, and a detection result by the saturated area detection unit. Accordingly, a correction unit that corrects the scale of the biometric image or the biometric feature extracted from the biometric image is provided.

This enables image processing that can improve authentication accuracy.

(A)-(c) is a figure for demonstrating the overexposure which arises in an image. (A)-(c) is a figure for demonstrating the relationship between distance and a scale. FIG. 6 is a diagram for explaining overexposure. (A) is a block diagram for explaining the hardware configuration of the image processing apparatus according to the first embodiment, and (b) is a block diagram of each function realized by executing an image processing program. It is a figure which illustrates the flowchart which an image processing apparatus performs. FIG. 3 is a diagram for explaining the outline of the first embodiment. It is a figure which illustrates extraction of a palm area. (A) is a figure which illustrates the flowchart which the image processing apparatus which concerns on Example 2 performs, (b) is a figure which illustrates the table stored in the table. FIG. 9 is a diagram illustrating a flowchart executed by an image processing apparatus according to a third embodiment.

Prior to the description of the embodiments, whiteout that occurs in a part of an image will be described. FIG. 1A is a diagram illustrating the image processing device 200. As illustrated in FIG. 1A, the image processing device 200 includes a touch panel 201 and a biosensor 202. The biometric sensor 202 is equipped with an illumination device such as an LED and a photographing device.

For example, as illustrated in FIG. 1A, the authentication guide is displayed on the screen of the touch panel 201. In the example of FIG. 1A, two authentication guides are displayed. The user of the image processing apparatus 200 traces these two authentication guides with two designated fingers, as illustrated in FIGS. 1B and 1C. For example, the user touches each of the authentication guides with two fingers, the thumb and the index finger, and moves the hand along the guide (for example, an arrow). In this case, the palm of the user moves while facing the touch panel 201.

The living body sensor 202 illuminates the palm with light. The imaging device of the biometric sensor 202 uses the reflected light from the palm to image the palm. Specifically, the imaging device of the biometric sensor 202 captures a predetermined range of the palm by continuously capturing the divided images. The image processing apparatus 200 reads the biometric feature from the captured palm.

However, the height of the palm becomes unstable due to individual differences in the size of the palm, the length of the finger, and the operation on different days, so that the height of the palm varies. For example, FIG. 2A is a diagram illustrating a case where the distance between the palm and the imaging device of the biometric sensor 202 is an appropriate value. FIG. 2B is a diagram illustrating a case where the distance between the palm and the imaging device of the biometric sensor 202 is smaller than an appropriate value.

If the case of FIG. 2A is when the biometric feature of the user is registered, and the case of FIG. 2B is when the user is authenticated, as shown in FIG. 2C, the biometric image for verification is used. The biometric image for collation or the biometric feature extracted from the biometric image is reduced according to the distance detected from the biometric image. As a result, the scales of the biometric image for registration and the biometric image for verification can be matched and verification can be performed, and the stability of biometric authentication can be maintained.

However, when the palm is close to the imaging device of the biometric sensor 202, the light intensity of the illumination light emitted from the illumination of the biometric sensor 202 becomes high. In this case, as illustrated in FIG. 3, whiteout may occur in part of the image. In FIG. 3, the shaded portion represents overexposure. When whiteout occurs, the accuracy in detecting the distance from the biometric image decreases. Thereby, the detection of the distance fails or a distance different from the actual distance is detected. As a result, the scale adjustment accuracy of the biometric image for collation decreases, and the stability of biometric authentication may decrease.

In the following embodiments, an image processing apparatus, an image processing method, and an image processing program that enable image processing that can improve authentication accuracy by utilizing whiteout will be described.

FIG. 4A is a block diagram for explaining the hardware configuration of the image processing apparatus 100 according to the first embodiment. As illustrated in FIG. 4A, the image processing device 100 includes a CPU 101, a RAM 102, a storage device 103, a display device 104, an input device 105, a biosensor 106, and the like. Each of these devices is connected by a bus or the like.

A CPU (Central Processing Unit) 101 is a central processing unit. The CPU 101 includes one or more cores. A RAM (Random Access Memory) 102 is a volatile memory that temporarily stores a program executed by the CPU 101, data processed by the CPU 101, and the like.

The storage device 103 is a non-volatile storage device. As the storage device 103, for example, a ROM (Read Only Memory), a solid state drive (SSD) such as a flash memory, a hard disk driven by a hard disk drive, or the like can be used. The image processing program according to the present embodiment is stored in the storage device 103.

The display device 104 is a liquid crystal display, an electroluminescence panel, or the like, and displays a result of each process described below and the like. The input device 105 is a touch sensor and is provided on the display device 104. Thereby, the display device 104 and the input device 105 function as a touch panel. The input device 105 can detect a region in which the user touches the screen of the display device 104 with a living body such as a finger of a hand.

The biometric sensor 106 is a sensor including an illumination 107, an imaging device 108, and the like. The illumination 107 irradiates a living body such as a palm with light. The imaging device 108 acquires the biometric image of the user by using the reflected light from the biological body. Specifically, the image capturing device 108 captures a predetermined range of the living body by continuously capturing the divided images.

The image processing program stored in the storage device 103 is expanded in the RAM 102 in an executable manner. The CPU 101 executes the image processing program loaded in the RAM 102. Thereby, each processing by the image processing apparatus 100 is executed. By executing the image processing program, registration processing, authentication processing, etc. are executed.

The registration process is a process of registering the biometric feature extracted from the biometric image for registration acquired by the biometric sensor 106 as a registered biometric feature. In this embodiment, the local feature extracted from the palm image is registered as the biometric feature. Specific wrinkles, which are part of the palm and palm prints, can be used as the local features. The authentication process is a process of matching the registered biometric feature with the verification biometric feature extracted from the biometric image for verification acquired by the biometric sensor 106 during the authentication process. In the present embodiment, as an example, if the similarity between the verification biometric feature acquired during the authentication process and the registered biometric feature is greater than or equal to a threshold value, it is determined that the person to be authenticated is the same person as the registered user. To be done.

FIG. 4B is a block diagram of each function realized by executing the image processing program. By executing the image processing program, the contact position detection unit 10, the guide movement unit 20, the movement amount calculation unit 30, the living body image acquisition unit 40, the whiteout detection unit 50, the table 55, the distance detection unit 60, the scale conversion unit 70, the living body. It functions as the feature extraction unit 80, the authentication unit 90, and the like.

The display device 104 displays an authentication guide and an arrow on the screen. The contact position detection unit 10 detects the contact position of the user's living body on the input device 105. The guide moving unit 20 determines whether the contact position of the user has moved along the authentication guide. When it is determined that the contact position of the user has moved along the authentication guide, the movement amount calculation unit 30 detects the change in the contact position detected by the contact position detection unit 10 to determine the movement amount of the contact position. calculate.

The illumination 107 illuminates the living body of the user with illumination light. The biometric image acquisition unit 40 acquires a biometric image from the imaging device 108. The whiteout detection unit 50 detects a whiteout area in the biometric image acquired by the biometric image acquisition unit 40. The whiteout area is a saturated area in which pixels are saturated in the biometric image. For example, when the brightness value is detected as a value from 0 to 255, the area has a brightness value of 255. The whiteout detection unit 50 can detect the whiteout area by determining the presence or absence of the saturated area of the pixel from the biometric image. The whiteout detection unit 50 determines the presence or absence of the whiteout area by detecting the whiteout area in the biometric image. The whiteout detection unit 50 also calculates the area of the detected whiteout area and the like. For example, the whiteout detection unit 50 may determine that there is a whiteout area when the area of the saturated region exceeds a predetermined threshold in order to suppress false detection of noise.

The distance detection unit 60 detects the distance between the palm and the imaging device 108 by using the moving amount of the living body in the image acquired by the biological image acquiring unit 40 and the moving amount of the contact position calculated by the moving amount calculating unit 30. .. For example, the distance detection unit 60 detects the distance between the palm and the biometric sensor 106 by utilizing the fact that the amount of movement of the feature point in the image changes according to the distance between the palm and the imaging device 108.

The scale conversion unit 70 corrects the scale of the biometric image using the detection result of the whiteout detection unit 50 or the detection result of the distance detection unit 60. The biometric feature extraction unit 80 extracts biometric features from the biometric image. The authentication unit 90 compares the collation biometric feature extracted by the biometric feature extraction unit 80 with the registered biometric feature registered in advance, and authenticates the person to be authenticated if the similarity is equal to or more than a threshold value.

FIG. 5 is a diagram illustrating a flowchart executed by the image processing apparatus 100. As illustrated in FIG. 5, the biometric image acquisition unit 40 acquires a biometric image (step S1). Next, the whiteout detection unit 50 determines whether the biometric image acquired in step S1 includes a whiteout area (step S2).

If it is determined to be “Yes” in step S2, it means that overexposure has occurred in the biometric image, and the distance between the palm and the imaging device 108 is small. Therefore, it is desired to reduce the scale of the biometric image. Therefore, when it is determined to be “Yes” in step S2, the scale conversion unit 70 acquires the individual scale (a value smaller than 1, for example, 0.8) stored in the table 55 as the scaling factor ( Step S3).

When it is determined to be “No” in step S2, it means that there is no whiteout in the biometric image. Therefore, when it is determined as “No” in step S2, the distance detection unit 60 detects the distance between the palm and the image capturing apparatus 108 (step S4). Next, the scale conversion unit 70 calculates the magnification according to the distance detected in step S4 (step S5). For example, the smaller the distance, the smaller the magnification, and the larger the distance, the larger the magnification.

After execution of step S3 or step S5, the scale conversion unit 70 corrects the scale of the biometric image by multiplying the scale of the biometric image by the scale factor (step S6). By this processing, the biometric image is reduced if the magnification is small and the biometric image is enlarged if the magnification is large. The biometric feature extraction unit 80 extracts a biometric feature from the scale-adjusted biometric image as a matching biometric feature (step S7). Next, the authentication unit 90 collates the collated biometric feature with the registered biometric feature (step S8). For example, if the similarity between the matching biometric feature and the registered biometric feature is greater than or equal to the threshold, the authentication unit 90 authenticates the user who is the authenticated user as the user.

According to the present embodiment, the presence/absence of a saturated region of a pixel is detected from a biometric image obtained by capturing an image of a living body moving while touching a touch panel. As illustrated in FIG. 6, when a saturated region is detected in the biometric image, the scale of the biometric image is corrected. In this embodiment, the biometric image is reduced using a fixed magnification. According to this configuration, the scale is matched between the biometric image for registration and the biometric image for matching by utilizing the whiteout. That is, it is possible to perform image processing that can improve authentication accuracy.

Although the scale of the biometric image is corrected in this embodiment, the present invention is not limited to this. For example, the scale of the biometric feature extracted from the biometric image may be corrected. The same applies to the following examples.

Since the background of the palm may be detected as a saturated region, the biometric image acquisition unit 40 may extract the palm region from the image acquired by the imaging device 108 and acquire the palm region as the biometric image. .. The same applies to the following examples. For example, as illustrated in FIG. 7, the biometric image acquisition unit 40 detects the contour of the hand from the biometric image for matching by using the edge method or the like, and extracts the inside of the contour of the hand as a palm region, Acquire as a biometric image. In this case, it is possible to suppress the detection of a saturated region other than the saturated region of the palm. When the whiteout area is detected from the biometric image of the palm area, the biometric image or the biometric feature extracted from the biometric image is reduced. Then, the collated biometric feature and the registered biometric feature are collated.

In the first embodiment, if the whiteout area is detected, the scale correction is performed using one individual scale, but the present invention is not limited to this. For example, from the illumination of the biometric sensor 106, diverging light is emitted so that the entire image capturing range of the camera is captured. If the intensity of the light is constant, the size of the whiteout area changes according to the distance between the palm and the biometric sensor 106. The second embodiment utilizes this phenomenon.

For example, the whiteout area becomes maximum when the palm comes closest to the biosensor 106, and the whiteout area becomes smaller as the palm moves away from the biosensor 106. When the palm of the hand further moves away from the biosensor 106, the whiteout area disappears. Utilizing this phenomenon, the scale of the biometric image is adjusted by applying individual magnifications according to the size of the whiteout area. In this case, the relationship between the whiteout area and the coefficient is stored in the table 55 in advance.

FIG. 8A is a diagram illustrating a flowchart executed by the image processing apparatus 100. The flowchart of FIG. 8A differs from the flowchart of FIG. 5 in that step S3′ is executed instead of step S3. The step S3' will be described. When it is determined to be “Yes” in step S2, the whiteout detection unit 50 calculates the area of the detected whiteout area. The scale conversion unit 70 reads the magnification corresponding to the area of the whiteout area from the table 55 (step S3′).

FIG. 8B is a diagram illustrating a table stored in the table 55. As illustrated in FIG. 8B, the area of the whiteout area and the magnification are stored in association with each other. This table can be created in advance. A predetermined range (width) may be set for the area associated with each magnification.

According to this embodiment, the magnification corresponding to the area of the whiteout area is acquired. With this configuration, the scales of the biometric image for registration and the biometric image for matching can be matched with higher accuracy.

In each of the above examples, the distance is detected when the whiteout area is not detected in the biometric image, but the present invention is not limited to this. In the third embodiment, an example in which distance detection is performed even when a whiteout area is detected in a biometric image will be described.

FIG. 9 is a diagram illustrating a flowchart executed by the image processing apparatus 100. As illustrated in FIG. 9, the biometric image acquisition unit 40 acquires a biometric image (step S11). Next, the whiteout detection unit 50 determines whether the biometric image acquired in step S11 includes a whiteout area (step S12).

When it is determined to be “Yes” in step S12, the whiteout detection unit 50 stores the result of determination to be “Yes” in step S12 (step S13). When it is determined to be “No” in step S12 or after step S13 is executed, the distance detection unit 60 detects the distance between the palm and the imaging device 108 (step S14).

Next, the distance detection unit 60 determines whether or not the distance between the palm and the imaging device 108 has been detected (step S15). When it is determined to be “Yes” in step S15, the scale conversion unit 70 calculates the magnification according to the distance detected in step S14 (step S16). For example, the smaller the distance, the smaller the magnification, and the larger the distance, the larger the magnification.

When it is determined to be “No” in step S15, the scale conversion unit 70 determines whether or not the result determined to be “Yes” in step S12 is stored in the whiteout detection unit 50 (step S17). When it is determined to be “No” in step S17, the scale conversion unit 70 reads the magnification 1 from the table 55 (step S18). When it is determined to be “Yes” in step S17, the scale conversion unit 70 reads the magnification 2 from the table 55 (step S19). The scaling factor 2 is a value smaller than the scaling factor 1, for example.

After execution of step S16, after execution of step S18, or after execution of step S19, the scale conversion unit 70 corrects the scale of the biometric image by multiplying the scale of the biometric image by the scale factor (step S20). By this processing, the biometric image is reduced if the magnification is small and the biometric image is enlarged if the magnification is large. The biometric feature extraction unit 80 extracts the matching biometric feature from the scale-adjusted biometric image (step S21). Next, the authentication unit 90 compares the verification biometric feature with the registered biometric feature (step S22). For example, if the similarity between the matching biometric feature and the registered biometric feature is greater than or equal to the threshold, the authentication unit 90 authenticates the user who is the authenticated user as the user.

According to the present embodiment, the magnification for correcting the scale of the biometric image or the biometric feature can be changed according to whether or not the distance detection unit 60 can detect the distance and the detection result of the whiteout detection unit 50. .. With this configuration, it is possible to correct the scale even when the whiteout has not been detected but the distance detection has failed.

In each of the above examples, the whiteout detection unit 50 functions as an example of a saturated region detection unit that detects a saturated region of a pixel from a biometric image obtained by capturing an image of a living body that moves while touching a touch panel. The scale conversion unit 70 functions as an example of a correction unit that corrects the scale of the biometric image or the biometric feature extracted from the biometric image according to the detection result of the saturated region detection unit. The biometric image acquisition unit functions as an example of an extraction unit that extracts a palm region from the biometric image. Distance at which the distance detection unit 60 detects the distance between the living body image capturing device and the living body based on the moving distance of the living body on the touch panel and the moving distance of the living body in the living body image. It functions as an example of the detection unit.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments, and various modifications and alterations are possible within the scope of the gist of the present invention described in the claims. It can be changed.

DESCRIPTION OF SYMBOLS 10 Contact position detection part 20 Guide moving part 30 Moving amount calculation part 40 Biometric image acquisition part 50 Whiteout highlight detection part 55 Table 60 Distance detection part 70 Scale conversion part 80 Biometric feature extraction part 90 Authentication part 100 Image processing device 104 Display device 105 Input device 106 Biometric sensor 107 Illumination 108 Imaging device

Claims (9)

A saturated area detection unit that detects a saturated area of a pixel from a biological image obtained by capturing an image of a living body moving while touching a touch panel,
An image processing apparatus, comprising: a correction unit that corrects a scale of the biometric image or a biometric feature extracted from the biometric image according to a detection result of the saturated region detection unit. The image processing apparatus according to claim 1, wherein the correction unit reduces the biometric image or the biometric feature by using a fixed magnification when the saturation region is detected by the saturation region detection unit. The image processing apparatus according to claim 1, wherein the correction unit reduces the biometric image or the biometric feature by using a magnification according to the size of the saturated region. An extraction unit for extracting a palm region from the biometric image,
The image processing apparatus according to claim 1, wherein the saturated area detection unit detects the saturated area in the palm area. Based on the movement amount of the living body on the touch panel and the movement amount of the living body in the living body image, a distance detection unit that detects a distance between the imaging device that acquires the living body image and the living body is provided. The image processing apparatus according to any one of claims 1 to 4, characterized in that: The image according to claim 5, wherein the distance detecting unit detects a distance between the imaging device and the living body when the saturated region is not detected in the living body image by the saturated region detecting unit. Processing equipment. The correction unit changes the magnification when correcting the scale of the biometric image or the biometric feature according to whether or not the distance detection unit can detect the distance and the detection result of the saturated region detection unit. The image processing apparatus according to claim 5. The saturation detection unit detects a saturated region of pixels from a biometric image obtained by capturing an image of a living body moving while touching a touch panel,
An image processing method, wherein the correction unit corrects the scale of the biometric image or the biometric feature extracted from the biometric image according to the detection result by the saturated region detection unit. On the computer,
A process of detecting a saturated region of pixels from a biometric image obtained by capturing an image of a living body moving while touching a touch panel,
An image processing program for executing a process of correcting the scale of a biometric image or a biometric feature extracted from the biometric image according to a detection result of the saturated region.

Images