JP6840652B2 - Image processing equipment, image processing method and image processing program - Google Patents

Description

The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

For example, in a biometric authentication device, there is a technique for measuring the height of the palm with respect to the biometric authentication device when acquiring an image of the palm of a living body (see, for example, Patent Document 1). Further, in the biometric authentication device, there is a technique of determining the height of the palm with respect to the biometric authentication device and acquiring an image of the palm when the height is within a predetermined range (see, for example, Patent Document 2).

Japanese Unexamined Patent Publication No. 2015-170322 Japanese Unexamined Patent Publication No. 2015-026201

In the above-mentioned prior art, it is premised that the entire palm is imaged by the biometric authentication device. Then, the biometric authentication device identifies the outer shape of the palm by using both ends of the hand, fingers, wrists, etc. included in the captured image as clues, and performs a process of cutting out an image of the entire palm from the background in the captured image. The biometric authentication device performs various processing in the subsequent stage based on the image of the entire palm cut out from the captured image in this way.

Here, in recent years, the size of the image pickup device used in the biometric authentication device has been miniaturized, and the size of the biometric authentication device has been reduced. When the imaging target is the palm, the entire palm is imaged by photographing the palm from a certain distance or more with respect to the biometric authentication device. If the size of the image sensor used in the biometric authentication device is a certain size, the entire palm is imaged even if the distance of the palm to the biometric authentication device is short.

However, if the size of the image sensor used in the biometric authentication device is small, the palm is partially imaged when the distance of the palm to the biometric authentication device is short. That is, in the prior art, the captured image of the palm captured at a short distance, for example, the left and right ends of the palm, the fingers, the wrist, or an image that does not include all of these, is not the entire palm but the palm. Since only a part can be taken out, there is a problem that the accuracy when calculating the direction of the palm is lowered. If the accuracy of calculating the direction of the palm is reduced, inconvenience will occur in various processes in the subsequent stage.

The disclosed technology of the present application has been made in view of the above, and is, for example, an image processing device, an image processing method, and an image processing program that suppresses a decrease in accuracy when calculating the direction of a living body part from an captured image of the living body part. The purpose is to provide.

In one example of the disclosed technology, the image processing apparatus includes a determination unit, an estimation unit, an angle calculation image creation unit, and an angle calculation unit. The determination unit determines whether it is the whole or a part of the living body part from the captured image captured including the whole or a part of the living body part. When the determination unit determines that the captured image includes a part of the living body part, the estimation unit estimates which part of the whole living body part is the part of the living body part. The angle calculation image creation unit is an angle calculation image that emphasizes information that characterizes the angle of the living body part from the captured image in which a part of the living body part is estimated by the estimation part as which part of the whole living body part is. To create. The angle calculation unit generates a vector from the information that characterizes the angle in the angle calculation image created by the angle calculation image creation unit, and calculates the angle of the living body portion with respect to a predetermined reference direction based on the generated vector.

According to an example of the disclosed technique, it is possible to suppress a decrease in accuracy when calculating the direction of the living body part from the captured image of the living body part.

FIG. 1 is a perspective view showing an example of the appearance of the biometric authentication device according to the first embodiment. FIG. 2 is a cross-sectional view taken along the line II showing an example of the biometric authentication device according to the first embodiment. FIG. 3 is a plan view showing an example of a substrate included in the biometric authentication device according to the first embodiment. FIG. 4 is a functional block diagram showing an example of the biometric authentication device according to the first embodiment. FIG. 5 is a functional block diagram showing a detailed example of the normalization unit of the biometric authentication device according to the first embodiment. FIG. 6 is a flowchart showing an example of the normalization process according to the first embodiment. FIG. 7 is a diagram showing a successful example of the angle calculation process in the prior art. FIG. 8 is a diagram showing an example in which an erroneous angle is calculated in the prior art. FIG. 9 is a diagram showing an example in which the angle calculation fails in the prior art. FIG. 10 is a diagram showing an example of the left / right hand determination result. FIG. 11 is a diagram showing an example of the estimation result of the palm portion in the first embodiment. FIG. 12 is a diagram showing an example of the estimation result of the palm portion in the first embodiment. FIG. 13A is a diagram showing an example of a plurality of estimation results of the palm portion when it is specified whether the palm is the left or right hand in the first embodiment. FIG. 13B is a diagram showing an example of a plurality of estimation results of the palm portion when it is specified whether the palm is the left or right hand in the first embodiment. FIG. 13C is a diagram showing an example of a plurality of estimation results of the palm portion when it is specified whether the palm is the left or right hand in the first embodiment. FIG. 14 is a diagram showing an example of an angle calculation image in which necessary information is emphasized after noise removal with respect to the captured image of the palm portion in the first embodiment. FIG. 15 is a diagram showing an example of area division of the angle calculation image of the palm portion in the first embodiment. FIG. 16 is a diagram showing an example of a directionality calculation result in which the directionality of each divided area is calculated based on necessary information with respect to the angle calculation image obtained by dividing the palm portion into an area in the first embodiment. FIG. 17 is a diagram showing an example of a palm model in the first embodiment. FIG. 18 is a diagram showing an example of estimation results of a plurality of palm partial candidates in the first embodiment. FIG. 19 is a diagram showing a successful example of the angle calculation process in the first embodiment. FIG. 20 is a functional block diagram showing a detailed example of the normalization unit of the biometric authentication device according to the second embodiment. FIG. 21 is a flowchart showing an example of the normalization process according to the second embodiment.

Hereinafter, examples of the image processing apparatus, the image processing method, and the image processing program according to the disclosed technology of the present application will be described in detail with reference to the drawings. The disclosure techniques of the present application are not limited by the following examples. In the following embodiment, a biometric authentication device using a palm vein will be described as an example in biometric authentication, including an image processing device that processes an image of the palm acquired by holding the palm of the hand over a biosensor with the part of the living body as the palm. To do. However, the disclosure technique is not limited to this, and is also applicable to image processing of a biological image obtained by holding other parts of the living body such as a finger, eyes, and face over a biological sensor in a non-contact manner.

The biometric authentication device registers the characteristic data of the palm vein extracted from the captured image of the biological part (for example, the palm) in advance as "registration data" in the database. Then, the biometric authentication device captures the living body part when logging in to the computer system, entering / exiting the door, etc., and the characteristic data of the palm vein extracted from the captured image is used as "authentication data" as "registration data". By collating with, the living body is authenticated.

In the following description of the examples and modifications, the same names or the same reference numerals are given to the existing components and processes, and the description thereof will be omitted. In addition, each of the following examples and each modification can be carried out in combination in part or in whole within a consistent range.

(Biometric authentication device according to Example 1)
FIG. 1 is a perspective view showing an example of the appearance of the biometric authentication device according to the first embodiment. As shown in FIG. 1, the biometric authentication device 1 has an imaging unit 1b housed in a housing 1a-1 and is covered with a cover 1a-2 having transparency to infrared light. The biometric authentication device 1 captures an image of the palm held over the cover 1a-2.

FIG. 2 is a cross-sectional view taken along the line II showing an example of the biometric authentication device according to the first embodiment. As shown in FIG. 2, the image pickup unit 1b of the biometric authentication device 1 has an image pickup element 200, a light emitting element 300, a lens unit 3, and a light guide body 4 on a substrate 2, a microcomputer and a light guide body 4 on the control substrate 5. It has a processing device 100 including a memory and the like. The light emitted by the light emitting element 300 passes through the ring-shaped light guide body 4 and is emitted upward from the cover 1a-2 as illumination light. This light illuminates the palm held over the cover 1a-2 and is reflected or absorbed by the palm. The image sensor 200 passes through the lens of the lens unit 3 and images the palm held over the cover 1a-2.

FIG. 3 is a plan view showing an example of a substrate included in the biometric authentication device according to the first embodiment. As shown in FIG. 3, the image sensor 200 is mounted in the center of the substrate 2. The lens unit 3 is arranged on the substrate 2 along a substantially circle around the image sensor 200. Further, a plurality of light emitting elements 300 are arranged on the substrate 2 along a substantially circle around the lens unit 3. The plurality of light emitting elements 300 are divided into two types of light emitting element groups having different wavelengths, but they are not distinguished in the first embodiment. Further, a light receiving element that receives light emitted by a plurality of light emitting elements 300 is arranged on the substrate 2, but the illustration is omitted.

Further, four distance measuring light emitting elements 400 for measuring the distance between the biometric authentication device 1 and the palm are provided at the four corners of the substrate 2. As shown in FIG. 3, the four distance measuring light emitting elements 400 are arranged on the diagonal line of the substrate 2, and the distance between the distance measuring light emitting elements 400 is arranged on the diagonal line farthest from the distance. From the distance measured by the four distance measuring light emitting elements 400, the distance between the biometric authentication device 1 and the object and the inclination of the object are detected. The number of distance measuring light emitting elements 400 is not limited to four.

It should be noted that FIGS. 1 to 3 are only an example of the biometric authentication device 1 according to the first embodiment, and may be another hardware form capable of realizing the same function.

(Functional configuration of biometric authentication device according to Example 1)
FIG. 4 is a functional block diagram showing an example of the biometric authentication device according to the first embodiment. As shown in FIG. 4, the biometric authentication device 1 includes an image pickup unit 110, a normalization unit 120, a feature extraction unit 140, an image pickup element 200, a light emitting element 300, a distance measurement light emitting element 400, an image pickup DB (Data Base) 500, and the like. It has a candidate DB 600. Note that FIG. 4 shows only the main configuration of the biometric authentication device 1.

The image sensor 200 is, for example, CMOS (Complementary Metal Oxide) for acquiring an image of the palm of the user based on the near-infrared light reflected by the palm of the user from the near-infrared light emitted by the light emitting element 300. Semiconductor) Image sensor, etc. The light emitting element 300 is, for example, an LED (Light Emitting Diode) for irradiating the palm of the user with near-infrared light. The distance measuring light emitting element 400 is, for example, an LED for irradiating the palm with spot light (beam light) in order to measure the distance between the biometric authentication device 1 and the palm held over the biometric authentication device 1. ..

The image pickup DB 500 is a semiconductor storage device or the like that stores an image of the palm imaged by the image pickup unit 110 via the image pickup element 200. The candidate DB 600 is a semiconductor storage device that stores one or a plurality of estimation results that indicate which part of the palm part the captured image of the palm is, which is estimated by the normalization unit 120.

The imaging unit 110, the normalization unit 120, and the feature extraction unit 140 are processing devices such as an MPU (Micro Processing Unit). The image pickup unit 110 controls the image pickup element 200 and the light emitting element 300 to acquire an image of the palm of a living body held above the biometric authentication device 1. The imaging unit 110 stores the acquired captured image of the palm in the imaging DB 500. Further, the imaging unit 110 calculates the distance between the biometric authentication device 1 and the palm held over the biometric authentication device 1 based on the reflected light of the spot light emitted from the distance measuring light emitting element 400 to the palm.

The normalization unit 120 performs a normalization process for correcting the rotation of the captured image of the palm acquired by the image pickup unit 110 after removing the background and noise. Details of the normalization unit 120 will be described later with reference to FIG. Note that rotation means that when the palm is imaged by the imaging unit 110, the palm is actually held over the biometric authentication device 1 with respect to the three-dimensional reference direction in which the palm should be held over the biometric authentication device 1. It is a deviation in direction and is represented by an angle.

The feature extraction unit 140 extracts palm vein information from the captured image of the palm normalized by the normalization unit 120. The registration unit 150 stores the palm vein information extracted by the feature extraction unit 140 and incidental information such as the registration environment of the palm vein information in a database (not shown). The collation unit 160 collates the palm vein information extracted by the feature extraction unit 140 with the palm vein information stored in the database by the registration unit 150.

(Details of the Normalization Unit of the Biometric Authentication Device According to Example 1)
FIG. 5 is a functional block diagram showing a detailed example of the normalization unit of the biometric authentication device according to the first embodiment. As shown in FIG. 5, the normalization unit 120 of the biometric authentication device 1 includes an overall determination unit 121, a left / right determination unit 122, a left / right designation unit 123, an overall hand estimation unit 124, a palm portion estimation unit 125, and a certainty calculation unit 126. , Angle calculation image creation unit 127, auxiliary information extraction unit 128, auxiliary information certainty calculation unit 129, angle calculation unit 1_130, certainty recalculation unit 131, first place confirmation unit 132, palm extraction unit 133, angle calculation unit 2_134, It has a rotating portion 135.

The normalization unit 120 is an example of an image processing device. Further, the palm extraction unit 133, the angle calculation unit 2_134, and the rotation unit 135 have the same configuration as the normalization unit according to the prior art.

The overall determination unit 121 generates an image in which the background and noise are removed from the captured image of the hand of the living body captured by the imaging unit 110, and determines whether or not this image is the entire palm image. The entire palm image is an image that includes the entire palm. The palm partial image is an image in which the palm is partially included.

The left-right determination unit 122 determines which of the left and right hands the palm is from the outer shell of the palm included in the palm partial image determined by the overall determination unit 121. For example, in the left / right determination unit 122, the palm of the palm is either the palm of the right hand or the palm of the left hand based on the shape between the thumb and the index finger of the palm included in the palm partial image determined by the overall determination unit 121. Determine if there is. For example, the left / right determination unit 122 determines that the shape between the thumb and index finger of the palm is the palm of the left hand if it is L-shaped, and that it is the palm of the right hand if it is mirror-written in the L shape. judge.

The left / right designation unit 123 is based on user input, log information of the biometric authentication device 1, or the like when it is not possible for the left / right determination unit 122 to determine whether the palm included in the palm partial image is the left or right hand. Specify which hand is left or right. When the left / right determination unit 122 can determine which of the left and right hands the palm included in the palm partial image is, the imaging area of the palm included in the palm partial image is at least 60 of the entire palm, for example. This is the case of ~ 70 [%]. Further, when it is not possible for the left / right determination unit 122 to determine which of the left and right palms is included in the palm partial image, the imaging area of the palm included in the palm partial image is the entire palm. For example, it is less than 30 [%].

The whole hand estimation unit 124 generates a palm skin image from the palm partial image designated as either the left or right hand by the left / right designation unit 123, and from the generated palm skin image and the palm model (see FIG. 17), It is estimated which part of the entire palm the palm part image is (see FIG. 11).

The palm model is an algorithm showing a model of the entire palm for estimating which part of the entire palm the palm partial image is based on the analysis result of the model image of the palm. The palm model may be a model for each main information (strong information) and each auxiliary information (weak information) described later, or a model including both main information (strong information) and auxiliary information (weak information). May be good.

The palm partial estimation unit 125 generates a palm epidermis image considering the palm outside the imaging range from the palm partial image, and estimates one or more palm partial candidates from the generated palm epidermis image and the palm model (Fig.). 11, FIG. 12, FIGS. 13A to 13C). The palm portion estimation unit 125 estimates one palm portion when it is determined that the imaged area of the palm included in the palm portion image is, for example, 60 to 90 [%] with respect to the total area of the palm (FIG. FIG. 11. See FIG. 12). On the other hand, the palm partial estimation unit 125 estimates a plurality of palm partial candidates when it is determined that the imaged area of the palm included in the palm partial image is, for example, 50 [%] or less with respect to the total area of the palm. (See FIGS. 13A to 13C, FIG. 17). The palm partial candidate is an image including information indicating which part of the entire palm the palm partial image is.

Then, the palm partial estimation unit 125 performs a predetermined sort on the estimated palm partial candidates, orders the palm partial candidates, assigns an index n according to the order, and stores the estimated palm partial candidates in the candidate DB 600. However, n = 1, ..., N (N is the estimated total number of palm partial candidates). If there is only one estimated palm portion candidate, the palm portion estimation unit 125 assigns an index 1 to this one palm portion candidate and stores it in the candidate DB 600.

Hereinafter, the certainty calculation unit 126, the angle calculation image creation unit 127, the auxiliary information extraction unit 128, the auxiliary information certainty calculation unit 129, the angle calculation unit 1_130, and the certainty recalculation unit 131 are estimated by the palm part estimation unit 125. When there are a plurality of (n) palm partial candidates, the series of steps S108 to S113 (see FIG. 6) performed for each of the plurality of palm partial candidates is repeated until all the palm partial candidates are completed. Do. However, n = 1, ..., N (N is the estimated total number of palm partial candidates).

Confidence calculator 126 calculates based estimated by palm portion estimating unit 125 one or more of palm candidates respectively confidence k _n in the following equation (1). Confidence k _n represents the likelihood of the palm portion indicated by palm candidate. In the following equation (1), n corresponds to the nth process of selecting one palm partial candidate from the unselected palm partial candidates in step S108 (see FIG. 6) described later. When n is a limit value, for example, 6 or more, it is calculated based on the following equation (2). However, if n exceeds the upper limit value, for example, 10, the determination cannot be made and the process is interrupted.

(Number 1)
_{k n = 50-n × 10} [%] ··· (1)
(Number 2)
_{k n = 50-50 / n [%} ] ··· (2)

When the left / right determination unit 122 or the left / right designation unit 123 specifies which hand the palm portion image is, for example, the certainty degree calculation unit 126 has a certainty degree k ₁ = 80 [%]. And. Further, the confidence degree calculation unit 126 sets, for example, the confidence degree k ₁ = 50 [%] when there is one palm portion candidate estimated by the palm portion estimation unit 125.

The angle calculation image creation unit 127 generates a composite image including the palm portion outside the image range from the palm partial image or the palm partial candidate and the palm model, and removes noise by smoothing or the like in the composite image to obtain necessary information. An emphasized angle calculation image is created (see FIG. 14). Necessary information is epidermis information that characterizes the direction (angle) of the palm, such as the epidermis of the palm and wrinkles of moving parts, and is strong information (hereinafter referred to as "main information") and weak information (hereinafter, "auxiliary information"). ”) Includes strength information. For example, the main information is epidermis information in which the edge strength of wrinkles in the epidermis is equal to or higher than a predetermined value, and the auxiliary information is epidermis information in which the edge strength of wrinkles in the epidermis is less than a predetermined value. The auxiliary information is more detailed epidermis information than the epidermis information included in the normal image of the palm imaged at a height equal to or higher than the normal predetermined height for the biometric authentication device 1.

The auxiliary information extraction unit 128 extracts auxiliary information from the angle calculation image created by the angle calculation image creation unit 127. The auxiliary information certainty calculation unit 129 calculates the certainty of the auxiliary information (auxiliary information certainty θ) extracted by the auxiliary information extraction unit 128 by a predetermined algorithm. The auxiliary information certainty degree θ represents the certainty of the palm portion indicated by the palm portion candidate based on the auxiliary information.

The angle calculation unit 1_130 divides the angle calculation image created by the angle calculation image creation unit 127 into a plurality of areas (divided areas), and extracts edges of main information and auxiliary information from each area. The area is, for example, a grid of m × n (m and n are positive integers of 2 or more). However, it is not limited to the grid, and may be a mesh, a hex, or the like. Then, the angle calculation unit 1_130 vectorizes the edges of the main information and the auxiliary information for each extracted area (see FIGS. 15 and 16). In edge vectorization, for example, a vector is generated in which the direction of the edge is the direction of the vector. Then, the angle calculation unit 1_130 calculates the inner product of the edge vector of the main information and the vector included in the predetermined model for each area, and the inner product of the edge vector of the auxiliary information and the vector included in the predetermined model. Is calculated for each area.

The predetermined model is a pre-prepared model in which the edges of the skin information of the entire palm as a model are vectorized. Hereinafter, the inner product of the edge vector of the main information of each area and the vector included in the predetermined model is referred to as "main inner product". Further, the inner product of the edge vector of the auxiliary information of each area and the vector included in the predetermined model is called "auxiliary inner product".

That is, the angle calculation unit 1_130 selects a partial region of a predetermined model to which the angle calculation image is applied, calculates the main inner product in area units, and calculates the auxiliary inner product in area units for each partial region. Then, the angle calculation unit 1_130 sets a partial region of a predetermined model in which the main inner product and the auxiliary inner product of all areas are substantially the same value within a predetermined error as the palm portion indicated by the angle calculation image, and the main of each area at that time. The information vector is the main vector in each area, and the auxiliary information vector is the auxiliary vector in each area. In addition, "the inner product of all areas has substantially the same value within a predetermined error" means that the directions of the vectors are substantially aligned within a predetermined range.

On the other hand, when there is an area where the main inner product or the auxiliary inner product does not have substantially the same value within a predetermined error, the angle calculation unit 1_130 assists after removing the edge of the auxiliary information that becomes noise from the angle calculation image by noise removal processing. Vectorize the edges of the information again. Then, the angle calculation unit 1_130 reselects the partial area of the predetermined model to which the angle calculation image is applied, recalculates each of the main inner product and the auxiliary inner product for each area, and recalculates the main vector and the auxiliary vector in each area.

The angle calculation unit 1_130 repeats the above process until a partial region of a predetermined model in which both the main inner product and the auxiliary inner product have substantially the same value within a predetermined error is selected for all areas. Then, the angle calculation unit 1_130 estimates the angle of the palm partial candidate from the main inner product when both the main inner product and the auxiliary inner product have substantially the same value within a predetermined error for all areas, and based on the auxiliary inner product. Correct the estimated angle of the palm part candidate. For example, the direction of the vector obtained by adding the main vectors in each area when both the main inner product and the auxiliary inner product have substantially the same value within a predetermined error for all areas is the estimated angle of the palm portion candidate. Further, for example, the direction of the vector obtained by adding the auxiliary vectors in each area is the correction information for correcting the estimated angle based on the palm portion candidate.

Confidence recalculation unit 131 is estimated from the confidence k _n calculated by the confidence factor computing unit 126, the auxiliary information confidence θ and calculated by the auxiliary information confidence factor computing unit 129, the palm portion estimating unit 125 a plurality of the palm portion candidates each modified confidence k _{n 'is} calculated according to the following equation (3). Fixed confidence k _{n 'is} based on a confidence k _n and auxiliary information confidence theta, indicating the likelihood of palm indicated palm candidate. In the following equation (3), n corresponds to the nth process of selecting one palm partial candidate from the unselected palm partial candidates in step S108 (see FIG. 6) described later.

(Number 3)
k _n '= k _n × θ ... (3)

As described above, it is estimated by the palm part estimation unit 125 by the certainty calculation unit 126, the angle calculation image creation unit 127, the auxiliary information extraction unit 128, the auxiliary information certainty calculation unit 129, the angle calculation unit 1_130, and the certainty recalculation unit 131. When the series of steps S108 to S113 (see FIG. 6) performed for each of the N palm partial candidates is completed for all the palm partial candidates, the 1st place determination unit 132 regains the certainty. calculator 131 fix confidence k _n calculated by _'the finalizing the 1-position of the palm portion candidates is largest.

Note that one of the decision unit 132 _'is not limited to finalizing the 1-position of the palm candidate is maximum, modified confidence k _n' confidence recalculation unit 131 fixes confidence k _n calculated by the A predetermined number of high-ranking palm partial candidates may be determined.

The palm extraction unit 133 removes the background and noise from the entire image of the palm determined by the overall determination unit 121, and cuts out the palm. The angle calculation unit 2_134 calculates the direction and inclination of the palm in the palm cut out by the palm extraction unit 133. The palm extraction unit 133 and the angle calculation unit 2_134 perform processing based on the existing technology.

The rotating unit 135 rotates the palm portion image in which the first-ranked palm portion candidate is determined by the first-position determining portion 132 according to the estimated angle and correction information based on the first-ranked palm portion candidate, thereby causing the palm vein image. Generate the original image to create. Further, the rotating unit 135 creates an original image for creating a palm vein image by rotating the palm image whose palm direction and inclination are calculated by the angle calculation unit 2_134 according to the direction and inclination of the palm. Generate.

The rotating unit 135 may output at least one of the estimated angle, correction information, and correction certainty of the entire palm image and the partial palm image together with the original image for creating the palm vein image.

(Normalization processing according to Example 1)
FIG. 6 is a flowchart showing an example of the normalization process according to the first embodiment. The normalization process according to the first embodiment is executed by the normalization unit 120 each time the palm of the living body is imaged by the image pickup unit 110 and input to the normalization unit 120. As shown in FIG. 6, first, in step S101, the overall determination unit 121 receives the input of the palm image of the living body imaged by the imaging unit 110.

Next, in step S102, the overall determination unit 121 performs an overall determination process on the palm image that received the input in step S101. Specifically, the overall determination unit 121 determines whether or not the entire palm image received the input in step S101 includes the entire palm.

When the palm image received the input in step S101 includes the entire palm (step S103Yes), the overall determination unit 121 shifts the process to step S123. Hereinafter, the palm image including the entire palm is referred to as the entire palm image. On the other hand, when the palm image that received the input in step S101 partially includes the palm (step S103 No.), the overall determination unit 121 shifts the process to step S104. Hereinafter, the palm image in which the palm is partially included is referred to as a palm partial image.

In step S104, the left-right determination unit 122 determines whether the palm included in the palm partial image determined by the overall determination unit 121 in step S103 is the left hand. When the left / right determination unit 122 determines that the palm included in the palm partial image is the left hand (step S104Yes), the left / right determination unit 122 shifts the process to step S116. On the other hand, when the left / right determination unit 122 cannot determine that the palm included in the palm partial image is the left hand (step S104No), the left / right determination unit 122 shifts the process to step S105.

In step S105, the left-right determination unit 122 determines whether the palm included in the palm partial image determined by the overall determination unit 121 in step S103 is the right hand. When the left / right determination unit 122 determines that the palm included in the palm partial image is the right hand (step S105Yes), the left / right determination unit 122 shifts the process to step S117. On the other hand, when the left / right determination unit 122 cannot determine that the palm included in the palm partial image is the right hand (step S105No), the left / right determination unit 122 shifts the process to step S106.

In step S106, the palm portion estimation unit 125 determines which part of the entire palm is included in the palm portion image in which the left / right determination unit 122 cannot determine which of the left and right hands is. Estimate one or more palm partial candidates shown. In step S107, the palm portion estimation unit 125 determines whether or not there are a plurality of palm portion candidates estimated in step S106. When the palm portion estimation unit 125 has a plurality of palm portion candidates estimated in step S106 (step S107Yes), the palm portion estimation unit 125 shifts the process to step S108. On the other hand, when the palm portion estimation unit 125 has one palm portion candidate estimated in step S106 (step S107 No), the process shifts to step S119.

Next, the certainty calculation unit 126, the angle calculation image creation unit 127, the auxiliary information extraction unit 128, the auxiliary information certainty calculation unit 129, the angle calculation unit 1_130, and the certainty recalculation unit 131 are in the palm part estimation unit in step S106. For all of the plurality of palm portion candidates estimated by 125, the processes of steps S108 to S113 are performed for each palm portion candidate.

In step S108, the certainty calculation unit 126 selects one unselected palm portion candidate from the plurality of palm portion candidates estimated by the palm portion estimation unit 125, and sets n = 1 in the above equation (1). Place, calculates the certainty factor k ₁ of palm candidates selected. Next, in step S109, the angle calculation image creation unit 127 creates an angle calculation image based on the palm portion candidate selected in step S108.

Next, in step S110, the auxiliary information extraction unit 128 extracts auxiliary information from the angle calculation image created by the angle calculation image creation unit 127 in step S109. Next, in step S111, the auxiliary information certainty calculation unit 129 calculates the certainty of the auxiliary information (auxiliary information certainty θ) extracted by the auxiliary information extraction unit 128 in step S110.

Next, in step S112, the angle calculation unit 1_130 extracts and vectorizes the angle from the angle calculation image created by the angle calculation image creation unit 127 in step S109, based on the edges of the main information and the auxiliary information. Calculate the angle of the image. Next, in step S113, confidence recalculation unit 131, a confidence _{k 1} calculated by the confidence factor computing unit 126 in step S108, the calculated auxiliary information convinced by the auxiliary information confidence factor computing unit 129 at step S111 and a degree theta, a modified confidence k _{1 'of} the selected palm candidate in step S108 is calculated based on the above equation (3).

Next, returning to step S108, the certainty calculation unit 126 selects one unselected palm portion candidate from the plurality of palm portion candidates estimated by the palm portion estimation unit 125, and in the above equation (1). n = 2 Distant, calculates the confidence factor k ₂ of palm candidates selected. Hereinafter, the angle calculation image creation unit 127, the auxiliary information extraction unit 128, the auxiliary information certainty calculation unit 129, the angle calculation unit 1_130, and the certainty recalculation unit 131 have selected the processes of steps S109 to S113 in step S108. This is done for palm part candidates.

Then, the certainty calculation unit 126, the angle calculation image creation unit 127, the auxiliary information extraction unit 128, the auxiliary information certainty calculation unit 129, the angle calculation unit 1_130, and the certainty recalculation unit 131 are estimated by the palm part estimation unit 125. All of the palm partial candidates not selected in step S108 among the plurality of palm partial candidates are set to n = 3, ..., N, and the processes of steps S108 to S113 are performed. When the process of step S113 for the palm portion candidate last selected in step S108 is completed, the normalization unit 120 shifts the process to step S114.

_{In step S114, the 1st place determination unit 132 has a modified certainty degree k n} '(n = 1, ...) Calculated by the certainty degree recalculation unit 131 among the N palm part candidates estimated in step S106. , N) is the largest, and the 1st place palm part candidate is confirmed. Next, in step S115, the rotating unit 135 uses the palm portion candidate determined by the 1st place determination unit 132 in step S114 or the palm image processed for angle calculation in step S122 or step S124 according to the estimated angle. By rotating and correcting the angle, an original image for creating a palm vein image is generated. When step S115 is completed, the normalization unit 120 ends the normalization process according to the first embodiment.

On the other hand, in step S116, the entire hand estimation unit 124 generates a palm skin image from the palm partial image determined to be the left hand by the left and right designation units 123 in step S104, and compares the generated palm skin image with the palm model. Then, it is estimated which part of the entire left palm part of the palm part image is. When step S116 is completed, the normalization unit 120 shifts the process to step S118.

Further, in step S117, the entire hand estimation unit 124 generates a palm skin image from the palm partial image determined to be the right hand by the left and right designation units 123 in step S105, and from the generated palm skin image and the palm model, Estimate which part of the entire right palm the palm part image is. When step S117 is completed, the normalization unit 120 shifts the process to step S118.

In step S118, the certainty calculation unit 126 sets the certainty of the palm partial image estimated in step S116 or step S117 to be, for example, 80 [%] of the entire left and right palms. When step S118 is completed, the normalization unit 120 shifts the process to step S120.

Further, in step S119, the certainty calculation unit 126 sets the certainty of the only palm partial candidate estimated in step S107 to, for example, 50 [%]. When step S119 is completed, the normalization unit 120 shifts the process to step S120.

In step S120, the angle calculation image creation unit 127 creates an angle calculation image based on the palm partial image or the palm partial candidate for which the certainty is set in step S118 or step S119. Next, in step S121, the auxiliary information extraction unit 128 extracts auxiliary information from the angle calculation image created by the angle calculation image creation unit 127 in step S120. Next, in step S122, the angle calculation unit 1_130 extracts and vectorizes the angle from the angle calculation image created by the angle calculation image creation unit 127 in step S120, based on the edges of the main information and the auxiliary information. Calculate the angle of the image. When step S122 is completed, the normalization unit 120 shifts the process to step S115.

On the other hand, in step S123, the palm extraction unit 133 performs a palm extraction process for removing the background and noise from the entire palm image determined in step S103. Next, in step S124, the angle calculation unit 2_134 calculates the angle and inclination of the palm based on the entire palm image processed by the palm extraction unit 133 in step S124. When step S124 is completed, the normalization unit 120 shifts the process to step S115.

The degree of certainty = 50 [%] in step S119 is lower than the degree of certainty = 80 [%] in step S118 because the palm part candidate in step S119 is compared with the palm part image in step S118. This is because it has not been determined which hand is left or right. Also, confidence k _n at the step S108 is confidence = 50 is the lower confidence than [%] is in step S119, palm candidates is for one to several present instead.

(Successful example of angle calculation processing in the prior art)
FIG. 7 is a diagram showing a successful example of the angle calculation process in the prior art. The overall determination unit 121 can determine that (a-1) in FIG. 7 is the entire image of the palm of the left hand. The palm extraction unit 133 cuts out the palm from the entire image of the left palm in FIG. 7 (a-1) as shown by the enclosed broken line in FIG. 7 (a-2). Then, the angle calculation unit 2_134 calculates the direction (angle) of the left palm cut out by the palm extraction unit 133, as shown by the arrow in FIG. 7 (a-3). Similarly, the overall determination unit 121 can determine that (b-1) in FIG. 7 is the entire image of the palm of the right hand. The palm extraction unit 133 cuts out the palm of the right hand from the entire image of the palm of the right hand in FIG. 7 (b-1) as shown by the enclosed broken line in FIG. 7 (b-2). Then, the angle calculation unit 2_134 calculates the direction (angle) of the right palm cut out by the palm extraction unit 133, as shown by the solid arrow in FIG. 7 (b-3). As described above, in the prior art, when the palm is cut out from the entire image of the palm, the angle of the palm can be calculated correctly.

(Example of calculating an incorrect angle in the prior art)
FIG. 8 is a diagram showing an example in which an erroneous angle is calculated in the prior art. The palm extraction portion 133 can only partially cut out the palm of the right hand from the image of the palm part of the right hand shown in FIG. 8 (a-1), as shown by the enclosed broken line in FIG. 8 (a-2). Can not. However, the angle calculation unit 2_134 determines the direction (angle) of the left palm as shown by the solid arrow in FIG. 8 (b-3) without recognizing that the cut out right palm is partial. calculate. Therefore, the solid line arrow in FIG. 8 (a-3) deviates from the correct direction (angle) of the left palm indicated by the broken line arrow.

Similarly, from the right palm partial image shown in FIG. 8 (b-1), the palm extraction portion 133 may partially remove the right palm as shown by the enclosed broken line in FIG. 8 (b-2). You can only cut it out. However, the angle calculation unit 2_134 determines the direction (angle) of the right palm as shown by the solid arrow in FIG. 8 (b-3) without recognizing that the cut out right palm is partial. calculate. Therefore, the solid line arrow in FIG. 8 (b-3) deviates from the correct direction (angle) of the palm of the right hand indicated by the broken line arrow. As described above, in the prior art, when the palm is cut out from the palm partial image, the angle of the palm cannot be calculated correctly.

(Example of angle calculation failure in the prior art)
FIG. 9 is a diagram showing an example in which the angle calculation fails in the prior art. In the examples shown in FIGS. 9A to 9D, the imaged area of the palm included in the palm partial image is, for example, less than 50 [%] with respect to the total area of the palm. Further, in the palm partial image shown in FIGS. 9 (a) to 9 (c), there is a portion that can be presumed to be between any finger, but in FIG. 9 (c), that portion is present. Is located in the center of the palm partial image, not on the left or right, so it cannot be determined whether the palm partial image is the left or right hand.

Further, since the palm partial image shown in FIG. 9D does not have a portion that can be presumed to be between any of the fingers, it is determined whether the palm partial image is the left or right hand. Can not do it. Therefore, in the palm extraction unit 133, it is unclear which part of the entire left or right palm the palm included in the palm partial images shown in FIGS. 9 (a) to 9 (d) corresponds to, and the palm The palm cannot be extracted from the partial image. Therefore, in the examples shown in FIGS. 9A to 9D, the palm angle cannot be calculated.

(Left and right hand judgment)
FIG. 10 is a diagram showing an example of the left / right hand determination result. In the example shown in FIG. 10A, the L-shaped portion (solid arrow portion in FIG. 10A) formed by the thumb and index finger of the right hand in the palm partial image is to the right of the palm partial image. Since it is located at, it can be determined that the palm partial image is the captured image of the right hand by the left and right hand determination. Further, in the example shown in FIG. 10 (b), the L-shaped mirror writing portion (the solid line arrow portion in FIG. 10 (b)) formed by the thumb and index finger of the left hand in the palm partial image is the palm. Since it is located to the left of the partial image, it can be determined that the palm partial image is the captured image of the left hand by the left and right hand determination.

Further, in the example shown in FIG. 10 (c), the L-shaped portion (solid arrow portion in FIG. 10 (c)) formed by the thumb and index finger of the left hand in the palm partial image is left of the palm partial image. Since it is located in the direction, it can be determined that the palm partial image is the captured image of the left hand by the left and right hand determination. Further, in the example shown in FIG. 10 (d), the L-shaped portion (the solid line arrow portion in FIG. 10 (d)) formed by the thumb and index finger of the right hand in the palm partial image is the palm partial image. Since it is located on the right side, it can be determined that the palm partial image is the captured image of the right hand by the left and right hand determination.

However, in the example shown in FIG. 10 (e), although the palm partial image includes an L-shaped or L-shaped mirror-shaped portion formed by the thumb and index finger of the hand, that portion is assumed. Since it is not located at the location where the image is formed, it is not possible to determine which of the left and right hands is the captured image of the palm portion by the left and right hand determination. Further, in the example shown in FIG. 10 (f), since the palm partial image does not include the L-shaped or L-shaped mirror writing portion formed by the thumb and index finger of the hand, the left and right hand determination can be performed. Therefore, it is not possible to determine whether the palm partial image is the captured image of the left or right hand.

(Estimation result of palm portion in Example 1)
11 and 12 are diagrams showing an example of the estimation result of the palm portion in the first embodiment. FIG. 11 shows that the palm portion estimation unit 125 estimates only one palm portion when the imaged area of the palm included in the palm portion image is, for example, 60 to 90 [%] with respect to the total area of the palm. The estimation result is shown. For example, in FIG. 11A, the palm portion estimation unit 125 takes the palm portion image shown in FIG. 8 (a-1) as an input, and indicates which part of the entire palm the palm portion image is. This is the result of estimating the entire outer shell (the broken line portion in (a) of FIG. 11). Further, for example, in FIG. 11B, the palm portion estimation unit 125 receives the palm portion image shown in FIG. 8 (b-1) as an input, and determines which part of the entire palm the palm portion image is. This is the result of estimating the outer shell of the entire palm shown (the broken line portion in (b) of FIG. 11).

FIG. 12 shows a plurality of palm portions estimated by the palm portion estimation unit 125 when the imaged area of the palm included in the palm portion image is, for example, 50 [%] or less with respect to the total area of the palm. For example, in (a-1) and (a-2) of FIG. 12, the palm part estimation unit 125 takes the palm part image shown in FIG. 9 (a) as an input, and the palm part image is any part of the entire palm. This is the result of estimating two ways of the outer shell of the entire palm (the broken line portion in (a-1) and (a-2) of FIG. 12). Further, for example, in (b-1) and (b-2) of FIG. 12, the palm portion estimation unit 125 inputs the palm portion image shown in FIG. 9 (b), and the palm portion image is any of the entire palm. This is the result of estimating two ways of the outer shell of the entire palm (broken line portion in (b-1) and (b-2) of FIG. 12) indicating whether or not the portion is.

13A to 13C are diagrams showing an example of a plurality of estimation results of the palm portion when it is specified whether the palm is the left or right hand in the first embodiment. For example, in FIG. 13A (a-1), when the “right hand” is designated by the left / right designation unit 123 with respect to the palm partial image shown in FIG. 9 (c), the entire hand estimation unit 124 determines the palm. This is the result of estimating the part of the palm part image in the whole. Further, for example, in FIG. 13A (a-1), when the “right hand” is designated by the left / right designation unit 123 with respect to the palm portion image shown in FIG. 9 (c), the palm portion estimation unit 125 This is the result of estimating the outer shell of the entire palm (the broken line portion in (a-2) of FIG. 13A) indicating which part of the entire palm the palm partial image is, using the palm partial image as an input.

Further, for example, (b-1-1) in FIG. 13B and (b-2-1) in FIG. 13C are “right hand” by the left and right designation portions 123 with respect to the palm partial image shown in FIG. 9 (d). When specified, the whole hand estimation unit 124 is the result of estimating two types of palm partial images in the entire palm. Further, for example, (b-1-2) in FIG. 13B and (b-2-2) in FIG. 13C are "right hand" by the left and right designated portions 123 with respect to the palm partial image shown in FIG. 9 (d). When specified, the palm partial estimation unit 125 takes the palm partial image as an input, and indicates which part of the entire palm the palm partial image is, the outer shell of the entire palm ((b-1-2) in FIG. 13B). ) Is one of the two estimation results.

(Angle calculation image based on the captured image of the palm portion in the example)
FIG. 14 is a diagram showing an example of an angle calculation image in which necessary information is emphasized after noise removal with respect to the captured image of the palm portion in the first embodiment. For example, in FIG. 14, the angle calculation image creation unit 127 generates a composite image including the palm portion outside the image range from the palm portion image (or palm portion candidate) shown in FIG. 9D and the palm model. , This is an angle calculation image in which noise is removed by smoothing or the like in a composite image to emphasize necessary information.

(Area division of the emphasized image of the palm portion in Example 1)
FIG. 15 is a diagram showing an example of area division of the angle calculation image of the palm portion in the first embodiment. For example, FIG. 15 shows an example in which the angle calculation unit 1_130 divides the angle calculation image created by the angle calculation image creation unit 127 into a 4 × 4 grid.

(Direction of each divided area of the area divided image of the palm part in Example 1)
FIG. 16 is a diagram showing an example of a directionality calculation result in which the directionality of each divided area is calculated based on necessary information with respect to the angle calculation image obtained by dividing the palm portion into an area in the first embodiment. In FIG. 16A, only the main information is extracted from the angle calculation image divided into the 4 × 4 areas shown in FIG. 15, and the edge vector of the main information for each area is used as the main information for each divided area. An example calculated as the directionality is shown. In FIG. 16B, only the auxiliary information is extracted from the angle calculation image divided into the 4 × 4 areas shown in FIG. 15, and the edge vector of the auxiliary information for each area is used as the auxiliary information for each divided area. An example calculated as the directionality is shown.

(Palm model in Example 1)
FIG. 17 is a diagram showing an example of a palm model in the first embodiment. FIG. 17A shows an example of a palm model regarding the main information (strong information) among the necessary information. The palm model for the main information (strong information) includes the outer shell of the entire palm and the edges of the main information contained within the outer shell. FIG. 17B shows an example of a palm model regarding auxiliary information (weak information) among necessary information. The palm model for auxiliary information (weak information) includes the outer shell of the entire palm and the edges of the auxiliary information contained within the outer shell.

(Estimation result of a plurality of palm partial candidates in Example 1)
FIG. 18 is a diagram showing an example of estimation results of a plurality of palm partial candidates in the first embodiment. The two solid rectangular boxes illustrated in FIG. 18 (a) are a palm partial image, a palm model for main information (strong information) (see (a) in FIG. 17), and a palm model for auxiliary information (weak information). It is a plurality of palm partial candidates estimated from (see (b) of FIG. 17). Further, FIG. 18B shows one of the estimated two palm partial candidates (see FIG. 18A), and FIG. 18C shows the estimated two palms, for example. The other of the partial candidates is shown. As described above, in the embodiment, a plurality of palm partial candidates of the palm partial image are estimated from one palm partial image.

(Successful example of angle calculation processing in Example 1)
FIG. 19 is a diagram showing a successful example of the angle calculation process in the first embodiment. FIG. 19A shows the direction (angle) of the palm indicated by the solid arrow (calculation result) and the correct orientation (angle) of the palm indicated by the dashed arrow (correct answer) shown in FIG. 8A-3. This is an example of a case where the calculation result and the correct answer match without any discrepancy. Similarly, FIG. 19 (b) shows the orientation (angle) of the palm indicated by the solid arrow (calculation result) and the correct orientation (correct answer) of the palm indicated by the dashed arrow (correct answer) shown in FIG. 8 (b-3). An example is shown in which the calculation result and the correct answer match without any deviation from the angle).

As described above, in the first embodiment, a plurality of palm partial candidates of the palm partial image are estimated, and for each of the plurality of estimated palm partial candidates, the palm partial candidate and the palm model are compared, and the main information is converted into the palm partial candidate. The based angle is calculated, and the calculated angle is corrected based on the auxiliary information. Then, the angle of the palm portion candidate that is probable as the palm is defined as the angle of the palm portion image. Therefore, the angle calculation processing result of the palm partial image can be made correct, and the decrease in the accuracy of the calculated angle can be suppressed.

Further, according to the first embodiment, when the biometric authentication device 1 is photographed at a short distance of less than a predetermined distance and, for example, at least about 60 to 70 [%] is photographed with respect to the total area of the palm. Can correctly calculate the angle of the palm even if the palm part image has only a part of the outer shell or almost no outer shell. Further, according to the first embodiment, when the biometric authentication device 1 is photographed at a short distance of less than a predetermined distance and less than 30 [%] is photographed with respect to the total area of the palm, rotation is performed. Since it may not be possible to determine which palm is on the left or right, the angle can be calculated accurately by specifying in advance which hand is on the left or right.

In the first embodiment, when the palm is only partially imaged, more detailed information (vulnerable information when the entire palm is imaged) is assisted as compared with the case where the entire palm is imaged. It is used as information to estimate the position of the imaged palm part of the palm. Further, in the first embodiment, the method of extracting the auxiliary information is dynamically changed according to the position of the imaged palm portion in the palm. Therefore, according to the first embodiment, the amount and accuracy of the auxiliary information can be improved, and the angle of the palm can be calculated more accurately, which in turn contributes to the improvement of the authentication accuracy.

In the second embodiment, a plurality of palm partial candidates are estimated from one palm partial image, a palm partial candidate having the highest degree of certainty is determined from the estimated palm partial candidates, and the determined palm partial candidate is used. It is assumed that the palm partial image is rotated by the base angle to generate the original image for creating the palm vein image. However, the present invention is not limited to this, and in the second embodiment, the first-ranked palm portion candidate is not determined, and the palm portion image is rotated at each of the estimated angles of the plurality of palm portion candidates to create a palm vein image. Generate multiple original images.

(Functional configuration of biometric authentication device according to Example 2)
The biometric authentication device 1A according to the second embodiment has a normalization unit 120A instead of the normalization unit 120 as compared with the biometric authentication device 1 according to the first embodiment (see FIG. 4).

(Details of the Normalization Unit of the Biometric Authentication Device According to Example 2)
FIG. 20 is a functional block diagram showing a detailed example of the normalization unit of the biometric authentication device according to the second embodiment. In the normalization unit 120A of the biometric authentication device 1A according to the second embodiment, the first place confirmation unit 132 is omitted as compared with the normalization unit 120 according to the first embodiment, and the rotation unit 135-1 is replaced with the rotation unit 135. Has. The normalization unit 120A is an example of an image processing device.

Further, in the second embodiment, the certainty calculation unit 126, the angle calculation image creation unit 127, the auxiliary information extraction unit 128, the auxiliary information certainty calculation unit 129, the angle calculation unit 1_130, the certainty recalculation unit 131, and the rotation unit 135- In case 1, when there are a plurality of (N) palm part candidates estimated by the palm part estimation unit 125, a series of processes performed for each of the plurality of palm part candidates is repeated until all the palm part candidates are completed. Do.

The rotating unit 135-1 creates a palm vein image by rotating the palm portion image according to the estimated angle and correction information based on the palm portion candidate whose correction certainty is calculated by the certainty recalculation unit 131. Generate the original image of. In this case, the original image for creating the palm vein image is generated for each of the plurality of palm portion candidates estimated by the palm portion estimation unit 125. Further, the rotating unit 135-1 is a source for creating a palm vein image by rotating the palm image whose palm direction and inclination are calculated by the angle calculation unit 2_134 according to the direction and inclination of the palm. Generate an image.

As described above, the palm part is formed by the certainty calculation unit 126, the angle calculation image creation unit 127, the auxiliary information extraction unit 128, the auxiliary information certainty calculation unit 129, the angle calculation unit 1_130, the certainty recalculation unit 131, and the rotation unit 135-1. When a series of processes performed for each of the N palm portion candidates estimated by the estimation unit 125 is completed for all the palm portion candidates, the normalization unit 120A displays the palm vein image generated by the rotating unit 135-1. Output the original image to create.

The rotating unit 135-1 may output at least one of the estimated angle, correction information, and correction certainty of the entire palm image and the partial palm image together with the original image for creating the palm vein image.

(Normalization processing according to Example 2)
FIG. 21 is a flowchart showing an example of the normalization process according to the second embodiment. In the normalization process according to the second embodiment, as compared with the normalization process according to the first embodiment, each time the correction certainty is calculated by the certainty recalculation unit 131, a plurality of normalization processes estimated by the palm partial estimation unit 125 are performed. The palm partial image is rotated according to the estimated angle and correction information based on the corresponding palm partial candidate among the palm partial candidates.

As shown in FIG. 21, the certainty calculation unit 126, the angle calculation image creation unit 127, the auxiliary information extraction unit 128, the auxiliary information certainty calculation unit 129, the angle calculation unit 1_130, the certainty recalculation unit 131, and the rotation unit 135- 1 performs a series of processes of steps S108 to S115a performed for each palm partial candidate for N palm partial candidates estimated by the palm partial estimation unit 125 for all palm partial candidates.

That is, the rotation unit 135-1, when the corrected confidence k _{n 'by} confidence recalculation unit 131 is calculated in step S113, in accordance with the estimated angle and the correction information based on the selected palm portion candidate in step S108 By rotating the palm partial image, the original image for creating the palm vein image is generated.

Then, the certainty calculation unit 126, the angle calculation image creation unit 127, the auxiliary information extraction unit 128, the auxiliary information certainty calculation unit 129, the angle calculation unit 1_130, the certainty recalculation unit 131, and the certainty recalculation unit 131 are in the palm of the hand. Of the plurality of palm partial candidates estimated by the partial estimation unit 125, all the palm partial candidates not selected in step S108 are processed in steps S108 to S113. When the process of step S115a for the palm portion candidate last selected in step S108 is completed, the normalization unit 120A ends the normalization process according to the second embodiment.

On the other hand, in step S115b, the rotating unit 135-1 creates a palm vein image by rotating the palm image whose angle has been calculated in step S122 or step S124 according to the estimated angle and correcting the angle. Generate the original image for. When step S115b is completed, the normalization unit 120A ends the normalization process according to the second embodiment.

The components of each device or device shown in the above-described embodiment are functional concepts, and do not necessarily have to be physically configured as shown in the figure. That is, the specific state of distribution or integration of each device is not limited to the one shown in the figure, and all or part of the distribution or integration is functionally or physically distributed in arbitrary units according to the processing load and processing efficiency. Or it can be integrated and configured.

For example, at least two of the certainty calculation unit 126, the auxiliary information certainty calculation unit 129, and the certainty recalculation unit 131 may be integrated. Further, the angle calculation unit 1_130 and the angle calculation unit 2_134 may be integrated.

Further, each processing function performed in each processing unit of the biometric authentication devices 1 and 1A is a program in which all or any part thereof is analyzed and executed by a semiconductor processing device such as a microcomputer and a semiconductor processing device. It can be realized.

The configuration of each part illustrated in the above examples can be changed, added or omitted to the extent that it does not deviate from the technical scope of the image processing apparatus, the image processing method and the image processing program according to the disclosed technology. In addition, the examples are merely examples, and the disclosed technology also includes other aspects described in the disclosure column of the invention and various modifications and improvements based on the knowledge of those skilled in the art. ..

1, 1A Biometric authentication device 100 Processing device 110 Imaging unit 120, 120A Normalization unit 121 Overall judgment unit 122 Left / right judgment unit 123 Left / right designation unit 124 Overall hand estimation unit 125 Palm part estimation unit 126 Confidence calculation unit 127 Angle calculation image creation Part 128 Auxiliary information extraction unit 129 Auxiliary information certainty calculation unit 130 Angle calculation unit 1
131 Conviction recalculation unit 132 1st place confirmation unit 133 Palm extraction unit 134 Angle calculation unit 2
135, 135-1 Rotating unit 140 Feature extraction unit 150 Registration unit 160 Matching unit 200 Image sensor 300 Light emitting element 400 Distance measuring light emitting element 500 Image pickup DB
600 candidate DB

Claims (7)

A determination unit for determining whether the living body part is the whole or a part from the captured image captured including the whole or a part of the living body part.
When the determination unit determines that the captured image includes a part of the living body part, it is estimated which part of the living body part is a part of the whole living body part. Department and
An angle calculation image emphasizing information that characterizes the angle of the living body part is created from the captured image in which a part of the living body part is estimated by the estimation unit as a part of the whole living body part. Angle calculation image creation unit and
An angle calculation unit that generates a vector from the information that characterizes the angle in the angle calculation image created by the angle calculation image creation unit, and calculates the angle of the living body portion with respect to a predetermined reference direction based on the generated vector. ,
An image processing device characterized by being equipped with. The information that characterizes the angle of the living body portion includes strong information having an intensity of a predetermined value or more and weak information having an intensity of less than the predetermined value.
The angle calculation unit generates a vector for each of the strong information and the weak information in the angle calculation image, and calculates the angle of the living body portion with respect to the predetermined reference direction based on the vector generated from the strong information. At the same time, the angle of the living body portion calculated above is corrected based on the vector generated from the weak information.
The image processing apparatus according to claim 1. The angle calculation unit divides the angle calculation image into a plurality of areas, and generates a vector from the information characterizing the angle for each divided area.
The image processing apparatus according to claim 1 or 2. The estimation unit estimates a plurality of partial candidates indicating which part of the whole living body part is a part of the living body part.
The angle calculation image creation unit creates the angle calculation image from the captured image for each of the partial candidates, and creates the angle calculation image.
The angle calculation unit generates a vector for each of the partial candidates from the information that characterizes the angle in the angle calculation image.
The image processing apparatus according to any one of claims 1 to 3. For each of the partial candidates, a confidence calculation unit that calculates the certainty of the palm portion indicated by the partial candidate, and
A confirmation unit that determines the angle of the living body portion with respect to the predetermined reference direction based on the certainty calculated by the certainty calculation unit.
The image processing apparatus according to claim 4, further comprising. This is an image processing method performed by an image processing device.
From the captured image captured including the whole or a part of the living body part, it is determined whether it is the whole or a part of the living body part.
When it is determined that the captured image includes a part of the living body part, it is estimated which part of the whole living body part is the part of the living body part.
From the captured image in which it is estimated which part of the living body part is a part of the whole living body part, an angle calculation image emphasizing the information characterizing the angle of the living body part is created.
A vector is generated from the information that characterizes the angle in the angle calculation image, and the angle of the living body portion with respect to a predetermined reference direction is calculated based on the generated vector.
An image processing method characterized by including each processing. Computer,
A determination unit for determining whether the living body part is the whole or a part from the captured image captured including the whole or a part of the living body part.
When the determination unit determines that the captured image includes a part of the living body part, it is estimated which part of the living body part is a part of the whole living body part. Department and
An angle calculation image emphasizing information that characterizes the angle of the living body part is created from the captured image in which a part of the living body part is estimated by the estimation unit as a part of the whole living body part. Angle calculation image creation unit and
An angle calculation unit that generates a vector from the information that characterizes the angle in the angle calculation image created by the angle calculation image creation unit, and calculates the angle of the living body portion with respect to a predetermined reference direction based on the generated vector.
An image processing program to function as.

Images