JP2022048464A - Nose print collation device, method and program - Google Patents

Abstract

To provide a nose print collation device, a method and a program capable of collating a nose print image with high accuracy.SOLUTION: A nose print collation device includes: a nose print position estimation unit 11 that accepts input of a face image of a captured arbitrary face into a neural network for detecting a nose print machine-learned by means of position information of nose prints included in a plurality of face images captured in advance, so as to estimate a position of a nose print included in the face image; a nose print image extraction unit 12 that specifies a nose print area based on the estimated position of the nose print to extract a nose print image in the nose print area; a nose print image feature amount acquisition unit 13 that accepts input of a plurality of nose print images to be collated to the neural network for classifying nose print images machine-learned by means of the plurality of nose print images acquired in advance, so as to acquire feature amounts of the plurality of nose print images; and a nose print image collation unit 14 that collates the plurality of nose print images based on the feature amounts of the plurality of nose print images, so as to output a result of the collation.SELECTED DRAWING: Figure 1

Description

The present invention relates to a nasal print collation device, a method, and a program for collating nasal prints extracted from a plurality of facial images.

Like human fingerprints, cows have their own unique nose prints, so individual cows are identified using the unique nose prints, and individual management is performed. For example, a nose print collation method is adopted in which a bovine nose print is registered in advance and an individual cow is identified by collation of the nose print at a later date.

Conventionally, the tip of the nose of a cow is inked, the nose print is collected and stored in advance, and the stored nose print is visually compared with the collected nose print at a later date. However, it is very time-consuming and stresses the cow.

Therefore, in recent years, a method has been proposed in which a cow's nose print is photographed with a camera and collation is performed by comparing the photographed images.

For example, Patent Document 1 proposes a method of extracting the longest line segment of a plurality of cells surrounded by a groove existing in a nose print and collating the nose print using the distribution state of the line segment. There is.

Further, in Patent Document 2, dog candidates are extracted using a machine-learned classifier, feature points are extracted from the nose print images of the dog candidates, and the extracted feature points and the nose print to be collated are extracted. A method of collating a dog's nose print by comparing it with a feature point of an image has been proposed.

Patent No. 4190209 Patent No. 6567638

However, in the method described in Patent Document 1, since it is necessary to extract line segments of a large number of cells, the processing is very heavy, and the collation accuracy is influenced by the extraction accuracy of the line segments, and high collation accuracy is obtained. Can't.

Further, in the method described in Patent Document 2, a smoothing process is executed a plurality of times on a nose print image, and feature points are calculated by detecting the gradient direction of luminance from the difference between the smoothed images. However, it was found that high collation accuracy cannot be obtained by collation by comparing feature points calculated by such a method.

Further, in Patent Document 2, the position of the nose is detected from six points of the dog's two eyes, nose, two ears, and the apex of the forehead. If there is no dog, or if the dog is not facing forward and contains only one ear, the positions of the two ears cannot be identified, so the position of the nose cannot be detected and a nose print image is obtained. I can't. That is, in Patent Document 2, the collation process cannot be performed unless the dog faces the front and the entire face of the dog is photographed.

In view of the above problems, it is an object of the present invention to provide a nose print collation device, a method and a program capable of collating a nose print image with high accuracy.

The nose print collating device of the present invention is a face image obtained by photographing an arbitrary face with respect to a neural network for detecting nose prints machine-learned using the position information of the nose prints included in a plurality of face images taken in advance. The nose print area is specified based on the nose print position estimation unit that estimates the position of the nose print included in the face image and the nose print position estimated by the nose print position estimation unit, and the identification is performed. A plurality of noses to be collated against a nose print image extraction unit that extracts a nose print image in a nose print region and a neural network for classifying a nose print image machine-learned using a plurality of previously acquired nose print images. A nose print image feature amount acquisition unit that acquires the feature amount of a plurality of nose print images by inputting a pattern image, and a plurality of nose print images are collated based on the feature amounts of the plurality of nose print images, and the same. It is provided with a nose print image collation unit that outputs the collation result.

Further, in the nose print collation device of the present invention, the nose print image feature amount acquisition unit can acquire a feature vector as a feature amount.

Further, in the nose print collation device of the present invention, the nose print collation unit can calculate the similarity of a plurality of nose print images based on the feature vectors of the plurality of nose print images.

Further, in the nose print collating device of the present invention, the neural network for detecting the nose print is input with an image having a mark surrounding all or a part of the nose print for a plurality of face images taken in advance. It can be machine-learned.

Further, in the nose print collating device of the present invention, the plurality of face images taken in advance are face images taken from a direction other than the front direction, and marks surrounding all or part of the nose print are present. The attached facial image can be included.

Further, in the nose print collating device of the present invention, the nose print can be a cow nose print.

The nose print collation method of the present invention is a face image obtained by photographing an arbitrary face with respect to a neural network for detecting nose prints machine-learned using the position information of the nose prints included in a plurality of face images taken in advance. By inputting, the position of the nose print included in the face image is estimated, the nose print area is specified based on the estimated position of the nose print, and the nose print image of the specified nose print area is extracted in advance. By inputting multiple nasal print images to be matched against a neural network for nasal print image classification machine-learned using multiple acquired nasal print images, feature quantities of multiple nasal print images are acquired. Then, based on the feature quantities of the plurality of nose print images, the plurality of nose print images are collated, and the collation result is output.

The nose print collation program of the present invention is a face image obtained by photographing an arbitrary face with respect to a neural network for detecting nose prints machine-learned using the position information of the nose prints included in a plurality of face images taken in advance. A step of estimating the position of the nose print included in the face image by inputting, and a step of specifying the nose print area based on the estimated position of the nose print and extracting the nose print image of the specified nose print area. And, by inputting a plurality of nasal print images to be collated into a neural network for classifying nasal print images machine-learned using a plurality of nasal print images acquired in advance, features of a plurality of nasal print images. A computer is made to perform a step of acquiring the amount and a step of collating a plurality of nose print images based on the feature quantities of the plurality of nose print images and outputting the collation result.

According to the nose print collation device, method and program of the present invention, the position of the nose print included in the face image is estimated using the nose print detection neural network, and a plurality of nose print classification neural networks are used. Since the feature amount of the nose print image was acquired and the multiple nose print images were collated based on the feature quantity of the acquired multiple nose print images, that is, the estimation of the nose print position and the feature of the nose print image were performed. Since the collation of the nose print image is performed using the neural network in two steps of the acquisition of the quantity, the collation of the nose print image can be performed with high accuracy.

A block diagram showing a schematic configuration of a nasal print matching system using an embodiment of the nasal print matching device of the present invention. A diagram showing an example of a cow's face image input to a neural network for detecting nose prints. A flowchart showing the processing flow of the nose print matching system shown in FIG.

Hereinafter, a nasal print collation system using an embodiment of the nasal print collation device of the present invention will be described in detail with reference to the drawings. FIG. 1 is a schematic configuration diagram of the nose print collation system 1 of the present embodiment.

As shown in FIG. 1, the nasal print collation system 1 of the present embodiment includes a nasal print collation device 10, an imaging device 20, and a display device 30. The nose print collating device 10 and the photographing device 20 and the display device 30 are communicably connected by wire or wirelessly, and are configured to enable exchange of various signals.

The nose print collation device 10 of the present embodiment receives a plurality of facial images obtained by photographing a cow's face, extracts each of the nose print images including the nose print from the plurality of face images, and collates these nose print images. It is possible to determine whether or not the image is the same cow's nose print.

As shown in FIG. 1, the nose print collation device 10 includes a nose print position estimation unit 11, a nose print image extraction unit 12, a nose print image feature amount acquisition unit 13, and a nose print image matching unit 14. There is.

The nose print collating device 10 is composed of, for example, a computer, and includes a CPU (Central Processing Unit), a storage medium such as a semiconductor memory and a hard disk, a communication I / F, and the like.

An embodiment of the nose print matching program of the present invention is installed in the semiconductor memory or the hard disk of the nose print matching device 10. Then, when this nose print matching program is executed by the CPU, the above-mentioned nose print position estimation unit 11, nose print image extraction unit 12, nose print image feature amount acquisition unit 13, and nose print image matching unit 14 function. do.

In addition, in this embodiment, all the functions of the above-mentioned parts are executed by the nose print collation program, but the present invention is not limited to this, and some or all the functions are performed by ASIC (Application Specific Integrated Circuit) or FPGA (Field). -Programmable Gate Array) and other hardware such as electric circuits may be configured.

The nose print position estimation unit 11 receives a face image of an arbitrary cow's face and estimates the position of the cow's nose print included in the face image. Specifically, the nose print position estimation unit 11 of the present embodiment stores a neural network for nose print detection machine-learned using the position information of the nose print included in the face images of a plurality of cows taken in advance. do. As the neural network for detecting the nose print, a convolutional neural network having a structure such as VGC16, ResNet, SENe, Darknet can be used.

FIG. 2 is a diagram showing an example of a cow face image input to the above-mentioned neural network for detecting nose prints. As shown in FIG. 2, the cow's face image is marked with, for example, a rectangle as an index indicating the position information of the nose print. A large number of cow face images with the above-mentioned rectangular marks are input to the neural network for detecting nose prints, and machine learning is performed.

The rectangular mark is preferably a rectangular mark that surrounds all or part of the nose print. As the rectangular mark that surrounds the entire nose print, when the outer circumference of the nose print is surrounded by a circle or an ellipse, a rectangular mark that touches the circle or the ellipse can be used. Further, as the rectangular mark surrounding a part of the nose print, for example, a rectangular mark surrounding the vicinity of the center of the nose print can be used. The shape of the mark surrounding the nose print is not limited to a rectangle, and may be a circle, an ellipse, or another polygon.

Further, the face image of the cow is not limited to the face image in which the cow faces the front toward a photographing device such as a camera, but also preferably includes the face image in which the cow faces in another direction. However, it is assumed that the above rectangular mark is always included in the face image facing in other directions.

By performing machine learning including a face image facing a direction other than the front as described above, for example, even if the face image to be collated is a face image taken in a direction other than the front, the nose print. The position of can be estimated appropriately.

Then, the nose print position estimation unit 11 inputs a face image obtained by photographing an arbitrary cow's face to the above-mentioned neural network for nose print detection, and the position of the cow's nose print included in the face image. To estimate. Specifically, the nose print position estimation unit 11 estimates a rectangle indicating the position of the nose print included in the input face image, and outputs the position information of the rectangle to the nose print image extraction unit 12.

The nose print image extraction unit 12 identifies the nose print region based on the position of the nose print estimated by the nose print position estimation unit 11, and extracts the nose print image of the specified nose print region. Specifically, the nose print image extraction unit 12 of the present embodiment extracts an image in a rectangle indicating the position of the nose print estimated by the nose print position estimation unit 11 as a nose print image. The range of the nose print image to be extracted from the face image is not limited to this, and may be extracted larger or smaller based on the range of the rectangle.

The nose print image feature amount acquisition unit 13 receives a plurality of nose print images extracted by the nose print image extraction unit 12 and acquires the feature amounts of the plurality of nose print images. Specifically, the nose print image feature amount acquisition unit 13 of the present embodiment stores a neural network for classifying nose print images machine-learned using a plurality of previously acquired nose print images. As the neural network for classifying the nose print, a convolutional neural network having a structure such as VGC16, ResNet, SENe, Darknet can be used.

Then, the nose print image feature amount acquisition unit 13 of the present embodiment inputs the nose print image to the neural network for classification of the nose print image, and converts the nose print image into a low-dimensional feature vector. That is, the nose print image is encoded and converted into a feature vector. Specifically, the feature vector of the nasal print image is acquired by extracting the output of the intermediate layer of the neural network for nasal print image classification.

In the present embodiment, a plurality of nose print images are input to the nose print image feature amount acquisition unit 13, and the feature vector of each nose print image is acquired.

The nose print image collation unit 14 collates a plurality of nose print images based on the feature vectors of the plurality of nose print images acquired by the nose print image feature amount acquisition unit 13, and outputs the collation result. Specifically, the nose print image matching unit 14 of the present embodiment calculates the similarity of the feature vectors of the two nose print images as the matching process of the two nose print images. As a method of calculating the similarity, for example, the Euclidean distance between the two feature vectors and the cosine similarity are calculated. In this way, by calculating the similarity of the feature vectors of the plurality of nose print images, it is possible to collate the plurality of nose print images with high accuracy.

Then, the nose print image collation unit 14 outputs the collation result based on the calculated similarity. As a method of outputting the collation result, for example, the similarity itself may be output, or it is determined whether or not the similarity is equal to or higher than a preset threshold value, and if it is equal to or higher than the threshold value, "collation is established". Information may be output, and if it is less than the threshold value, the information of "verification failure" may be output.

The photographing device 20 is provided with a camera or the like, photographs the face of the cow to be collated, and outputs the photographed face image to the nose print collating device 10.

The display device 30 includes a display device such as a liquid crystal display, and presents various information to the user. Specifically, the display device 30 of the present embodiment outputs a cow face image input to the nose print collation device 10, a nose print image extracted by the nose print image extraction unit 12, and a nose print image matching unit 14. Display the collation result.

Further, the nose print collation device 10, the photographing device 20, and the display device 30 may be configured by a tablet terminal having a camera function.

Next, the processing flow of the nose print collation system 1 of the present embodiment will be described in more detail with reference to the flowchart shown in FIG.

First, a face image of a cow to be collated is photographed using the photographing device 20, and the photographed face image is input to the nose print collation device 10 (S10).

The face image input to the nose print collating device 10 is input to the above-mentioned neural network for detecting nose prints, and the position of the nose print of the cow included in the face image is estimated (S12).

Next, a nasal print region is identified based on the estimated position of the bovine nasal print, and a nasal print image of the identified nasal print region is extracted from the facial image (S14).

The extracted nose print image is input to the above-mentioned neural network for nose print image classification, encoded, and a feature vector is acquired (S16).

Then, based on the feature vector of the nose print image acquired from the cow face image taken this time and the cow face image taken before, it is acquired by performing the same processing as S10 to S16 described above. The collation process is performed by calculating the similarity with the feature vector of the nose print image (S18).

Then, the collation result is displayed on the display device 30 and presented to the user (S20).

According to the nose print matching system of the above embodiment, the position of the nose print included in the face image is estimated using the nose print detection neural network, and a plurality of nose print images are estimated using the nose print image classification neural network. The feature amount of the Since the collation of the nose print image is performed using the two-step neural network of the above, the collation of the nose print image can be performed with high accuracy.

In the description of the above embodiment, the two cow face images are collated, but the present invention is not limited to this, and the face images of a plurality of cows are stored in advance and the cow face images taken this time are used. The feature vector of the acquired nose print image may be collated with the feature vector of the plurality of nose print images acquired from the plurality of face images stored in advance.

Then, the cow photographed this time may be identified by specifying the face image of the same cow as the cow photographed this time from the face images of the plurality of cows stored in advance.

Further, in the nose print collation system 1 of the above embodiment, the nose print of a cow is collated, but the nose print of not only a cow but also other organisms such as dogs and cats is collated. May be good.

1 Nose print matching system 10 Nose print matching device 11 Nose print position estimation unit 12 Nose print image extraction unit 13 Nose print image feature amount acquisition unit 14 Nose print image matching unit 20 Imaging device 30 Display device

Claims (8)

By inputting a face image of an arbitrary face into a neural network for detecting a nose print machine-learned using the position information of the nose print included in a plurality of face images taken in advance, the face image can be obtained. The nose print position estimation unit that estimates the position of the included nose print,
A nasal print image extraction unit that identifies a nasal print region based on the position of the nasal print estimated by the nasal print position estimation unit and extracts a nasal print image of the specified nasal print region.
By inputting the plurality of the nasal print images to be collated into the neural network for classifying the nasal print images machine-learned using the plurality of the nasal print images acquired in advance, the plurality of nasal print images can be obtained. Nose print image to acquire features
A nose print collation device including a nose print image collation unit that collates the plurality of nose print images based on the feature amounts of the plurality of nose print images and outputs the collation result. The nose print collation device according to claim 1, wherein the nose print image feature amount acquisition unit acquires a feature vector as the feature amount. The nose print matching device according to claim 1 or 2, wherein the nose print matching unit calculates the similarity of the plurality of nose print images based on the feature vectors of the plurality of nose print images. The claim that the neural network for detecting a nose print is machine-learned by inputting an image with a mark surrounding all or a part of the nose print on a plurality of face images taken in advance. The nose print collating device according to any one of 1 to 3. The fourth aspect of claim 4, wherein the plurality of face images taken in advance are face images taken from a direction other than the front direction, and include a face image with a mark surrounding all or part of the nose print. Nose print matching device. The nose print collating device according to any one of claims 1 to 5, wherein the nose print is a cow nose print. By inputting a face image of an arbitrary face into a neural network for detecting a nose print machine-learned using the position information of the nose print included in a plurality of face images taken in advance, the face image can be obtained. Estimate the position of the included nose print and
The nasal print region is specified based on the estimated position of the nasal print, and the nasal print image of the specified nasal print region is extracted.
By inputting the plurality of the nasal print images to be collated into the neural network for classifying the nasal print images machine-learned using the plurality of the nasal print images acquired in advance, the plurality of nasal print images can be obtained. Get the feature amount,
A nose print collation method in which the plurality of nose print images are collated based on the feature amounts of the plurality of nose print images, and the collation result is output. By inputting a face image of an arbitrary face into a neural network for detecting a nose print machine-learned using the position information of the nose print included in a plurality of face images taken in advance, the face image can be obtained. Steps to estimate the position of the included nose print,
A step of identifying a nasal print region based on the estimated position of the nasal print and extracting a nasal print image of the specified nasal print region.
By inputting the plurality of the nasal print images to be collated into the neural network for classifying the nasal print images machine-learned using the plurality of the nasal print images acquired in advance, the plurality of nasal print images can be obtained. Steps to acquire features and
A nose print collation program that causes a computer to perform a step of collating the plurality of nose print images based on the feature amounts of the plurality of nose print images and outputting the collation result.

Images