KR20200013160A - Nose pattern recognition system and method based on deep learning - Google Patents

Abstract

The present invention relates to a deep learning-based nose pattern recognition system and to a method thereof. The deep learning-based nose pattern recognition system can input a face image of an animal belonging to a canine family, extract a nose pattern feature point image from the input face image, determine a detailed species of the animal by comparing the extracted nose pattern feature point image with a plurality of images stored in a first database unit, and output the determined detailed species result. Accordingly, the deep learning-based nose pattern recognition system of the present invention prevents the animal from being thrown away through the division of the species, the detailed species, and the like of the animal belonging to the canine family, and is capable of finding abandoned animals. In addition, according to the present invention, by determining the species based on deep learning for both the primary species determination and the secondary detailed species determination, the animal can be more objectively, accurately, and quickly recognized.

Description

Nose pattern recognition system and method based on deep learning

The present invention relates to inscription recognition, and more particularly, to a deep learning based inscription recognition system and method for recognizing inscriptions of animals belonging to a canine based on deep learning to determine detailed species.

Humans are using the field of biometrics as a way to verify and verify their identity, which has been developed for a long time. Such biometric authentication methods include fingerprint recognition, iris recognition, and vein recognition.

In this regard, as human fingerprints vary from person to person, the inscriptions of animals belonging to the canine vary from animal to animal. Therefore, the method of identifying the species of the animal or identifying the animal that the owner wants to find may enable systematic management of the animal.

However, the fingerprint recognition method of the general person and the inscription recognition method of the animal are different from the specific recognition method, so it is necessary to prepare a new model for this.

Korean Patent Publication No. 1494716 Japanese Patent Publication No. 4190209

Accordingly, the present invention has been proposed in consideration of the above-described matters, and aims to prevent the animal from being discarded by distinguishing between the species and detailed species of the animal belonging to the canine and to find the abandoned animal. .

In addition, an object of the present invention is to implement an optimized image processing according to the inscription characteristics of each animal by easily changing the parameters according to the image processing in accordance with the inscription characteristics of each animal belonging to the canine.

In addition, the present invention aims to effectively extract the image of the network form from the inscription of the animal belonging to the canine.

It is also an object of the present invention to extract more stable feature points from the inscriptions of animals belonging to the canine.

In addition, the present invention aims to minimize error and prevent distinct ambiguity through primary species determination and secondary detailed species determination.

In addition, the present invention aims to recognize species more objectively, accurately and quickly by judging species based on deep learning in both primary species determination and secondary detailed species determination.

The objects of the present invention are not limited to the above-mentioned objects, and other objects that are not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, the deep learning-based inscription recognition system according to the present invention provides an image input unit for inputting a face image of an animal belonging to a canine, and extracts an inscription feature point image from the input face image. And a detailed species determination unit for determining the detailed species of the animal through comparison with the extracted inscription feature point image and the plurality of images stored in the first database unit, and a detailed species result output unit for outputting the determined detailed species result. .

In this case, the inscription feature point image extractor sets an image corresponding to a region of interest (ROI) among the input face images, and then a preprocessing unit preprocessing the filtered image and a preprocessed ROI image to extract a pattern value. A filter image output unit for outputting an image, and a feature point image output unit for outputting an inscription feature point image by extracting feature points from the output filter image.

The feature point extracted from the feature point image output unit may include at least one inscription feature point position, scale, and direction.

The preprocessor may apply an adaptive thresholding algorithm to the input face image for setting the ROI image.

The filter image output unit may apply a Gabor filter to the ROI image preprocessed for output of the filter image.

The feature point image output unit may apply a shift algorithm to the output filter image for extracting feature points.

The detail type determination unit is an image comparison unit which calculates and compares Euclidean distance values (Euclidean distance, L2-Norm) between the extracted feature points and the feature points of the plurality of images stored in the first database unit, and the plurality of images stored in the first database unit. The matching feature point selection unit may select a feature point of the image having the smallest Euclidean distance value among the feature points as a matching feature point.

The matching feature point selector may further select the matching feature points in order of decreasing Euclidean distance value in addition to the selected matching feature points, and determine the matching reliability of the selected matching feature points using a matching decision equation defined by the following equation. have.

Equation

E2 / E1> (Threshold)

(E1: selected matching feature point, E2: additionally selected matching feature point)

The apparatus may further include a species determination unit that determines a species of the animal by comparing the face image input from the image input unit with the image stored in the second database unit.

The first database unit may include a plurality of variation images collected with standardized data capable of determining the detailed species of the animal.

The second database unit may include a plurality of breed images collected with standardized data that may determine the species of the animal.

In order to achieve the above object, the deep learning-based inscription recognition method according to the present invention includes an image input step of receiving a face image of an animal belonging to a canine from an image input unit, and a face image input from an inscription feature point image extractor. An inscription feature point image extraction step of extracting an inscription feature point image, a detail species determination step of determining a detailed species of an animal by comparing the inscription feature point image extracted by the detail species determination unit with a plurality of images stored in the first database unit, and the detail species The detailed seed result output step of outputting the detailed seed result determined by the result output unit may be included.

The inscription feature point image extraction step sets an image corresponding to a region of interest (ROI) among face images input by the preprocessor, and then preprocesses the preprocessing step, and filters the preprocessed ROI image from the filter image output unit. The filter image output step may include outputting a filter image from which a pattern value is extracted, and outputting a feature point image by extracting feature points from the filter image output from the feature point image output unit.

The feature point extracted in the feature point image output step may include at least one inscription feature point position, scale, and direction.

In the preprocessing step, an adaptive thresholding algorithm may be applied to the input face image for setting the ROI image.

The filter image output step may apply a Gabor filter to the ROI image preprocessed for output of the filter image.

In the feature point image output step, a shift algorithm may be applied to the output filter image for extraction of the feature point.

The detailed species determination step includes an image comparison step of calculating and comparing Euclidean distance values (L2-Norm) between the inscription feature points extracted from the image comparison unit and the feature points of the plurality of images stored in the first database unit, and the matching feature point line. And a matching feature point selecting step of selecting, as a matching feature point, a feature point of the image having the smallest Euclidean distance value among the feature points of the plurality of images stored in the first database unit.

In the matching feature point selection step, the matching feature points are further selected in order of the calculated Euclidean distance values in addition to the selected matching feature points, and then the matching reliability of the selected matching feature points is determined using a matching decision equation defined by the following equation. Can be.

Equation

E2 / E1> (Threshold)

(E1: selected matching feature point, E2: additionally selected matching feature point)

The species determination unit may further include a species determination step of determining the species of the animal by comparing the face image input from the image input step with the image stored in the second database unit.

The first database unit may include a plurality of variation images collected with standardized data capable of determining the detailed species of the animal.

The second database unit may include a plurality of breed images collected with standardized data that may determine the species of the animal.

According to the deep learning-based inscription recognition system and method as described above,

First, it is possible to prevent the animal from being discarded by distinguishing between the species, detailed species, etc. of the animal belonging to the canine, and has the effect of finding an abandoned animal.

Second, by easily changing the parameters according to the image processing according to the inscription characteristics of each animal belonging to the canine, it has the effect that can implement the optimized image processing according to the inscription characteristics of each animal.

Third, it has an effect that can effectively extract the image of the network form from the inscription of the animal belonging to the canine.

Fourth, it has the effect of extracting more stable feature points from the inscriptions of animals belonging to the canine.

Fifth, it is possible to minimize error and prevent distinct ambiguity through the determination of the primary species and the secondary detailed species.

Sixth, the primary species judgment and the secondary detailed species judgment both judge the species based on deep learning, thereby having an effect that can be recognized more objectively, accurately and quickly.

)에 따른 가보 필터링 이미지 결과를 나타낸 도면.
도 10은 본 발명의 실시예로서 강아지(또는 개)의 비문 매칭 과정을 나타낸 도면.
도 11은 본 발명의 실시예로서 5마리 강아지(또는 개)에 대한 점수 비교를 나타낸 그래프.
도 12는 본 발명의 실시예로서 시프트(Shift), 서프(Surf), 오브(Orb) 매칭을 비교한 도면.
도 13은 본 발명의 실시예로서 매칭 방법에 따른 정확도 및 시간 결과를 나타낸 그래프. 1 is a block diagram showing a deep learning-based inscription recognition system according to a first embodiment of the present invention.
2 is a block diagram showing a deep learning based inscription recognition system according to a second embodiment of the present invention.
3 is a flowchart illustrating a deep learning based inscription recognition method according to a first embodiment of the present invention.
4 is a flowchart illustrating a deep learning based inscription recognition method according to a second embodiment of the present invention.
5 is a flowchart illustrating an inscription feature point image extraction step S300 according to an embodiment of the present invention.
Figure 6 is a flow chart showing a detailed species determination step (S400) according to an embodiment of the present invention.
7 is a view showing a process of extracting the inscription feature point of the dog (or dog) as an embodiment of the present invention.
8 is a diagram showing Gabor filtering image results according to the thickness (8-15 [pixel]) of an inscription line of a dog (or dog) as an embodiment of the present invention.
9 is an embodiment of the present invention in the parallel direction (

Figure shows the Gabor filtered image results according to).
10 is a view showing an inscription matching process of a dog (or dog) as an embodiment of the present invention.
11 is a graph showing a score comparison for five puppies (or dogs) as an embodiment of the invention.
FIG. 12 is a diagram comparing shift, surf and orb matching as an embodiment of the present invention. FIG.
13 is a graph showing the accuracy and time results according to the matching method as an embodiment of the present invention.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects achieved by the practice of the present invention, reference should be made to the accompanying drawings which illustrate preferred embodiments of the present invention and the contents described in the accompanying drawings. The features and advantages of the present invention will become more apparent from the following detailed description based on the accompanying drawings. Prior to this, terms or words used in the present specification and claims are defined in the technical spirit of the present invention on the basis of the principle that the inventor can appropriately define the concept of terms in order to explain in the best way of their own invention. It should be interpreted to mean meanings and concepts. In addition, it is to be noted that the detailed description of the known functions and configurations related to the present invention has been omitted when it is determined that the gist of the present invention may be unnecessarily obscured.

1 is a block diagram showing a deep learning based inscription recognition system according to a first embodiment of the present invention, Figure 2 is a block diagram showing a deep learning based inscription recognition system according to a second embodiment of the present invention.

Referring to FIGS. 1 and 2, the deep learning based inscription recognition system outputs an image input unit 100, a species determination unit 200, an inscription feature point image extractor 300, a detailed species determination unit 400, and detailed species result output. The unit 500 may include a first database unit D1 and a second database unit D2.

The image input unit 100 may input a face image of an animal belonging to a canine. This may be referred to as a component for inputting an image to be determined in order to determine the species or detailed species of the animal belonging to the canine. In this case, animals generally belonging to the canine may include a dog, a dog, and the like.

The species determination unit 200 may determine the species of the animal belonging to the dog family by comparing the face image input from the image input unit 100 with the image stored in the second database unit D2. This can be said to be a component that primarily selects candidate groups before determining the detailed species of animals belonging to the canine. In this case, the "species" in the species determination unit 200 may include color information of the animal, and when the animal is called a puppy (or dog), the breed may be included. The species determination unit 200 may be easily changed by the user according to the quantity, characteristics, and the like of the candidate group to be primarily selected. As shown in FIG. 1, the species determination unit 200 determines only the detailed species without primary species determination according to the user's deep learning environment construction, or secondarily after determining the species primarily as shown in FIG. 2. Detailed species can be determined.

Meanwhile, the second database unit D2 may be a storage medium including a plurality of breed images collected with standardized data that can determine the species of the animal. For example, when an animal to be judged as a dog is a dog (or dog), a dog image such as 100 Siberian husky species images and 100 Caucasuselian images may be included.

The inscription feature point image extractor 300 may extract the inscription feature point image from the face image input from the image input unit 100. At this time, the inscription may be the nose pattern of each animal belonging to the canine. This can be said to be an element for extracting the image of the inscription feature point by optimizing the image according to the inscription characteristic of each animal belonging to the canine. To this end, the inscription feature point image extractor 300 may include a preprocessor 310, a filter image output unit 330, and a feature point image output unit 350.

The preprocessor 310 may set an image corresponding to a region of interest (ROI) among the face images input from the image input unit 100 and then preprocess it. This is for extracting data corresponding to the region of interest from the face image, and it can be said to be a component that can increase recognition speed and accuracy by extracting only 'nose' which is a necessary area among various data such as eyes, nose and mouth. To this end, the preprocessor 310 may apply an adaptive thresholding algorithm to the face image input from the image input unit 100 to set an image corresponding to the ROI. The reason is that the adaptive threshold algorithm can effectively extract the image of the network form from the inscription of the animal belonging to the canine. The nose pattern of an animal belonging to the canine can be divided into a protruding part and a recessed part, and when the recessed part is connected, a net-shaped image can be obtained.

The filter image output unit 330 may output the filter image from which the pattern value is extracted by filtering the ROI image preprocessed from the preprocessor 310. This can be said to be an element that can implement the optimized image processing according to the thickness of the inscription line of the animal belonging to the canine. To this end, the filter image output unit 330 may apply a Gabor filter to the ROI image preprocessed from the preprocessor 310 to output the filter image. The reason is that the Gabor filter has a wave and can output continuous data of the thickness and direction desired by the user through the directionality. For this reason, the Gabor filter is a filter for optimizing the dog's nose while changing parameters, and can effectively extract the inscription line of the animal belonging to the canine.

The feature point image output unit 350 may output the inscription feature point image by extracting the feature point from the filter image output from the filter image output unit 330. This can be said to be a component for extracting inscription features according to the inscription characteristics of each animal belonging to the canine. To this end, the feature point image output unit 350 may apply a shift algorithm to the filter image output from the filter image output unit 330 to extract the feature points. The reason is that the shift algorithm can extract more stable feature points from the inscriptions of the animals belonging to the canine. In this case, the extracted feature point may be a feature point more stably extracted, including at least one inscription feature point location, scale, and direction.

The detailed species determination unit 400 may determine the detailed species of the animal belonging to the canine by comparing the feature point extracted from the feature point image output unit 350 with a plurality of images stored in the first database unit D1. This can be said to be a component for judging the detailed species by using the characteristic that the inscription of the animal belonging to the canine is different for each animal. To this end, the detailed seed determiner 400 may include an image comparator 410 and a matching feature point selector 430.

Meanwhile, the first database unit D1 may be a storage medium including a plurality of variation images collected with standardized data for determining the detailed species of the animal. The reason is that even animals of the same species may have different characteristics such as various sizes and colors. For example, when an animal belonging to a canine is a puppy (or a dog), the second database unit D2 may include data for determining a poodle, a siberian husky, a coca spaniel, and the like, which are dog species. In the first database unit D1, when it is determined that the puppy is a 'Poodle' species, data for determining a detailed species such as who owns the 'Poodle' or the name of the 'Poodle' is included. Can be.

The image comparison unit 410 calculates and compares Euclidean distance values L2-Norm between the feature points extracted from the feature point image output unit 350 and the feature points of the plurality of images stored in the first database unit D1, respectively. Can be. This may be referred to as a component for calculating an objective numerical value for a descriptor comparison between a descriptor vector of the extracted feature point and a feature point of a plurality of images stored in the first database unit D1. In this case, when the number of extracted feature points is N and the number of feature points of the plurality of images stored in the first database unit D1 is M, a total of N X M times can be compared.

The matching feature point selector 430 may select the feature point of the image having the smallest Euclidean distance value calculated from the image comparator 410 among the feature points of the plurality of images stored in the first database unit D1 as the matching feature point. This can be said to be a component using the feature of the Euclidean distance value that the Euclidean distance value of the comparative objects having a similar pattern is small.

Meanwhile, in addition to the selected matching feature point, the matching feature point selector 430 may further select the matching feature points in order of decreasing Euclidean distance value calculated from the image comparator 410, and then determine the matching reliability of the selected matching feature points. have. In this case, the matching reliability determination may use a matching determination equation defined by the following equation.

Equation

E2 / E1> (Threshold)

(E1: selected matching feature point, E2: additionally selected matching feature point)

A method of recognizing an inscription based on deep learning by the recognition system illustrated in FIGS. 1 and 2 may be described with reference to FIGS. 3 to 6. 3 is a flowchart illustrating a deep learning based inscription recognition method according to a first embodiment of the present invention, FIG. 4 is a flowchart illustrating a deep learning based inscription recognition method according to a second embodiment of the present invention, and FIG. FIG. 6 is a flowchart illustrating an inscription feature point image extraction step S300 according to an exemplary embodiment of the present invention, and FIG. 6 is a flowchart illustrating the detailed species determination step S400 according to an exemplary embodiment of the present invention.

3 and 4, the deep learning-based inscription recognition method includes an image input step S100, a species determination step S200, an inscription feature point image extraction step S300, a detailed species determination step S400, and a detailed species result. It may include an output step (S500).

First, the image input unit 100 may input the face image of the animal belonging to the canine (S100). This may be referred to as a step for inputting an image to be determined in order to determine the species or detailed species of the animal belonging to the canine. In this case, animals generally belonging to the canine may include a dog, a dog, and the like.

The species determination unit 200 may determine the species of the animal belonging to the dog family by comparing the face image input from S100 with the image stored in the second database unit D2 (S200). This can be said to be a step for initially selecting candidate groups before determining the detailed species of animals belonging to the canine. In this case, the “species” in S200 may include animal color information, and may include a breed when the animal is called a puppy (or dog). S200 may be easily changed by the user according to the amount, characteristics, and the like of the candidate group to be primarily selected. As shown in FIG. 3, the S200 determines only the detailed species without primary species determination according to the user's deep learning environment construction, or the secondary species after the primary species determination as shown in FIG. 4. Can be.

Meanwhile, the second database unit D2 may be a storage medium including a plurality of breed images collected with standardized data that can determine the species of the animal. For example, when an animal to be judged as a dog is a dog (or dog), a dog image such as 100 Siberian husky species images and 100 Caucasuselian images may be included.

The inscription feature point image extractor 300 may extract the inscription feature point image from the face image input from S100 (S300). At this time, the inscription may be the nose pattern of each animal belonging to the canine. This is a step for extracting the image of the inscription feature point by optimizing the image according to the inscription characteristic of each animal belonging to the canine.

As shown in FIG. 5, in S300, a preprocessing step S310, a filter image output step S330, and a feature point image output step S350 may be included.

The preprocessor 310 may set an image corresponding to a region of interest (ROI) among the face images input from S100 and then preprocess it (S310). This is to extract the data corresponding to the region of interest from the face image, and it is a step to increase the recognition speed and accuracy by extracting only the 'nose' which is a necessary area among various data such as eyes, nose and mouth. To this end, in S310, an adaptive thresholding algorithm may be applied to the face image input from S100 to set an image corresponding to the ROI. The reason is that the adaptive threshold algorithm can effectively extract the image of the network form from the inscription of the animal belonging to the canine. The nose pattern of an animal belonging to the canine can be divided into a protruding part and a recessed part, and when the recessed part is connected, a net-shaped image can be obtained.

The filter image output unit 330 may output a filter image from which a pattern value is extracted by filtering the ROI image preprocessed from S310 (S330). This can be said to be a step to implement the optimized image processing according to the thickness of the inscription line of the animal belonging to the canine. To this end, in S330, a Gabor filter may be applied to the ROI image preprocessed from S310 to output the filter image. The reason is that the Gabor filter has a wave and can output continuous data of the thickness and direction desired by the user through the directionality. For this reason, the Gabor filter is a filter for optimizing the dog's nose while changing parameters, and can effectively extract the inscription line of the animal belonging to the canine.

The feature point image output unit 350 may output the inscription feature point image by extracting the feature point from the filter image output from S330 (S350). This is a step for extracting the inscription feature points according to the inscription characteristics of each animal belonging to the canine. To this end, in S350, a shift algorithm may be applied to the filter image output from S330 for feature point extraction. The reason is that the shift algorithm can extract more stable feature points from the inscriptions of the animals belonging to the canine. In this case, the feature point image may be a feature point that is more stably extracted, including at least one inscription feature point location, scale, and direction.

The detailed species determination unit 400 may determine the detailed species of the animal belonging to the canine by comparing the feature point extracted from S350 and the plurality of images stored in the first database unit D1 (S400). This is a step for judging the detailed species by using the characteristic that the inscription of the animal belonging to the canine is different for each animal.

For this purpose, as shown in FIG. 6, in S400, an image comparison step S410 and a matching feature point selection step S430 may be included.

Meanwhile, the first database unit D1 may be a storage medium including a plurality of variation images collected with standardized data for determining the detailed species of the animal. The reason is that even animals of the same species may have different characteristics such as various sizes and colors. For example, when an animal belonging to a canine is a puppy (or a dog), the second database unit D2 may include data for determining a poodle, a siberian husky, a coca spaniel, and the like, which are dog species. In the first database unit D1, when it is determined that the puppy is a 'Poodle' species, data for determining a detailed species such as who owns the 'Poodle' or the name of the 'Poodle' is included. Can be.

The image comparison unit 410 may calculate and compare Euclidean distance values L2-Norm between the feature points extracted from S350 and the feature points of the plurality of images stored in the first database unit D1 (S350). This may be referred to as a step for calculating an objective numerical value for comparing the descriptors of the extracted feature points with the descriptors of the plurality of images stored in the first database unit D1. In this case, when the number of extracted feature points is N and the number of feature points of the plurality of images stored in the first database unit D1 is M, a total of N X M times can be compared.

The matching feature point selecting unit 430 may select the feature point of the image having the smallest Euclidean distance value calculated from S410 among the feature points of the plurality of images stored in the first database unit D1 as the matching feature point (S430). This is a step using the feature of the Euclidean distance value that the Euclidean distance value of the comparative objects showing a similar pattern is small.

Meanwhile, in S430, in addition to the selected matching feature point, the matching feature point may be further selected in order of decreasing Euclidean distance value calculated from S410, and then the matching reliability of the selected matching feature point may be determined. In this case, the matching reliability determination may use a matching determination equation defined by the following equation.

Equation

E2 / E1> (Threshold)

(E1: selected matching feature point, E2: additionally selected matching feature point)

)에 따른 가보 필터링 이미지 결과를 나타낸 도면이고, 도 10은 본 발명의 실시예로서 강아지(또는 개)의 비문 매칭 과정을 나타낸 도면이다.7 is a diagram illustrating a process of extracting an inscription feature point of a dog (or dog) as an embodiment of the present invention, and FIG. 8 is a thickness (8 to 15 [pixel] of an inscription line of a dog (or dog) as an embodiment of the present invention. FIG. 9 is a diagram illustrating Gabor filtering image results according to an embodiment of the present invention, and FIG. 9 is an embodiment of the present invention.

FIG. 10 is a diagram illustrating a Gabor filtering image result, and FIG. 10 illustrates an inscription matching process of a dog (or dog) as an embodiment of the present invention.

Referring to FIG. 7, it may be assumed that a dog's nose image is input as an input value in a user interface. In this case, the user interface may include various media that a user wants to receive, such as web or mobile.

Figure 7 (a) is a dog's nose image. At this time, the size of the dog nose image in Figure 7 (a) is 400X400 [pixel], which can be input in various sizes depending on the resolution of the nose image.

FIG. 7B is a region of interest (ROI) image of the dog nose image received from FIG. 7A. As shown in FIG. 7 (a), if the feature is extracted from the dog's nose, the feature is drawn from the line at the bottom of the nose across the nostrils and the pattern located between the nostrils and the top of the nose. Can be. Based on this, assuming that the nose is input in the form of a straight circle, the height of the region of interest in FIG. 7 (b) is 1/8 to 7/8 of the height of the nose image, and the region of interest is 1/4 to the width of the nose image. Can be set to 3/4 rectangle. However, the height and width values of the ROI may be easily changed and set according to the size or nose shape of the received nose image. On the other hand, as shown in Figure 7 (b), the nose pattern of the animal belonging to the canine can be divided into a protruding portion and a recessed portion, wherein the overall pattern made of the protruding portion and the recessed portion is called 'secret' One specific pattern may be referred to as a 'feature point'.

FIG. 7C is an image obtained by preprocessing the ROI image set from FIG. 7B. As illustrated in FIGS. 7B and 7C, an adaptive threshold algorithm may be used to effectively extract a network-shaped image from a region of interest of a set dog nose image. The adaptive threshold algorithm can binarize the entire global image without changing the threshold using the average value or Gaussian filtering of the surrounding area, instead of applying the threshold to the entire area. . In this case, the Gaussian method is used in FIG. 7B, and the size of the peripheral area may be 4 × 4 [pixel].

FIG. 7D is an image obtained by filtering the preprocessed image from FIG. 7C. As shown in FIG. 7D, a Gabor filter may be applied to the preprocessed image. Gabor filters are linear filters used for edge detection, texture analysis, and feature extraction in image processing. In other words, it is a filter that analyzes whether a specific frequency component is localized in a specific direction in an image. You can use the features of the periodic function to find out where the edges or textures change. In the two-dimensional space domain, the 2D gabor filter is a Gaussian filter modulated with a sine function.

The formula for the Gabor filter is as follows:

<Equation 1>

는 가보 필터의 표준편차이다.

는 필터의 평행선의 방향이다.

는 함수의 파동 크기이다.

는 평행선의 종횡비이다.

는 중심으로부터 이동한 거리이다. 여기서 강아지 코의 선들을 잘 추출하기 위한 파라미터는

와

이다.Where x and y are the size of the heirloom filter,

Is the standard deviation of the Gabor filter.

Is the direction of the parallel lines of the filter.

Is the wave size of the function.

Is the aspect ratio of the parallel lines.

Is the distance traveled from the center. Here, the parameter to extract the lines of the dog's nose

Wow

to be.

은 파라미터가 변함에 따라, 주기함수의 파동 크기가 변하게 된다. 즉 선형적으로 필터링할 수 있는 굵기에 영향을 끼친다. 도 8에 도시된 바와 같이, 강아지 코에 대해서는 굵기를 실험적으로 8~15[pixel]까지의 굵기가 적당하였다.

As the parameter changes, the wave size of the periodic function changes. This affects the thickness that can be filtered linearly. As shown in FIG. 8, the thickness of the dog's nose was experimentally 8 to 15 [pixel] thick.

은 필터의 평행선 방향을 나타내는데, 도 9에 도시된 바와 같이 보다 정밀한 측정을 위해

/60의 간격으로 60개의 방향을 나누었다.

Indicates the parallel direction of the filter, for more precise measurements as shown in FIG.

The 60 directions were divided at intervals of / 60.

FIG. 7E is an image in which feature points are extracted from the filtered image from FIG. 7D. As shown in FIG. 7E, the filtered image may apply a shift algorithm. This is because the shift algorithm is a feature-based recognition algorithm that extracts features that are invariant to rotation.

The method of extracting feature points by the shift algorithm may be performed in four steps as follows.

1. Scale-space extrema detection step: When extracting feature points to make objects invariant, a Gaussian blur image is reduced and scaled up to create a Gaussian pyramid. Then, the DOG (different of Gaussian), which is the difference between Gaussian pyramid images, is found to find the extreme point. Values extracted as feature points across multiple scales can be said to be invariant in scale.

2. Keypoint localization step: To make the rotation invariant feature additional, the algorithm normalizes the direction and limits the extreme to the keypoint. This step is to find a keypoint that can interpolate to the secondary development of the Taylor series to clearly indicate the feature point.

3. Orientation assignment step: This step is to find the direction of the keypoint specified in the previous step (Keypoint localization) in order to detect the feature point even in the rotation change.

4. Keypoint descriptor step: It is a step to define a descriptor for each feature point. The descriptor is the directions of the gradient values around the feature point. If the descriptor is applied to each feature point using this, as shown in FIG. 7E, the position, scale, direction, etc. of the feature point can be known. It may have a feature that can extract a more stable feature point.

FIG. 10 (a) illustrates an inscription matching process for another puppy (or dog) using a shift matching method, and FIG. 10 (b) illustrates another puppy (or shift) using a shift matching method. A diagram showing the inscription matching process for dog).

Referring to FIGS. 10A and 10B, the Euclidean distance value between each key between the image (input image) from which the feature point is extracted and the image of the database is calculated by the number of images in the database. can do. The image of the database corresponding to the minimum value E1 among the calculated Euclidean distance values may be selected as a matching key matching the extracted image.

On the other hand, two values obtained through a ratio of the minimum value (E1) and the second closest value can be accurately matched. This further selects an image of a database corresponding to a value E2 having a smaller Euclidean distance value next to the minimum value E1, and then uses this to determine the matching reliability of the minimum value E1. In this case, the matching reliability determination may use a matching determination equation defined by the following equation.

Equation

E2 / E1> (Threshold)

(E1: minimum value (selected matching feature point), E2: additionally selected value (additionally selected matching feature point))

FIG. 11 is a graph showing a score comparison for five puppies (or dogs) as an embodiment of the present invention, and FIG. 12 is a shift, surf and orb matching as an embodiment of the present invention. FIG. 13 is a graph showing accuracy and time results according to a matching method as an embodiment of the present invention.

Table 1 is a table showing the score comparison for five dogs, Table 2 is a table showing the accuracy and time results according to the matching method.

Referring to FIG. 11 and Table 1, the present invention used Python 3.6.4 on a computer with a CPU i5 2.2 Ghz to compare the scores for five dogs as an example, cross the nose image of five dogs The score values were compared by matching. Here, the score value is the number of key points of the shift matching, and it can be confirmed that the puppy having the highest value as shown in Table 1 is the puppy.

Dog1 Dog2 Dog3 Dog4 Dog5 Dog1 44.5 9.17 10.2 8.67 5.16 Dog2 9.33 58.7 9.33 8 5.27 Dog3 7.5 8 92.1 6.83 5.33 Dog4 6.5 7.8 7.1 42.5 6.67 Dog5 6.8 5.33 8 5.5 75.8

12, 13 and Table 2, according to an embodiment of the present invention, the Surf and Orb methods are used together with the Shift matching method to use the Real time matching method. Can be used to measure accuracy and time.

12 (a) and 12 (b) show shift matching, FIGS. 12 (c) and 12 (d) show Surf matching, and FIGS. 12 (e) and 12 (f). ) Is a result of matching using Orb matching. 12 (a), 12 (c) and 12 (e) show nose pictures of the same dog, and FIGS. 12 (b), 12 (d) and 12 (f) show nose pictures of other dogs. The result is a comparison.

Here, Matching Rate (MR) refers to an average of image scores of a plurality of nose images of the same dog as Same Image Score (SIS), and an average of image scores of a plurality of nose images of different dogs is referred to as Different Image Score. It can be called (DIS). The matching rate MR may be defined as SIS / (DIS + SIS) * 100.

As a result of calculating the matching rate according to the definition of the matching rate (MR), as shown in FIG. 13 and Table 2 below, it was found that the shift matching method has the highest matching rate (MR) in accuracy and time.

Time /
per Time
(s) DIS
(score) SIS
(score) MR
(%) SIFT 2.18 157 5.2 25.8 83.22 SURF 1.81 131 29.1 37.2 56.10 ORB 1.76 127 297 265 47.15

On the other hand, the embodiment of the present invention shown in Figure 7 to 13 used the ImageNet database in the pre-training process, and additionally used the image for the animal species as fine-tuning, which varies depending on the recognition system environment and user settings Can be changed.

In addition, the deep learning method of the present invention may use a variety of methods such as alexNet, VGGNet, GoogleNet, and resNet using a deep neural architecture for finding the kind of an object included in an image.

Although described and illustrated in connection with a preferred embodiment for illustrating the technical spirit of the present invention above, the present invention is not limited to the configuration and operation as shown and described as such, it is a deviation from the scope of the technical idea It will be understood by those skilled in the art that many changes and modifications can be made to the invention without departing from the scope of the invention. Accordingly, all such suitable changes and modifications should be considered to be within the scope of the present invention.

Image input unit: 100 species determination unit: 200
Inscription Feature Point Image Extraction Unit: 300 Preprocessor: 310
Filter image output unit: 330 Feature point image output unit: 350
Species species judgment unit: 400 image comparison unit: 410
Matching feature point selector: 430 Detailed result output: 500
1st database part: D1 2nd database part: D2

Claims (22)

An image input unit for inputting an image of a face of an animal belonging to a dog family;
An inscription feature point image extracting unit for extracting an inscription feature point image from the input face image;
A detailed species determination unit that determines the detailed species of the animal by comparing the extracted inscription feature point image with a plurality of images stored in a first database unit; And
Deep learning-based inscription recognition system comprising a; detailed species result output unit for outputting the determined detailed species result. The method of claim 1, wherein the inscription feature point image extraction unit,
A preprocessor configured to set an image corresponding to a region of interest (ROI) among the input face images and then preprocess it;
A filter image output unit which filters the preprocessed ROI image and outputs a filter image from which a pattern value is extracted; And
And a feature point image output unit configured to extract a feature point from the output filter image and output an inscription feature point image. The method of claim 2, wherein the extracted feature point,
A deep learning based inscription recognition system comprising at least one inscription feature point location, scale and direction. The method of claim 2, wherein the pretreatment unit,
And a deep learning based inscription recognition system applying an adaptive thresholding algorithm to the input face image for setting the ROI image. The method of claim 2, wherein the filter image output unit,
A deep learning based inscription recognition system applying a Gabor filter to the preprocessed ROI image for outputting the filter image. The method of claim 2, wherein the feature point image output unit,
A deep learning based inscription recognition system applying a shift algorithm to the output filter image for the feature point extraction. The method of claim 3, wherein the detailed species determination unit,
An image comparison unit configured to calculate and compare Euclidean distance values (L2-Norm) between the extracted feature points and the feature points of the plurality of images stored in the first database unit; And
And a matching feature point selector for selecting a feature point of the image having the smallest Euclidean distance value among the feature points of the plurality of images stored in the first database unit as a matching feature point. The method of claim 7, wherein the matching feature point selection unit,
After further selecting the matching feature points in order of the calculated Euclidean distance value in addition to the selected matching feature points, deep learning for determining matching reliability of the selected matching feature points using a matching decision equation defined by the following equation: Based inscription recognition system.
Equation
E2 / E1> (Threshold)
(E1: selected matching feature point, E2: additionally selected matching feature point) The method of claim 1,
And a species determination unit for determining the species of the animal by comparing the face image input from the image input unit with the image stored in the second database unit. The method of claim 1, wherein the first database unit,
Deep learning based inscription recognition system including a plurality of variation (variation) images collected with standardized data that can determine the detailed species of the animal. The method of claim 9, wherein the second database unit,
Deep learning based inscription recognition system including a plurality of images (Breed) collected with standardized data that can determine the species of the animal. An image input step of receiving an image of a face of an animal belonging to a canine at an image input unit;
An inscription feature point image extraction step of extracting an inscription feature point image from the input face image by an inscription feature point image extractor;
A detailed species determination step of determining a detailed species of the animal by comparing the extracted inscription feature point image with a plurality of images stored in a first database unit in a detailed species determination unit; And
And a detailed seed result output step of outputting the determined detailed seed result in a detailed seed result output unit. The method of claim 12, wherein the extracting the inscription feature point image comprises:
A pre-processing step of setting an image corresponding to a region of interest (ROI) among the input face images by a pre-processing unit and pre-processing the image;
A filter image output step of filtering the preprocessed ROI image by a filter image output unit to output a filter image from which a pattern value is extracted; And
And a feature point image output step of outputting an inscription feature point image by extracting the feature point from the output filter image in the feature point image output unit. The method of claim 13, wherein the extracted feature point,
A deep learning based inscription recognition method comprising at least one inscription feature point location, scale and direction. The method of claim 13, wherein the pretreatment step,
A deep learning-based inscription recognition method applying an adaptive thresholding algorithm to the input face image to set the ROI image. The method of claim 13, wherein the filter image output step,
A deep learning based inscription recognition method applying a Gabor filter to the preprocessed ROI image for outputting the filter image. The method of claim 13, wherein the outputting the feature point image comprises:
Deep learning based inscription recognition method for applying a shift algorithm to the output filter image for the extraction of the feature point. The method of claim 12, wherein the detailed species determination step,
An image comparison step of calculating and comparing Euclidean distance values (L2-Norm) between the extracted feature points in the image comparison unit and the feature points of the plurality of images stored in the first database unit, respectively; And
A matching feature point selection step of selecting a feature point of an image having the smallest Euclidean distance value among the feature points of a plurality of images stored in the first database unit as a matching feature point in a matching feature point selection unit; . The method of claim 18, wherein the matching feature point selection step,
After further selecting the matching feature points in order of decreasing calculated Euclidean distance value in addition to the selected matching feature points, deep learning for determining matching reliability of the selected matching feature points using a matching decision equation defined by the following equation: Based inscription recognition method.
Equation
E2 / E1> (Threshold)
(E1: selected matching feature point, E2: additionally selected matching feature point) The method of claim 12,
And a species determination step of determining a species of the animal by comparing a face image input from the image input step by a species determination unit with an image stored in a second database unit. The method of claim 12, wherein the first database unit,
Deep learning based inscription recognition method comprising a plurality of variations (variation) images collected with standardized data that can determine the detailed species of the animal. The method of claim 20, wherein the second database unit,
Deep learning based inscription recognition method comprising a plurality of species (Breed) images collected with standardized data that can determine the species of the animal.

Images