KR20220087310A - Image based method and device for identifying dog - Google Patents

Abstract

The disclosed technology relates to an image-based dog entity identification method and apparatus, comprising: detecting, by an entity identification device, an image of a dog dog into a deep learning model to detect the dog's eyes and nose; checking, by the object identification device, whether the direction of the image is the front according to the positions and sizes of the eyes and nose; determining, by the entity identification device, extraction of an inscription region from the image according to the noise included in the image; performing pre-processing by adjusting the size of the inscription region and normalizing the histogram by the entity identification device; extracting, by the entity identification device, feature information from the pre-processed inscription area using a feature point extraction algorithm; and comparing, by the entity identification device, a plurality of pieces of characteristic information included in a plurality of pre-registered pieces of entity information with the characteristic information. Therefore, it is possible to accurately identify the dog and has the effect of easily registering individual information.

Description

Image-based dog entity identification method and device {IMAGE BASED METHOD AND DEVICE FOR IDENTIFYING DOG}

The disclosed technology relates to a method and apparatus for identifying an object by recognizing an inscription of a dog using an image.

As social problems such as deliberate abandonment or accidental loss of dogs have also emerged, technology for registering dog information in a database has recently been used.

The currently used dog registration method uses an external attachment device to wear a device that can recognize an ID on the dog's body or a method to recognize each individual by inserting a small chip into the dog's body. However, in the method of using an external device, there is a possibility of losing the device, and there is a inconvenience of having to wear the device whenever going out. And the method of using the built-in device is a situation in which the dog owner has a sense of rejection, so a new type of recognition technology is required.

On the other hand, there are organs having unique biometric information in animals, such as human iris or fingerprints. For example, an inscription pattern formed on a dog's nose indicates a unique characteristic of each individual, so it is possible to use it as information for recognizing an individual. However, unlike humans, there are cases where dogs refuse to look at the camera or are distracted, which causes difficulties in taking accurate images of inscriptions.

Korean Patent Registration No. 10-1527801

The disclosed technology is to provide a method and apparatus for recognizing a dog's inscription using an image to identify an object.

A first aspect of the disclosed technology to achieve the above technical task is the steps of, by an entity identification device, inputting an image of a dog into a deep learning model to detect the eyes and nose of the dog, wherein the entity identification device uses each of the eyes and nose determining whether the direction of the image is the front according to a location and size; determining, by the object identification device, to extract an inscription region from the image according to noise included in the image; performing pre-processing by adjusting the size and normalizing the histogram; extracting, by the entity identification device, characteristic information from the pre-processed inscription area using a key point extraction algorithm; and a plurality of entity information previously registered by the entity identification device An object of the present invention is to provide a method for identifying a dog individual comprising the step of comparing the characteristic information with a plurality of characteristic information included in the .

A second aspect of the disclosed technology to achieve the above technical task is a camera for photographing a face image of a dog, a memory for storing a plurality of individual information about the dogs, and inputting the facial image of the dog into a deep learning model to input the dog's face image into the deep learning model A processor that detects the eyes and nose of , extracts an inscription region from the face image, inputs the inscription region into a feature point extraction algorithm to extract feature information, and compares the extracted feature information with feature information included in the entity information To provide a dog entity identification device comprising a.

Embodiments of the disclosed technology may have effects including the following advantages. However, since it does not mean that the embodiments of the disclosed technology should include all of them, it should not be understood that the scope of the disclosed technology is limited thereby.

According to an embodiment of the disclosed technology, an image-based dog entity identification method and apparatus have an effect of identifying an entity using characteristic information extracted from a dog's inscription.

In addition, it has the effect of preventing rejection by the dog or dog owner by identifying the object in a non-contact manner.

In addition, the dog's individual information can be easily registered, which is effective in preventing dog abandonment and loss.

1 is a diagram illustrating a process of using an image-based dog entity identification system according to an embodiment of the disclosed technology.
2 is a flowchart of a dog entity identification method according to an embodiment of the disclosed technology.
3 is a block diagram of an apparatus for identifying a dog entity according to an embodiment of the disclosed technology.
4 is a diagram illustrating extraction of an inscription region from a face image of a dog.
5 is a diagram illustrating division of an inscription area into a plurality of areas.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

Terms such as first, second, A, and B may be used to describe various components, but the components are not limited by the above terms, and only for the purpose of distinguishing one component from other components. is used only as For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

In terms of terms used herein, the singular expression is to be understood as including the plural expression unless the context clearly dictates otherwise. And terms such as "comprising" mean that the specified feature, number, step, operation, component, part, or a combination thereof exists, but one or more other features or number, step operation component, part It should be understood as not excluding the possibility of the presence or addition of or combinations thereof.

Prior to a detailed description of the drawings, it is intended to clarify that the classification of the constituent parts in the present specification is merely a division according to the main function each constituent unit is responsible for. That is, two or more components to be described below may be combined into one component, or one component may be divided into two or more for each more subdivided function.

In addition, each of the constituent units to be described below may additionally perform some or all of the functions of other constituent units in addition to the main function it is responsible for. Of course, it may be carried out by being dedicated to it. Accordingly, the existence or non-existence of each component described through the present specification should be interpreted functionally.

1 is a diagram illustrating a process of using an image-based dog entity identification system according to an embodiment of the disclosed technology. Referring to FIG. 1 , the dog entity identification system 100 includes an entity identification device 110 and a user terminal 120 .

The entity identification device 110 identifies the entity by using the characteristic information included in the dog's inscription. First, in order to obtain the dog's inscription, you can drive the camera to take an image containing the dog's inscription. Images with inscriptions may include the entire body of the dog or only the face. The object identification apparatus 110 may acquire an image including an inscription by using a camera. For example, an image may be acquired by directly controlling a camera mounted on the object identification device or using a separate camera. A camera used for taking an image may use a general optical camera, and if necessary, may take an image using a plurality of cameras.

On the other hand, the object identification device 110 may compare the photographed image of the dog with object information about a plurality of dogs stored in the database. Entity information may include an image of the inscription of the dog, characteristic information extracted from the image of the inscription, and personal information such as the name, address, and contact information of the owner. The entity identification device 110 may search whether entity information including an inscription image matching the inscription image of the dog exists among a plurality of entity information stored in the database. The object identification device 110 may increase accuracy by comparing the size of the inscription image included in the object information to be compared with the size of the inscription image of the dog to the same size. Here, adjusting the size of the image means adjusting the size while maintaining the ratio of the original image.

Also, the object identification apparatus 110 may check whether the direction of the image is the front in order to extract the inscription area. The object identification device 110 includes a deep learning model trained in advance to read the image direction, and using this, it is possible to check whether the image direction is the front. The object identification device 110 may determine whether the image is for the front of the face by identifying the positions of the dog's eyes and nose included in the image using the deep learning model. For example, it is possible to identify the dog's eyes and nose, and determine whether the two eyes are the same size and position or whether the nose position is located below the two eyes to determine whether the dog is in front.

Meanwhile, the object identification apparatus 110 may determine extraction of the inscription region according to noise included in the image. Here, the noise includes the light reflectance included in the image and the sharpness of the image. For example, if the light reflectivity included in the image is too high, it may be determined that the image is not suitable for extraction of the inscription area and may be dropped or a new image may be requested. Of course, even if the sharpness of the image is so low that it is difficult to discern the pattern contained in the inscription, we can request that a new image be sent again instead of extracting the inscription area.

The object identification device 110 may extract the inscription region by determining whether the noise is small enough not to affect the extraction of the inscription region as described above, and obtain a more accurate result value when comparing the feature information included in the extracted inscription region. The size of the inscription area can be adjusted to the same size as the inscription image included in the object information, while maintaining the original ratio. For example, if the original inscription area has a width and height of 500x600, the width can be set to an arbitrary fixed value and the height can be adjusted according to the ratio of the original image. For example, if the fixed width value is 300, the height can be adjusted to 360.

Processing the inscription area in this way is because, even if the inscription area of the same object, the size of the inscription area may be included differently due to the direction and distance in the process of image collection, unintentional data loss may occur during the preprocessing process. . Accordingly, it is possible to adjust the size of the inscription area while maintaining the original ratio so as to avoid data loss as much as possible. And by normalizing the histogram, the overall contrast of the image can be processed evenly.

Meanwhile, the entity identification apparatus 110 may extract characteristic information from the inscription area performed in the pre-processing process and compare characteristic information included in a plurality of entity information. Extraction of feature information is performed through a feature point extraction algorithm. As the keypoint extraction algorithm, various conventional keypoint extraction algorithms may be applied. For example, feature points in an image, such as a shift (SIFT) algorithm, a speed-up Bobust Feature (SURF) algorithm, a Brisk (Binary Robust Invariant Scalable Keypoints, BRISK) algorithm, and an Oriented FAST and Rotated BRIEF (ORB) algorithm. Various algorithms for extracting . The characteristic information included in the entity information is extracted from the images of the inscriptions of previously registered dogs, and it can be checked whether the characteristic information matches whether the dog is a previously registered entity or not. The object identification device 110 may output feature information according to the inscription of the dog by using the inscription area as an input value to the feature point extraction algorithm.

On the other hand, it is possible to compare with the characteristic information included in the individual information by excluding some values indicating abnormal values among the characteristic information output through the characteristic point extraction algorithm. For example, among the characteristic information of the dog output through the characteristic point extraction algorithm, the remaining characteristic information excluding outliers and duplicate values may be compared with the characteristic information included in the plurality of entity information.

As described above, the object identification apparatus 110 compares the post-processed characteristic information with a plurality of previously registered characteristic information. Extraction and comparison of feature information may be performed using various conventional algorithms. The entity identification device 110 may compare the characteristic information of the dog with a plurality of registered characteristic information, or may divide the inscription area of the dog into a plurality and use it to compare the extracted characteristic information. For example, the inscription area is divided into a plurality of areas to extract a plurality of characteristic information for each area, and a plurality of pieces of characteristic information included in the plurality of individual information are divided into the same number of areas as the inscription area, and the degree of similarity for each divided area can be compared respectively. If the number of areas with similarity exceeding the standard is more than a certain number, the dog is judged as a registered entity, and if the number of areas with similarity exceeding the standard is less than a certain number, the dog's image and characteristic information can be saved as new object information. .

Meanwhile, the result of comparing the characteristic information is transmitted to the user terminal 120 . The entity identification device 110 may transmit the entity information of the dog to the user terminal 120 if the characteristic information of the dog is included in the plurality of entity information. In addition, if the characteristic information of the dog is not included in the plurality of individual information, information for registering the individual information of the dog may be transmitted to the user terminal. In the case of transmitting the dog's individual information, the dog's name, image, and information about the owner may be transmitted. You can request information.

2 is a flowchart of a dog entity identification method according to an embodiment of the disclosed technology. Referring to FIG. 2 , the dog entity identification method may be performed through an entity identification device and includes the following steps 210 to 260 .

In step 210, the object identification device inputs the dog's image to the deep learning model to detect the dog's eyes and nose. The eyes and nose can be detected from the dog image acquired in real time using the deep learning model mounted on the object identification device.

In step 210, the object identification device checks whether the direction of the image is the front according to the position and size of each of the eyes and nose. Through the relative position and size of the eyes and nose, it is possible to determine whether the dog's face is facing forward or not. If the direction of the face included in the image is not the front, of course, the image of the inscription is also not the front, so the object identification rate may be lowered. Therefore, it detects the eyes and nose included in the dog's face, and checks whether it is in front through the relative size and position of each.

In step 230, the object identification apparatus determines extraction of the inscription region from the image according to the noise included in the image. If the noise is less than the standard, the object identification apparatus may extract the inscription region of the image as a region of interest (ROI). The noise included in the image may be light reflectivity and sharpness as described above. In the case of light reflectivity, when the histogram is less than a threshold for the image determined as a front view, it is determined that there is no light reflection, and the inscription region can be extracted. In the case of sharpness, it is possible to extract the inscription region by determining whether the pattern of the inscription has a level of sharpness that can be identified. Of course, the inscription area is extracted only when both are less than the standard, and if any one exceeds the standard, the inscription area is not extracted and an image with less light reflection or an image with high clarity can be requested again.

Here, each criterion of light reflectivity and sharpness may be a different threshold value. For example, the threshold value of the light reflectivity may be a first threshold value and the threshold value of the sharpness may be a second threshold value. That is, when the light reflectivity is less than the first threshold and the sharpness exceeds the second threshold, the object identification apparatus may extract the inscription area. In addition, when the light reflectivity is equal to or greater than the first threshold and the sharpness is less than or equal to the second threshold, the image may be re-requested.

In step 240, the object identification apparatus performs preprocessing by adjusting the size of the inscription area and normalizing the histogram. In one embodiment, the image is divided into a plurality of equal regions using the Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm, and histogram normalization for each of the regions is repeated so that the contrast values of the regions are within the standard. have. This can be used as a method to increase the accuracy of object identification because higher contrast uniformity can be obtained compared to adjusting the contrast value for the entire image.

Meanwhile, in step 240 , the size of the inscription region of the entity identification device may be adjusted to be the same as the size of the plurality of inscription images included in the entity information. The original proportion of the inscription area is maintained as it is, but only the size can be adjusted so that it can be resized to be the same as a plurality of inscription images.

In step 250, the object identification device extracts feature information by inputting the pre-processed inscription area to the feature point extraction algorithm. The keypoint extraction algorithm is a shift (SIFT) algorithm, a speed-up Bobust Feature (SURF) algorithm, a BRISK (Binary Robust Invariant Scalable Keypoints, BRISK) algorithm, and Oriented FAST and Rotated BRIEF, ORB, which are also used in the prior art. ) algorithm can be used.

In step 260, the entity identification apparatus compares the plurality of pieces of characteristic information included in the plurality of previously registered entity information with the characteristic information. In order to compare the characteristic information, the entity identification apparatus may apply a RANSAC (Random Sample Consensus) algorithm to the characteristic information to remove outliers and duplicate values. That is, only accurate samples can be extracted and compared with a plurality of feature information.

Meanwhile, in order to indicate higher object identification accuracy, the object identification device divides the inscription area into a plurality of areas, extracts a plurality of characteristic information for each area, and divides the plurality of characteristic information included in the plurality of object information into the inscription area. It can be divided into regions of the same size as In addition, the degree of similarity can be compared for each divided region. For example, if the number of regions in which the degree of similarity exceeds the standard is greater than or equal to a certain level, the dog may be determined as a registered entity. Here, the number of regions may be automatically set to a certain ratio or more from the total number or may be determined according to a user's input. For example, when there are 9 divided regions, if the similarity of more than half of the 5 regions exceeds a criterion, it may be determined that the entity is a registered entity. Of course, if the number of areas with similarity exceeding the standard is less than a certain amount, the dog's image and characteristic information can be stored as new entity information.

Meanwhile, in step 260 , the entity identification apparatus may further perform a step of transmitting a result of comparing the characteristic information to the user terminal. The entity identification device transmits the dog's individual information to the user terminal if the dog's characteristic information is included in a plurality of individual information, and is used to register the dog's individual information if the dog's characteristic information is not included in the plurality of individual information Information may be transmitted to the user terminal.

3 is a block diagram of an apparatus for identifying a dog entity according to an embodiment of the disclosed technology. Referring to FIG. 3 , the object identification device 300 includes a camera 310 , a memory 320 , a processor 330 , and a communication device 340 .

The camera 310 takes an image of the dog's inscription. The camera 310 may use a small camera mounted on a device such as a smart terminal or a general type camera. The camera 310 takes an image of the dog's inscription according to the user's input.

The memory 320 stores entity information about a plurality of dogs. The entity information includes the inscription image of each dog, characteristic information, and information about the dog owner. Information on the dog owner may be the dog owner's name, address, and contact information. The memory 320 is a storage device mounted on the object identification device and may be implemented as a device capable of storing data for a long time, such as a general HDD or SSD.

The processor 330 inputs the dog's face image into the deep learning model to detect the dog's eyes and nose, extracts the inscription region from the face image, and inputs the extracted inscription region to the feature point extraction algorithm to extract feature information. Then, the extracted characteristic information and the characteristic information included in the entity information are compared.

The communication device 340 transmits the comparison result to the user terminal. The communication device may transmit the comparison result using a general Internet network or a wireless communication network.

On the other hand, the above-described object identification apparatus 300 may be implemented as a program (or application) including an executable algorithm that can be executed in a device such as a computer or a smartphone equipped with a camera. The program may be provided by being stored in a temporary or non-transitory computer readable medium.

The non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, and the like, and can be read by a device. Specifically, the various applications or programs described above are CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM (read-only memory), PROM (programmable read only memory), EPROM (Erasable PROM, EPROM) Alternatively, it may be provided by being stored in a non-transitory readable medium such as an EEPROM (Electrically EPROM) or flash memory.

Temporarily readable media include Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (Enhanced) SDRAM, ESDRAM), Synchronous DRAM (Synclink DRAM, SLDRAM) and Direct Rambus RAM (Direct Rambus RAM, DRRAM) refers to a variety of RAM.

4 is a diagram illustrating extraction of an inscription region from a face image of a dog. As shown in FIG. 4 , when the dog's face image is confirmed from the front, the inscription region 401 can be extracted as a region of interest. The entity identification device may determine whether the dog is in front according to the position and size of the dog's eyes and nose.

5 is a diagram illustrating division of an inscription area into a plurality of areas. Referring to FIG. 5 , the dog entity identification device may divide the inscription area, extract characteristic information for each, and compare it with characteristic information included in the entity information. In an embodiment, the inscription region may be divided into a plurality of regions with a uniform size, such as the 501 region. In FIG. 5, the inscription area is equally divided in a 3X3 shape, but it is also possible to divide the inscription area into a larger number of areas.

On the other hand, the object identification apparatus may compare the extracted feature information into a plurality of regions between regions at the same location. For example, feature information extracted from a region located at the upper right of the inscription region, such as region 501, may be compared with feature information extracted from a region located at the same position in the inscription image included in the entity information. In the remaining regions, feature information may be compared between regions at positions corresponding to each other as well. According to this process, the degree of similarity for each area is determined, and when the number of areas satisfying the similarity exceeds a certain value, it is determined as a registered entity, and when it is less than the predetermined value, it can be determined as a new object. By following this process, you can accurately identify each individual using the dog's inscription and easily register the dog's individual information.

Although the image-based dog entity identification method and apparatus according to an embodiment of the disclosed technology have been described with reference to the embodiment shown in the drawings to help understanding, this is merely exemplary, and those of ordinary skill in the art It will be understood that various modifications and equivalent other embodiments are possible. Accordingly, the true technical protection scope of the disclosed technology should be defined by the appended claims.

Claims (18)

detecting, by an entity identification device, the dog's eyes and nose by inputting the image of the dog into the deep learning model;
checking, by the object identification device, whether the direction of the image is the front according to the position and size of each of the eyes and nose;
determining, by the entity identification device, extraction of an inscription region from the image according to the noise included in the image;
performing pre-processing by adjusting the size of the inscription region and normalizing the histogram by the entity identification device;
extracting, by the entity identification device, feature information from the pre-processed inscription area using a feature point extraction algorithm; and
and comparing, by the entity identification device, a plurality of characteristic information included in a plurality of previously registered entity information with the characteristic information. The method of claim 1, wherein determining extraction of the inscription region comprises:
If the noise included in the image is less than a standard, the object identification device extracts the inscription area as a region of interest, and if the noise exceeds the standard, the dog object identification method requests the image again. The method of claim 1,
The noise includes light reflectivity and sharpness included in the image,
When the light reflectivity is less than a first threshold value and the sharpness exceeds a second threshold value, the object identification device extracts the inscription area, the light reflectivity is greater than or equal to the first threshold value and the sharpness is less than or equal to a second threshold value On the other hand, a dog object identification method that re-requests the image. The method of claim 1, wherein performing the pre-processing comprises:
A dog entity identification method in which the entity identification device adjusts the size of the inscription area to be the same as the size of a plurality of inscription images included in the entity information. The method of claim 1, wherein performing the pre-processing comprises:
The object identification device divides the image into a plurality of equal regions using a Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm and repeats histogram normalization for each of the regions so that the contrast values of the regions are within the standard. Pre-processing dog object identification method. The method of claim 1, wherein the comparing of the feature information comprises:
A dog entity identification method in which the entity identification device applies a RANSAC (Random Sample Consensus) algorithm to the characteristic information to remove outliers and duplicate values and compares them with the plurality of characteristic information. The method of claim 1,
The apparatus for identifying the object divides the inscription area into a plurality of areas, extracts a plurality of characteristic information for each area, and divides the plurality of pieces of characteristic information included in the plurality of object information into areas having the same size as the inscription area. A dog object identification method that compares the similarity for each divided area. 8. The method of claim 7,
The entity identification device determines that the dog is a registered entity if the number of regions in which the degree of similarity exceeds the criterion is greater than or equal to a certain level, and if the number of regions in which the similarity exceeds the criterion is less than a predetermined number, the image and characteristic information of the dog A method of identifying dog objects that stores them as new object information. The method of claim 1, wherein the comparing of the feature information comprises:
Further comprising the step of transmitting a result of comparing the characteristic information to the user terminal,
In the transmitting step, if the characteristic information of the dog is included in the plurality of individual information, the individual information of the dog is transmitted to the user terminal, and if the characteristic information of the dog is not included in the plurality of individual information, the A dog entity identification method for transmitting information for registering dog entity information to the user terminal. a camera that takes an image of the dog's face;
a memory for storing a plurality of individual information about dogs; and
The dog's face image is input into a deep learning model to detect the dog's eyes and nose, and the inscription area is extracted from the face image, and the inscription area is input to a feature point extraction algorithm to extract feature information and the extracted features A dog entity identification device comprising a; processor comparing the information with the characteristic information included in the entity information. 11. The method of claim 10,
The processor determines whether the direction of the face image is the front according to the position and size of each of the eyes and nose, and extracts the inscription area. 11. The method of claim 10,
The processor extracts the inscription area from the image according to the light reflectivity and sharpness included in the image,
If the light reflectivity is less than a first threshold and the sharpness exceeds a second threshold, the inscription region is extracted, and if the light reflectivity is greater than or equal to the first threshold and the sharpness is less than or equal to a second threshold, the image is reproduced Requesting dog entity identification device. 11. The method of claim 10,
and the processor adjusts the size of the inscription region to be the same as the size of a plurality of inscription images included in the entity information. 11. The method of claim 10,
The processor divides the image into a plurality of equal regions using a Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm and repeats the histogram normalization for each of the regions to preprocess so that the contrast values of the regions are within the standard. Dog object identification device. 11. The method of claim 10,
The processor applies a RANSAC (Random Sample Consensus) algorithm to the characteristic information to remove outliers and duplicate values, and compares them with the plurality of characteristic information. 11. The method of claim 10,
The processor compares a plurality of characteristic information included in the plurality of entity information with the characteristic information, and divides the inscription region into a plurality of regions to extract a plurality of characteristic information for each region, and the plurality of entity information A dog entity identification device that divides a plurality of characteristic information included in the inscription area into areas of the same size as the inscription area, and compares the degree of similarity for each divided area. 17. The method of claim 16,
The processor determines the dog as a registered entity if the number of regions in which the degree of similarity exceeds the criterion is greater than or equal to a certain level, and if the number of regions in which the similarity exceeds the criterion is less than a certain amount, the image and characteristic information of the dog are new Dog object identification device that stores object information. 11. The method of claim 10,
The dog inscription recognition device further comprises a communication device for transmitting the comparison result to the user terminal,
The communication device transmits the individual information of the dog to the user terminal if the characteristic information of the dog is included in the plurality of entity information, and if the characteristic information of the dog is not included in the plurality of entity information, the dog dog A dog entity identification device that transmits information for registering entity information of a dog to the user terminal.

Images