KR102246471B1 - Apparatus for detecting nose of animal in image and method thereof - Google Patents

Abstract

The present disclosure relates to a method for an electronic device to detect an object in an image. An electronic device for identifying an object in an image according to one embodiment of the present invention comprises: a display; at least one camera; a memory for storing one or more instructions; and at least one processor for executing the one or more instructions. The processor is configured to, by executing the one or more instructions, obtain an image including the object, determine at least one guide region corresponding to the object in the obtained image, determine feature information for identifying the object from the at least one guide region, determine a type of the object in the image using the determined feature information, and store at least one pixel value in the image including the object when the type of the identified object is determined as part objects related to a nose of an animal. Accordingly, the type of the object in the image can be accurately identified by using the guide region.

Description

A method and apparatus for detecting the nose area of an animal in an image {APPARATUS FOR DETECTING NOSE OF ANIMAL IN IMAGE AND METHOD THEREOF}

The present disclosure relates to an electronic device and method for identifying an object in an image. More particularly, it relates to an electronic device and method for identifying an object from an image including at least one object.

As information and communication technology develops, technologies for identifying objects included in an image including a plurality of frames are being developed. In particular, technologies are being developed that allow the electronic device to identify an object in an image by itself or to identify a predetermined object set in advance by applying a human recognition method to an electronic device.

In addition, based on recent image recognition technology, biometric information of a person is acquired from an image, and technology using biometric authentication is actively progressing as an authentication method for banking, shopping, and unlocking using the obtained biometric information. Has become. For example, as a representative biometric authentication method, iris recognition through the image of the eye area detected in the image, face recognition through the image of the face area detected in the image, the hand fingerprint image detected in the image is analyzed, and the result of the analysis Technology for performing fingerprint recognition on the basis of is being developed.

However, since authentication technology using biometric information requires high accuracy due to authentication characteristics, there is still a need to develop a technology for accurately authenticating an object through biometric information. In addition, development of biometric authentication technology targeting animals as well as biometric authentication technology targeting humans is required.

Korean Patent Publication No. 10-2018-0109665

According to an embodiment, an electronic device and method for identifying an object in an image may be provided.

In addition, according to an embodiment, an electronic device may be provided in which the electronic device identifies a type of an object in an image and stores at least one pixel value in the image based on the type of the identified object.

According to an embodiment of the present disclosure for achieving the above-described technical problem, an electronic device for identifying an object in an image includes: a display; At least one camera; A memory for storing one or more instructions; And at least one processor executing the one or more instructions. Including, wherein the processor executes the one or more instructions to obtain an image including the object, determine at least one guide area corresponding to the object in the obtained image, and the at least one guide area Determining feature information for identifying the object from, and identifying the type of the object in the image using the determined feature information, and the type of the identified object is the animal's nose or partial objects related to the animal's nose When identified as, an electronic device that stores at least one pixel value in an image including the object may be provided.

In addition, according to another embodiment of the present disclosure for solving the technical problem, a method of identifying an object in an image by an electronic device includes: obtaining an image including the object; Determining at least one guide area corresponding to the object in the acquired image; Determining feature information for identifying the object from the at least one guide area; Identifying a type of an object in the image using the determined feature information; And when the type of the identified object is identified as an animal's nose or partial objects related to the animal's nose, storing at least one pixel value in the image including the object. It may include.

In addition, a method for identifying an object in an image by an electronic device, the method comprising: obtaining an image including the object; Determining at least one guide area corresponding to the object in the acquired image; Determining feature information for identifying the object from the at least one guide area; Identifying a type of an object in the image using the determined feature information; And when the type of the identified object is identified as an animal's nose or partial objects related to the animal's nose, storing at least one pixel value in the image including the object. A computer-readable recording medium in which a program for executing a method comprising a computer is recorded on a computer may be provided.

According to an exemplary embodiment of the present disclosure, a type of an object in an image can be accurately identified by using a plurality of guide areas according to different scales.

In addition, according to an embodiment, the object in the image may be more accurately identified by using a plurality of conditions for identifying the object.

1 is a diagram schematically illustrating a process of identifying an object in an image by an electronic device according to an exemplary embodiment.
2 is a flowchart illustrating a method of identifying an object in an image by an electronic device according to an exemplary embodiment.
3 is a flowchart illustrating a method of determining at least one guide area according to a plurality of scales by an electronic device according to an exemplary embodiment.
4 is a diagram illustrating a method of determining at least one guide area by an electronic device according to an exemplary embodiment.
5 is a flowchart illustrating a method of identifying, by an electronic device, a type of an object in an image based on an object identification result for each guide area, according to an exemplary embodiment.
FIG. 6 is a diagram illustrating a method of identifying a type of an object in an image, based on a result of identifying an object for each guide area, by an electronic device according to an exemplary embodiment.
7 is a diagram illustrating a method of preprocessing an image in order to remove at least one noise in an image by an electronic device according to an exemplary embodiment.
FIG. 8 is a diagram illustrating a process of determining feature information from some of at least one guide area in which an electronic device is generated according to an exemplary embodiment.
9 is a diagram for describing a process of detecting an outline of an object from at least one guide area by an electronic device according to an exemplary embodiment.
FIG. 10 is a diagram illustrating a process of detecting an outline from an image in at least one guide area that has been binarized and identifying a type of an object based on the detected outline, according to an exemplary embodiment.
11 is a diagram for describing identification conditions used by an electronic device to determine a probability that an object detected in one guide area corresponds to a nose, according to an exemplary embodiment.
13 is a block diagram of an electronic device according to an exemplary embodiment.
14 is a block diagram of an electronic device according to another embodiment.
15 is a block diagram of a server according to an embodiment.
16 is a diagram for describing a process of identifying an object in an image by interworking with each other between an electronic device and a server, according to an exemplary embodiment.

The terms used in the present specification will be briefly described, and the present disclosure will be described in detail.

Terms used in the present disclosure have selected general terms that are currently widely used as possible while considering functions in the present disclosure, but this may vary according to the intention or precedent of a technician engaged in the art, the emergence of new technologies, and the like. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning of the terms will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the overall contents of the present disclosure, not a simple name of the term.

When a part of the specification is said to "include" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary. In addition, terms such as "... unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. .

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted in order to clearly describe the present disclosure, and similar reference numerals are attached to similar parts throughout the specification.

1 is a diagram schematically illustrating a process of identifying an object in an image by an electronic device according to an exemplary embodiment.

According to an embodiment, the electronic device 1000 may acquire an image including at least one object, identify an object area within the acquired image, and identify a type of an object represented by the identified object area. According to an embodiment of the present disclosure, the electronic device 1000 may acquire an animal image including an animal by capturing an image including an animal using at least one camera.

According to an embodiment, the object identified by the electronic device 1000 may include a part of body parts of an animal. For example, the object identified by the electronic device 1000 may include at least one of an animal's nose or partial objects related to the animal's nose. According to an embodiment, the partial objects according to the present disclosure include at least one of an animal's nostril, a throat located between the nostrils (eg, a concave rib between the nose and upper lip), and nose hairs located around the nose. Can include. The electronic device 1000 according to the present disclosure detects a partial object region including a nostril, a gravitational weight, and a nose hair located around the nose as a region for an animal nose region or partial objects related to the animal nose in an animal image And, it is possible to identify the type of object or partial object in the detected area.

According to an embodiment, the electronic device 1000 may be implemented as various types of devices including a communication module capable of communicating with the server 2000. For example, the electronic device 1000 includes a digital camera, a mobile terminal, a smart phone, a laptop computer, a tablet PC, an e-book terminal, a digital broadcasting terminal, personal digital assistants (PDA), and a PMP ( Portable Multimedia Player), navigation, MP3 player, and the like, but are not limited thereto. In addition, according to an embodiment, the electronic device 1000 may be a device that can be worn by a user.

For example, wearable devices include accessory type devices (e.g., watches, rings, wrist bands, ankle bands, necklaces, glasses, contact lenses), head mounted display devices (HMD), textile or clothing integrated devices ( Examples: electronic clothing), a body-attached device (eg, a skin pad), or a living body implantable device (eg, an implantable circuit) may be included, but is not limited thereto.

According to an embodiment, the server 2000 may include other computing devices capable of transmitting and receiving data to and from the electronic device by being connected to the electronic device 1000 through the network 3000. In addition, according to an embodiment, the server device 2000 may be a Wearable Business Management Server (W-BMS) for managing wearable devices. According to an embodiment, the network 3000 includes a local area network (LAN), a wide area network (WAN), a value added network (VAN), a mobile radio communication network, and It is a data communication network in a comprehensive sense that includes a satellite communication network and a mutual combination thereof, and enables each network component shown in FIG. 1 to communicate with each other smoothly, and includes wired Internet, wireless Internet, and mobile wireless communication network. I can.

According to an embodiment, the electronic device 1000 acquires an animal image based on a user's input, displays the acquired animal image, and displays at least one guide area 114 among a plurality of guide areas. It can be displayed on the screens 102 and 104. In addition, according to an embodiment, while the user photographs an animal image, the electronic device 1000 displays a guide message 112 and 116 such as'Please take a picture of the dog's nose' for convenience of photographing the user. 102, 104).

The electronic device 1000 according to the present disclosure identifies the type of object photographed in the guide area 114, and when the type of the identified object matches the preset type of object, photographs the body of another animal in the animal image. You can print out a guide message for doing so. According to an embodiment, the electronic device 1000 identifies the type of the object included in the guide area 112 in the image acquired based on the user's input, and the type of the identified object coincides with a preset type of object. In this case, the guide area 112 may be displayed by blinking according to a preset period or by changing the color of the currently displayed guide area 112 to indicate to the user that the image has been accurately captured. According to an embodiment, the electronic device 1000 recognizes a type of object in an image acquired based on a user input as'nose' or'partial objects related to the nose', and recognizes the recognized'nose' or'partial object'. Information on an image including'fields' may be further stored in the electronic device.

2 is a flowchart illustrating a method of identifying an object in an image by an electronic device according to an exemplary embodiment.

In S210, the electronic device 1000 may acquire an image including an object. For example, the electronic device 1000 may acquire an image using at least one camera included in the electronic device. After obtaining the first image using at least one camera, the electronic device 1000 may obtain a second image of the next frame. That is, the electronic device 1000 may acquire an image including a plurality of images of an object.

In S220, the electronic device 1000 may determine at least one guide area corresponding to an object in the acquired image. For example, the electronic device 1000 may generate a first guide area and generate a plurality of second guide areas by scaling the generated first guide area according to a preset scale. According to an embodiment, the electronic device 1000 may display the first guide area on the screen of the electronic device, and may not display the second guide area on the screen of the electronic device.

According to an embodiment, the electronic device 1000 may determine the first guide area according to a preset position. According to another embodiment, the electronic device 1000 acquires a sensor value according to a user's movement through at least one sensor included in the electronic device, and calculates the coordinates of the first guide area based on the acquired sensor value. You can also decide.

In S230, the electronic device 1000 may determine feature information for identifying an object from at least one guide area. For example, the electronic device 1000 may determine coordinate information of the contour by using the contour of the object detected from at least one guide area, and determine feature information based on the coordinate information of the contour. In more detail, the electronic device 1000 detects a first partial contour for the nostril and a second partial contour for the nasal midline from at least one guide region, and Based on the identification condition regarding the location, it is possible to determine the feature information.

In S240, the electronic device 1000 may identify the type of the object in the image using feature information. For example, the electronic device 1000 determines an identification condition regarding the relative position of the first partial contour and the second partial contour based on the coordinate information of the first partial contour and the coordinate information of the second partial contour in the feature information. And, the type of the object in the image can be identified using the feature information generated by weighting the scores for each of the determined identification conditions.

According to another embodiment, the electronic device 1000 may identify a type of an object in an image by inputting feature information into an artificial intelligence model. According to an embodiment, before inputting the feature information to the artificial intelligence model, the electronic device 1000 may resizing the feature information to a predetermined size suitable for inputting the feature information into the artificial intelligence model.

According to an embodiment, in the artificial intelligence model used by the electronic device 1000, when feature information is input, the type of the object is a probability by a predefined label (eg, nostril, nose, nasal midline, etc.) You can print the values. According to another embodiment, in the artificial intelligence model used by the electronic device 1000, when feature information is input, a label corresponding to the input feature information with the highest probability (e.g., nose, nose hair, nostril, midline) It can be displayed as the type of object. According to another implementation, when feature information is input, the artificial intelligence model may output a value of whether an object in an image corresponding to the feature information is an animal's nose.

The artificial intelligence model used by the electronic device 1000 may include an artificial neural network model formed in a deep neural network structure having multiple layers or a machine learning model that can be trained according to other artificial intelligence learning algorithms. According to an embodiment, the artificial neural network model used by the electronic device is a convolutional neural network, a deep neural network (DNN), a recurrent neural network (RNN), a bidirectional recurrent deep neural network (BRDNN), and a multilayer perceptron (MLP). It may include, but is not limited thereto.

According to an embodiment, the artificial neural network model used by the electronic device 1000 is a multilayer perceptron (MLP), and there is a connection between perceptrons of an adjacent layer, but there is no connection between perceptrons of the same layer. ) May be an artificial neural network. According to another embodiment, the artificial neural network used by the electronic device 1000 may have a structure in which a fully-connected layer is connected to a CNN structure in which a convolutional layer and a pooling layer are repeatedly used.

In S250, when the type of the identified object is identified as an animal's nose or partial objects related to the animal's nose, the electronic device 1000 may store at least one pixel value in the image including the object. For example, when the object in the acquired image is identified by the nose of the animal, the electronic device 1000 stores information on the image identified by the nose of the animal in the memory of the electronic device as a means for authenticating the animal later, The image containing the nose stored in the memory can be used for biometric authentication.

In addition, according to an embodiment, the electronic device 1000 acquires an image of an animal according to a user input in response to an output of a guide message such as'Please take a picture of the dog's nose', and When the type is identified by the dog's nose, information on the currently acquired image may be stored in the memory.

In addition, according to an embodiment, when the type of the identified object is identified as a nose or partial objects related to the nose of an animal, the electronic device 1000 does not store information on the entire image, but stores the identified object. It is also possible to store only information on pixels in the included guide area. In addition, when the type of the identified object is identified as an animal's nose or partial objects related to the animal's nose, the electronic device 1000 not only stores at least one pixel value in the image including the object, but also stores at least one The type of object identified by matching the pixel values may be further stored.

3 is a flowchart illustrating a method of determining at least one guide area according to a plurality of scales by an electronic device according to an exemplary embodiment.

In S310, the electronic device 1000 may determine a first guide area corresponding to the object to include a preset reference pixel in the image. For example, the electronic device 1000 may determine coordinates of a reference pixel based on the size of the screen of the electronic device, and may determine a first guide area to include a reference pixel positioned at the determined coordinates.

According to another embodiment, the electronic device 1000 includes a current direction of the electronic device and a current position of the electronic device and a location of a photographing target based on a sensing value acquired through at least one sensor included in the electronic device. Position information indicating a relative relationship may be determined, and a first guide area may be determined based on the direction and position information. In addition, according to an embodiment, the electronic device 1000 may determine a first guide area having a rectangular shape having a line segment extending from a position of a reference pixel by a predetermined width and height. According to an embodiment, the first guide region may be formed in a square shape, but is not limited thereto.

In S320, the electronic device 1000 may determine at least one second guide area corresponding to an object by scaling the first guide area according to a preset scale. For example, the electronic device 1000 determines the scaling factor to be 1.1, and applies the determined scaling factor to the horizontal length in the first guide area, thereby increasing the horizontal length by 10% than the horizontal length of the first guide area. The branches may determine the second guide area. Also, the electronic device 1000 may determine a second guide region having a vertical length that is 10% greater than the vertical length of the first guide region by applying a scaling factor of 1.1 to the vertical length of the first guide region.

4 is a diagram illustrating a method of determining at least one guide area by an electronic device according to an exemplary embodiment.

As described above in FIG. 3, after determining the first guide area 416, the electronic device 1000 scales the determined first guide area 416 according to a preset scale, so that at least one of the second guide areas ( 412, 414, 418, 422) can be created. According to an embodiment, the first guide areas 416 may be displayed on the screen of the electronic device, but the second guide areas may not be displayed on the screen of the electronic device. The electronic device 1000 may not display the second guide areas generated based on the first guide area on the screen, but may use only the calculation process for identifying an object in the image.

For example, the electronic device 1000 determines the lower left corner of the first guide area 416 as the coordinates of the reference pixel, and extends from the determined reference pixel position by a predetermined length of the horizontal width 402 The first guide area 416 including the horizontal line segment and the vertical line segment extended by the length of the vertical height 404 may be displayed on the screen. The user of the electronic device can easily determine the area to be photographed based on the screen of the electronic device displayed in the first guide area 416.

In addition, according to an embodiment, the electronic device 1000 determines a scaling factor for increasing or decreasing at least one of a horizontal length or a vertical length of the first guide area, and determines the determined scaling factor as the first guide area. By applying to, it is possible to determine the second guide areas. According to an embodiment, the electronic device 1000 determines two second guide areas 412 and 414 having a larger area than the first guide area, and a second guide area having a narrower area than the first guide area. Two of them 418 and 422 can be determined.

The electronic device 1000 according to the present disclosure may extract feature information from each of the first guide area and at least one second guide area, and identify an object in the image using the extracted feature information. However, according to another embodiment, the electronic device 1000 according to the present disclosure determines a probability value corresponding to an object for each of the first guide area and the second guide area, and based on whether the determined probability value is greater than a preset threshold. Thus, it is possible to primarily identify objects for the guide areas. The electronic device 1000 according to the present disclosure determines that objects for guide areas having a probability value smaller than a preset threshold are not identified, and only feature information extracted from guide areas having a probability value larger than a preset threshold. It can be used to identify the type of object secondary. According to another embodiment, the electronic device 1000 may identify the type of object in the image by inputting only feature information extracted from guide areas having a probability value greater than a preset threshold value into the artificial intelligence model.

Accordingly, the electronic device 1000 according to the present disclosure does not perform an artificial intelligence operation on all guide areas in which the first image is generated, but uses feature information extracted for each of at least one guide area, based on a preset reference. Accordingly, after first identifying an object, the object identification process is not secondly performed for guide areas that are not identified as an object, thereby increasing the computational efficiency of the electronic device. A method of identifying the type of the object in the entire image by the electronic device 1000 based on the object identification value determined for each of the plurality of guide areas will be described in more detail with reference to FIGS. 5 to 6 to be described later.

5 is a flowchart illustrating a method of identifying, by an electronic device, a type of an object in an image based on an object identification result for each guide area, according to an exemplary embodiment.

In S520, the electronic device 1000 identifies a type of an object for each of the first guide area and at least one second guide area. For example, the electronic device 1000 detects a contour of an object for each of the first guide area and the second guide area, and applies different weights to the identification condition determined using the detected contour components. , It is possible to determine a probability that the detected contour for each guide area corresponds to the object. The electronic device 1000 may identify the type of the object for each guide area based on a probability value that the contour detected for each guide area corresponds to the object.

In S540, the electronic device 1000 may determine the frequency of the guide area identified as an object of the same type among the types of the identified object. For example, the electronic device 1000 may calculate the number of guide areas identified as the same type by counting the types of objects identified for each guide area. In S560, the electronic device 1000 identifies the type of the object in the image based on the frequency of the guide area identified as the same type of object. According to an embodiment, the electronic device 1000 may identify the type of the object counted as the largest number among the guide areas as the type of the object in the image.

FIG. 6 is a diagram illustrating a method of identifying a type of an object in an image, based on a result of identifying an object for each guide area, by an electronic device according to an exemplary embodiment.

Referring to FIG. 6, cases in which the electronic device 1000 identifies the type of the object in the entire image by using the type of the object identified for each guide area are illustrated. For example, referring to case 602, an object corresponding to a first guide area (eg, BOX 0) is identified as a nose, and an object corresponding to the second guide area (eg, Box 1) is identified as a nose ( Nose), and as the probability value is determined to be less than a preset threshold for the remaining guide areas (eg, Box 2, 3, 4), a case where objects corresponding to the remaining guide areas are not identified is shown. . In the case of the case 602, the electronic device 1000 may identify the type of the object in the entire image as a nose because the frequency of the guide area identified as a nose is the largest as 2. According to an embodiment, when the electronic device 1000 counts the number of guide areas representing the same type of frequency, the frequency of the unidentified guide area may not be counted.

According to another embodiment, for example, referring to the case 604, an object corresponding to a first guide area (eg, BOX 0) is identified as a left eye, and a second guide area (eg, Box 0) The object corresponding to 1) is identified by nose, the object corresponding to the third guide area (eg, Box 2) is identified by the left eye, and the remaining guide areas (eg, Box 3, 4 For ), as the probability value is determined to be less than or equal to a preset threshold, a case in which objects corresponding to the remaining guide areas are not identified is shown.

In the case of the case 604, the electronic device 1000 has a frequency of 2 of a guide area identified as a left eye, a frequency of 1 of a guide area identified as a nose, and no object is identified. Since the frequency of the guide area is 2, the frequency of the guide area identified by the left eye is compared with the frequency of the guide area in which the object is not identified. However, according to an embodiment, since the electronic device 1000 may not use the frequency of the guide area in which the object is not identified for counting calculations, the electronic device 1000 is a guide identified as a left eye. After determining that the frequency of the region is the highest, the type of the object in the image can be identified with the left eye.

According to another embodiment, referring to the case 606, an object corresponding to a first guide area (eg, BOX 0) is identified as a left eye, and corresponds to a second guide area (eg, Box 1). The object is identified as the right eye, the object corresponding to the third guide area (eg, Box 2) is identified as a nose, and the remaining guide areas (eg, Box 3, 4) are identified as probability As the value is determined to be less than or equal to a preset threshold, a case in which objects corresponding to the remaining guide areas are not identified is shown.

In the case 606, the electronic device 1000 has a frequency of 1 of a guide identified as a left eye, and a frequency of a guide identified as a right eye of 1, and is identified as a nose. Since the frequency of the guide area is 1, it may not be possible to determine the frequency of the guide area representing the same type of object among the guide areas. Like the case 606, when the electronic device 1000 fails to determine the frequency of the guide area representing the same type of object, it may determine that the type of the object in the image is not identified. When the type of the object in the image is not identified, the electronic device 1000 may output a character such as'none' on the screen.

According to another embodiment, referring to the case 608, an object corresponding to the first guide area (eg, BOX 0) is identified as a left eye, and the second, third, and fourth guide areas (eg, Box 0) All objects corresponding to 1, 2, and 3) are identified as noses, and an object corresponding to the fifth guide area (eg, Box 4) is identified as a right eye. In the case 608, in the electronic device 1000, the frequency of the guide area identified by the nose is 3, the frequency of the guide area identified by the left eye is 1, and the frequency of the guide area identified by the right eye is Can be determined by 1. Accordingly, the electronic device 1000 may identify the type of the object in the image as a nose because the frequency of the guide region identified by the nose is the highest as 3.

According to another embodiment, referring to the case 610, objects corresponding to the first and second guide areas (eg, BOX 0 and 1) are identified as noses, respectively, and the third guide area (eg, Box 2) is identified as a nose. ) Is identified as a right eye, and an object corresponding to the fourth and fifth guide areas (eg, Box 3, 4) is identified as a left eye. In the case 610, the electronic device 1000 determines the frequency of the guide area identified by the nose is 2, the frequency of the guide area identified by the right eye is 1, and the frequency of the guide area identified by the left eye. Can be determined by 2. Accordingly, the electronic device 1000 may determine that the frequency of the guide area currently identified by the nose and the frequency of the guide area identified by the left eye are greater than the frequency of the guide area identified by the right eye. However, since the electronic device 1000 determines that the frequency of the guide area identified by the nose and the frequency of the guide area identified by the left eye is equal to 2, which is determined to be greater than the frequency of the guide area identified by the right eye, the type of object in the image It can be determined that it cannot be identified. Accordingly, similar to the case 606, the electronic device 1000 may determine that the object in the image cannot be identified and may output a guide message such as'none' on the screen.

7 is a diagram illustrating a method of preprocessing an image in order to remove at least one noise in an image by an electronic device according to an exemplary embodiment.

According to an embodiment of the present disclosure, the electronic device 1000 may pre-process the image in order to remove noise in the acquired image and improve the accuracy of nose detection. According to an embodiment, in S720, the electronic device 1000 may remove noise in the image by applying a predetermined kernel to pixel values in the image.

According to an embodiment, the electronic device 1000 may remove Gaussian noise in an image by applying a Gaussian kernel indicating a Gaussian distribution. For example, noises following a Gaussian distribution may be included in the image acquired by the electronic device 1000, such as noise due to a characteristic of an image sensor that photographed the image. These noises may deteriorate the quality of an image or an image including a plurality of images. The electronic device 1000 may remove Gaussian noise in the image by applying a Gaussian kernel having a preset size, representing a Gaussian distribution, to an image. According to an embodiment, the electronic device 1000 may divide an image into predetermined partial regions and apply a Gaussian kernel to each of the divided partial regions. However, the present invention is not limited thereto, and the electronic device 1000 may remove noise by using other image filters for removing noise in an image.

In S740, the electronic device 1000 may convert pixel values in the image from which noise has been removed. According to an embodiment, when a Gaussian kernel is used, the electronic device 1000 may equalize a histogram of pixel values in an image from which Gaussian noise has been removed. For example, the electronic device 1000 may obtain brightness values of an image for each pixel based on pixel values in the image. The electronic device 1000 may generate a histogram of the brightness values in the image by classifying the brightness values of pixels in the image into bins in which the brightness values for each pixel in the image are classified at a predetermined level.

According to an embodiment, the histogram may include a bin including the number of pixels representing a specific brightness value in an image, and the bins may have a predetermined range for classifying brightness values. The electronic device 1000 may smooth the histogram indicating the distribution of brightness values for the image, so that the brightness values in the image are evenly distributed throughout the image. Also, the electronic device 1000 may improve the contrast of the entire image by smoothing the histogram of the image.

However, according to another embodiment, the electronic device 1000 may equalize the intensity of pixels in the image, not the histogram. In addition, the electronic device 1000 may level pixel values in an image by using a method for leveling other known pixel values in addition to the above-described leveling method.

In S760, the electronic device 1000 may divide the images from which pixel values are converted into at least one partial region, and binarize pixel values in the partial regions using threshold values determined for each divided partial region. For example, the electronic device 1000 may convert the image into a gray scale image and may binarize the image using a preset threshold.

However, the electronic device 1000 according to an embodiment of the present disclosure divides an image into one or more partial regions, and determines a threshold value for each of the divided partial regions, so that a threshold value determined differently for each partial region of the image is determined. By using, the pixel values for each partial area can be binarized. According to an embodiment, when the image is converted to gray scale, the electronic device 1000 may binarize the brightness value for each pixel in the partial region using a threshold value determined for each partial region. The electronic device 1000 may more accurately detect a contour corresponding to an object from the binarized image by binarizing the image using different threshold values for each partial region. According to another embodiment, the electronic device 1000 may detect an outline of an object from a guide area including a binary brightness value for each couple area.

In addition, according to an embodiment, when the electronic device 1000 uses the histogram smoothing technique, the electronic device 1000 divides the histogram-smoothed images into at least one partial region, and a threshold determined for each divided partial region. It is also possible to binarize pixel values in partial regions using values.

FIG. 8 is a diagram illustrating a process of determining feature information from some of at least one guide area in which an electronic device is generated according to an exemplary embodiment.

In S820, the electronic device 1000 may generate at least one guide area in the image. S820 may correspond to a process in which the electronic device 1000 determines a first guide area in FIGS. 3 to 4 and creates second guide areas by scaling the determined first guide area according to a preset scale, so detailed description Will be omitted.

In S840, the electronic device 1000 may detect an outline of an object for each of at least one guide area. For example, the electronic device 1000 may detect a contour corresponding to an object in the image by extracting a feature point from a binarized image and identifying a set of points representing the same image value among the extracted feature points. . According to an embodiment, the electronic device 1000 may detect a plurality of contours corresponding to an object in an image, and may determine outermost contours among the detected contours.

According to an embodiment, the electronic device 1000 may detect a max contour among contours detected in an image using a convex hull algorithm. In addition, according to an embodiment, the electronic device 1000 detects a contour from at least one binarized guide region, and then smoothes the boundary of the contour by applying an intermediate value filter to the detected contour. You can also do more. The electronic device 1000 may detect the flattened candidate contour as a contour of an object for each of at least one guide area.

In S860, the electronic device 1000 may determine a probability that the detected contour corresponds to the object. For example, the electronic device 1000 may detect an outermost contour for each of at least one guide area including a binarized image area, and determine a probability that each of the detected outermost contours corresponds to an object.

In more detail, the electronic device 1000 may determine contour components related to an outline of at least one guide area, and determine a first identification condition for identifying an object using the determined contour components. The electronic device 1000 may determine a probability that contours in at least one guide area correspond to the object by applying different weights to the determined identification condition.

According to an embodiment, the first identification condition is a condition for identifying the nose area of the animal in the animal image, and includes the area of the detected contour, the coordinates of the detected contour, the height of the detected contour, and It may include a ratio of a width, an area of the outermost contour, and a ratio of an area within the contour excluding an area of the outermost contour.

For example, the electronic device 1000 determines the height of the contour, the width of the contour, the area of the contour, the area of the outermost contour among the contours, and pixels constituting the contour as contour components of the contour detected in one guide area. I can. The electronic device 1000 includes an area of the outline, a coordinate report on the boundary of the outline, an aspect ratio of the detected outline height and a width of the outline, an area of the outline, and an area of the outermost outline based on outline components. It is possible to determine first identification conditions such as solidity of the area within the contour except for.

The electronic device 1000 determines the first identification condition as described above from the contour detected in one guide area and the contour components constituting the contour, and then the detected contours have a first identification condition having different weights. You can decide the score for each of your items. The electronic device 1000 allocates points for each identification condition to the contour detected in the guide area, and then weights the allocated points according to different weights assigned to each identification condition, so that the outline within the guide area corresponds to the object. You can determine the probability of becoming.

In S880, the electronic device 1000 may determine whether probability values corresponding to an object determined for each of at least one guide area are equal to or greater than a preset threshold. Also, the electronic device 1000 may identify which of the at least one guide area is a guide area including an outline larger than a preset threshold. The electronic device 1000 extracts feature information only from a guide area including an outline representing a probability value that is greater than a preset threshold value, and performs another process of identifying an object in the image by using the extracted feature information.

According to an embodiment, the electronic device 1000 may resize the guide areas to a preset size before extracting feature information from guide areas including outlines representing a probability value greater than a preset threshold. . The electronic device 1000 may determine feature information from at least one guide area resized to a preset size.

The electronic device 1000 according to the present disclosure determines probability values that contours extracted from at least one guide region in the image correspond to an object, and that the object is not identified for a guide region whose determined probabilities are less than a preset threshold. It is determined, and by extracting feature information only from a guide region whose probability values are larger than a preset threshold, and inputting the extracted feature information into the artificial intelligence model, the amount of artificial intelligence computation can be reduced.

That is, the electronic device 1000 first determines a probability value corresponding to the object with respect to the guide regions in the image, and then uses the feature vector extracted from the guide region having a high probability value corresponding to the object, By performing an artificial intelligence operation using an artificial intelligence model only for a guide region having a high probability value to be matched, it is possible to quickly and accurately identify the type of object in the image.

9 is a diagram for describing a process of detecting an outline of an object from at least one guide area by an electronic device according to an exemplary embodiment.

The electronic device 1000 may detect a contour corresponding to an object for each guide area by using the contour detection method corresponding to step S840. For example, the electronic device 1000 may extract a feature point in an image and connect the extracted feature points to detect an outline of an object. According to an embodiment, the electronic device 1000 may detect a contour from the guide area of the binarized image based on threshold values determined for each partial area in the image.

According to an embodiment, the electronic device 1000 may determine contours having an area larger than a preset area by clustering contours detected from the binarized guide area. According to another embodiment, the electronic device 1000 may determine a contour having a relatively large area among the plurality of contours by approximating a plurality of contours extracted from the binarized guide region. The electronic device 1000 may determine a first partial contour of the nostril and a second partial contour of the gravitational line of the nose of the animal by clustering contours detected from the binarized guide region according to a predetermined criterion.

10 is a flowchart illustrating a process of detecting an outline from an image in at least one guide area that has been binarized and identifying a type of an object based on the detected outline, according to an exemplary embodiment.

According to an embodiment, the electronic device 1000 may binarize a plurality of guide areas in an image, and detect an outline of an object from each of the plurality of binarized guide areas. For example, in S970, the electronic device 1000 determines a first identification condition based on contour components of contours detected from each of the guide regions, and based on the determined first identification condition, the contours in the guide region correspond to the object. You can determine the probability of becoming. Among the guide areas, the electronic device 1000 removes guide areas in which the probability that the contours in the guide area correspond to the object is lower than the preset threshold value, and the probability that the contours in the guide area correspond to the object is greater than the preset threshold. The object identification process can be performed secondarily only for the area.

For example, the electronic device 1000 may detect a first partial contour of the animal's nostril in the guide area and a second partial contour of the animal's nasal midline. According to an embodiment, the electronic device 1000 determines coordinate information of the first partial contour and the second partial contour detected from the guide area, and features based on the determined coordinate information of the first partial contour and the second partial contour. Information can be determined. The electronic device 1000 identifies the probability that contours in the guide area correspond to the object according to the first identification process, and binarizes the guide area in which the probability that the identified contours within the guide area correspond to the object is greater than a preset threshold. In addition, the first partial contour and the second partial contour may be detected from the binarized guide region.

In S980, the electronic device 1000 determines a second identification condition regarding the relative positions of the detected first and second partial contours from a guide area including an outline having a probability corresponding to the object greater than a preset threshold. In addition, feature information may be determined based on a score value for each item of the second identification condition, and a secondary identification process for an object in the image may be performed based on a score value indicated by the determined feature information.

FIG. 11 is a diagram for describing identification conditions used by an electronic device to determine a probability that an object detected in at least one binarized guide area corresponds to a nose of an animal, according to an exemplary embodiment.

For example, the electronic device 1000 uses the first identification condition 1152 to determine a probability value that the contour detected from each of the plurality of guide regions in the image corresponds to the nose, and the determined probability values Only for the guide area larger than the threshold, the secondary object (eg, the nose of the animal) is identified again. The electronic device 1000 performs an object recognition process using the second identification condition 1156 for only some of the initially generated guide areas, thereby quickly and accurately recognizing the object in the image. You can do it.

For example, the electronic device 1000 may acquire an image including at least a part of an object, and may generate a plurality of guide areas in the acquired image. According to an embodiment, the electronic device 1000 detects contours of an object from each of the guide regions, and determines a first identification condition 1152 using the detected contour components. The first identification condition 1152 may include a contour area and coordinate information of the detected contour. In addition, according to an embodiment, the first identification condition is a ratio of the detected height and width of the contour (aspect ratio), the area of the outermost contour, and the solidity of the area within the contour excluding the area of the outermost contour. It may further include, but is not limited thereto. In addition, according to an embodiment, the contour component 1146 may include a height of a contour, a width of the contour, an area of the contour, pixels constituting the contour, and coordinate information of pixels constituting the contour.

According to an embodiment, the contour component 1146, the first identification condition 1152, and the second identification condition 1156 used by the electronic device 1000 are matched to the identification numbers 1142, 1148, and 1154, respectively, and Can be stored in advance.

The electronic device 1000 uses the first identification condition 1152 to determine a probability that the contour detected from each of the plurality of guide regions corresponds to the object, and determines a contour whose probability of corresponding to the object is greater than a preset threshold. At least one partial contour may be detected from at least one included guide area. For example, the electronic device 1000 may detect a first partial contour and a second partial contour from at least one guide area. The electronic device 1000 may determine the second identification condition 1156 by using information about the relative positions of the partial contours based on the detected coordinate information of the first partial contour and the coordinate information of the second partial contour. For example, the electronic device 1000 may determine a second identification condition by using information about a relative position between coordinate information of a first partial contour and coordinate information of a second partial contour detected from at least one guide area. have.

According to an embodiment, the second identification condition 1156 is a condition for the position of the vertical nose contour and the nostril contour, the condition for the position range of both nostril contours, and the position range of the vertical nose contour and the nostril contour. The condition, a condition for an overlapping range of the vertical nose contour and the nostril contour, and a condition for the maximum size of the nostril contour may be included, but are not limited thereto.

According to another embodiment, the second identification condition 1156 is determined whether the largest Y-axis component value among the coordinate information of the second partial contour is less than the largest Y-axis component value among the coordinate information of the first partial contour. A first partial identification condition regarding, a second partial identification condition regarding whether the center line of the first partial contour and the second partial contour overlap, and regarding whether the total area of the first partial contour is less than or equal to a preset threshold area It may also include a third part identification condition.

For example, the electronic device 1000 may provide a first partial outline for the nostril of an animal and a guideline for the nose of the animal from a guide region in which the probability of corresponding to an object by the identification process is greater than or equal to a preset threshold. 2 It is possible to detect a couple's contour and determine coordinate information of the first couple's contour and the second partial contour. The electronic device 1000 includes the largest Y-axis component of the coordinate information of the second partial contour (eg, the contour of the nose line) and the largest Y-axis of the coordinate information of the first partial contour (eg, the contour of the nostril). By comparing the component values, it is possible to identify whether the vertical nose line in the image is located below the nostril.

According to another implementation, the electronic device 1000 determines whether the first partial contour is located within a preset first area range in the image or based on the coordinate information of the second partial contour based on the coordinate information of the first partial contour. Thus, it is possible to identify whether the second partial outline is located within a preset second area range in the image. Based on the coordinate information of the first partial contour, the electronic device 1000 may determine that the first partial contour is located in a preset first area range in the image, and the second partial contour is based on the coordinate information of the second partial contour. When located within the preset second area range, an object within the guide area can be identified with a nose.

According to another embodiment, the electronic device 1000 includes a value of a rightmost X-axis component value of a left nostril contour among a first partial contour, and a right side of the first partial contour, based on coordinate information of the first partial contour. It can be identified whether the value of the leftmost X-axis component of the nostril contour is not included in the X-axis component range of the midline of the nose in the second partial contour. The electronic device 1000 includes a value of the rightmost X-axis component of the left nostril contour among the first partial contours, and the leftmost X-axis component value of the right nostril contour among the first partial contours. If it is not within the range of the X-axis component of the gravitational line, the object in the guide area can be identified with the nose.

According to another embodiment, the electronic device 1000 may determine an area where the first partial contour and the second partial contour overlap, based on coordinate information of the first partial contour and coordinate information of the second partial contour. The electronic device 1000 identifies whether the area where the first partial contour and the second partial contour overlap is within a preset range, and when the area where the first partial contour and the second partial contour overlap is within the preset range, the guide Objects in the area can be identified by nose.

According to another embodiment, the electronic device 1000 may determine the area of the first partial contour based on coordinate information of the first partial contour, and only when the determined area of the first partial contour is within a preset range, the guide The type of object in the area can be identified with a nose.

According to an embodiment, when the first partial contour and the second partial contour detected from the guide area do not satisfy even one of the second identification conditions, the electronic device 1000 cannot identify the object in the guide area with a nose. May be. However, according to another embodiment, when the first partial contour and the second partial contour satisfy both the second identification condition 1156, the electronic device 1000 may identify an object in the guide area with a nose.

In addition, according to an embodiment, the electronic device 1000 may determine a score for each item of the second identification condition based on coordinate information of the first partial contour and the second partial contour. The electronic device 1000 determines a score for each item of the partial identification conditions included in the second identification condition 1156 for the first partial contour and the second partial contour, and scores for each item of the determined second identification condition. Feature information can also be determined by weighting the values. The electronic device 1000 may identify an object in the image with a nose only when the final score value indicated by the feature information is equal to or greater than a preset threshold.

12 is a block diagram of an electronic device according to an exemplary embodiment.

13 is a block diagram of an electronic device according to another embodiment.

As shown in FIG. 12, the electronic device 1000 may include a processor 1300, a memory 1700, a camera module 1610, and a display 1210. However, not all of the illustrated components are essential components. The electronic device 1000 may be implemented by more components than the illustrated components, or the electronic device 1000 may be implemented by fewer components.

For example, as shown in FIG. 13, the electronic device 1000 according to an embodiment includes a user input interface 1100 in addition to the processor 1300, the memory 1700, the camera module 1610, and the display 1210. ), an output unit 1200 further including a sound output unit 1220 and a vibration motor 1230, a sensing unit 1400, a network interface 1500, an A/V input unit 1600, a fastening unit (not shown) It may further include.

The user input interface 1100 refers to a means for a user to input a sequence for controlling the electronic device 1000. For example, the user input interface 1100 includes a key pad, a dome switch, and a touch pad (contact capacitive method, pressure resistive film method, infrared sensing method, surface ultrasonic conduction method, red A method for measuring a powdered tension, a method for measuring a piezo effect, etc.), a jog wheel, a jog switch, etc. may be provided, but the present invention is not limited thereto. The user input interface 1100 may receive a user input sequence for a screen output on the display by the electronic device 1000. In addition, the user input interface 1100 may receive a touch input of a user who touches the display or a key input through a graphic user interface on the display.

The output unit 1200 may output an audio signal, a video signal, or a vibration signal, and the output unit 1200 may include a display unit 1210, an audio output unit 1220, and a vibration motor 1230. have.

The display 1210 includes a screen for displaying and outputting information processed by the electronic device 1000. In addition, the screen may display a guide message for allowing a user to photograph an animal image and at least one guide area. The sound output unit 1220 outputs audio data received from the network interface 1500 or stored in the memory 1700. Also, the sound output unit 1220 outputs sound signals related to functions performed by the electronic device 1000. The vibration motor 1230 may output a vibration signal. For example, the vibration motor 1230 may output a vibration signal corresponding to the output of functions performed by the electronic device 1000.

The processor 1300 generally controls the overall operation of the electronic device 1000. For example, the processor 1300, by executing programs stored in the memory 1700, the user input unit 1100, the output unit 1200, the sensing unit 1400, the communication unit 1500, the A/V input unit 1600 ) And so on. Also, the processor 1300 may perform functions of the electronic device 1000 illustrated in FIGS. 1 to 11 by executing programs stored in the memory 1700.

Specifically, the processor 1300 may obtain an input of a user who touches the screen of the electronic device by controlling the user input unit. According to an embodiment, the processor 1300 may control the microphone to acquire a user's voice. The processor 1300 may acquire images including the object based on a user input, and if the object in the image is not identified, the processor 1300 may obtain an image including the object again based on a user input re-received from the user. I can.

According to an embodiment, the processor 1300 acquires an image including the object by executing the one or more instructions stored in a memory, determines at least one guide area corresponding to the object in the acquired image, and , From the at least one guide area, feature information for identifying the object may be determined, and the type of the object in the image may be identified using the determined feature information. Also, when the type of the identified object is identified as an animal's nose or partial objects related to the animal's nose, the processor 1300 may store at least one pixel value in an image including the object.

According to another embodiment, the processor 1300 identifies the type of the object in the image by inputting the determined feature information into an artificial intelligence model, and the type of the identified object is an animal's nose or the animal When identified as partial objects related to the nose, at least one pixel value in the image including the object may be stored. According to an embodiment, the artificial intelligence model may include an artificial neural network model, and the artificial neural network model includes an input layer, an output layer, and one or more hidden layers between the input layer and the output layer, and the input layer , It may be a feedforward network in which a weight related to a connection strength between an output layer and the hidden layers is updated.

In addition, according to an embodiment, the processor 1300 determines a first guide area corresponding to the object to include a preset reference pixel in the acquired image by executing one or more instructions, and the determined first guide By scaling an area according to a preset scale, at least one second guide area corresponding to the object may be determined, and feature information for each of the first guide area and the at least one second guide area may be determined.

In addition, according to an embodiment, the processor 1300 may control the display so that the first guide area is displayed on the display and the at least one second guide area is not displayed on the display.

Also, when the feature information extracted from at least one guide area is input, the processor 1300 may identify the type of the object using artificial intelligence that outputs a probability value for the type of the object. .

According to an embodiment, the processor 1300 identifies a type of an object for each of the first guide area and the at least one second guide area, and among the identified object types, the processor 1300 is identified as an object of the same type. The frequency of the guide area may be determined, and the type of the object in the image may be identified based on the determined frequency.

According to an embodiment, the processor 1300 removes Gaussian noise in the acquired image by applying a Gaussian kernel representing a Gaussian distribution to pixel values in the acquired image, and removes the Gaussian noise. The histogram of values may be equalized, and the at least one guide area in the image in which the histogram is smoothed may be determined.

In addition, the processor 1300 detects a contour of the object for each of the at least one guide area, determines a probability that the detected contour corresponds to the object, and includes a contour having the determined probability greater than a preset threshold. The characteristic information may be determined from the at least one guide area.

In addition, the processor 1300 determines a first identification condition for identifying the object using the detected contour components in the contour, and applies different weights to the determined first identification condition, so that the at least one guide It is possible to determine the probability that the contour within the region corresponds to the object.

In addition, the processor 1300 divides the histogram-smoothed images into at least one partial region of a preset size, determines threshold values for each partial region based on pixel values of the divided partial regions, and The pixel values in the partial regions may be binarized based on the determined threshold values, and an outline of the object may be detected for each of the at least one guide region in an image including the binarized pixel values.

In addition, the processor 1300 provides a first partial contour for the nostril of the animal and a first partial contour for the animal's nasal midline from the at least one guide region including an outline having a probability corresponding to the object greater than a preset threshold. 2 Detect the partial contour, determine the detected coordinate information of the first partial contour and the coordinate information of the second partial contour, based on the determined coordinate information of the first partial contour and the coordinate information of the second partial contour Thus, the characteristic information can be determined.

In addition, the processor 1300 determines a second identification condition with respect to the detected first partial contour and the relative position of the detected second partial contour, and coordinate information of the first partial contour and the second partial contour. A score value for each item of the second identification condition may be determined based on coordinate information, and the feature information may be determined based on a score value for each item of the determined second identification condition.

The sensing unit 1400 may detect a state of the electronic device 1000 or a state around the electronic device 1000 and transmit the sensed information to the processor 1300. The sensing unit 1400 includes a magnetic sensor 1410, an acceleration sensor 1420, a temperature/humidity sensor 1430, an infrared sensor 1440, a gyroscope sensor 1450, and a position sensor. (For example, a GPS) 1460, an atmospheric pressure sensor 1470, a proximity sensor 1480, and an RGB sensor 1490 may be included, but are not limited thereto. Since the function of each sensor can be intuitively inferred by a person skilled in the art from its name, a detailed description will be omitted.

The network interface 1500 may include one or more components that allow the electronic device 1000 to communicate with other devices (not shown) and the server 2000. Another device (not shown) may be a computing device such as the electronic device 1000 or a sensing device, but is not limited thereto. For example, the network interface 1500 may include a wireless communication interface 1510, a wired communication interface 1520, and a mobile communication unit 530.

The wireless communication interface 1510 includes a short-range wireless communication unit, a Bluetooth communication unit, a Near Field Communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared data association (IrDA). ) May include a communication unit, a Wi-Fi Direct (WFD) communication unit, etc., but is not limited thereto. The wired communication interface 1520 may connect the server 2000 or the electronic device 1000 by wire.

The mobile communication unit 1530 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include a voice signal, a video call signal, or various types of data according to transmission and reception of text/multimedia messages.

According to an embodiment, the network interface 1500 may transmit information on an image of an object or information on images composed of a plurality of frames to a server under the control of a processor. In addition, the network interface 1500 may further receive information on a result of recognizing an object in the image from the server.

The A/V (Audio/Video) input unit 1600 is for inputting an audio signal or a video signal, and may include a camera 1610 and a microphone 1620. The camera 1610 may obtain an image frame such as a still image or a video through an image sensor in a video call mode or a photographing mode. The image captured through the image sensor may be processed by the processor 1300 or a separate image processing unit (not shown). For example, the camera module 1610 may acquire an animal image a plurality of times based on a user input.

The microphone 1620 receives an external sound signal and processes it as electrical voice data. For example, the microphone 1620 may receive an acoustic signal from an external device or a user. The microphone 1620 may receive a user's voice input. The microphone 1620 may use various noise removal algorithms for removing noise generated in a process of receiving an external sound signal.

The memory 1700 may store a program for processing and controlling the processor 1300, and may store data input to or output to the electronic device 1000. In addition, the memory 1700 is an image including an object, information on an image, information on at least one guide area displayed on the image or image, and detection from the guide area obtained by the electronic device 1000 based on a user input. Information on the defined contour, information on the identification result of the object for each guide area, a plurality of contour components to determine the probability that the contour for each guide area will correspond to the object, or information on the primary identification condition or the secondary identification condition. You can also save more.

The memory 1700 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), and RAM. (RAM, Random Access Memory) SRAM (Static Random Access Memory), ROM (ROM, Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk And at least one type of storage medium among optical disks.

Programs stored in the memory 1700 may be classified into a plurality of modules according to their functions, for example, a UI module 1710, a touch screen module 1720, a notification module 1730, and the like. .

The UI module 1710 may provide a specialized UI, GUI, etc. that are linked with the electronic device 1000 for each application. The touch screen module 1720 may detect a user's touch gesture on a touch screen and transmit information about the touch gesture to the processor 1300. The touch screen module 1720 according to some embodiments may recognize and analyze a touch code. The touch screen module 1720 may be configured with separate hardware including a controller.

The notification module 1730 may generate a signal for notifying the occurrence of an event of the electronic device 1000. Examples of events occurring in the electronic device 1000 include call signal reception, message reception, key signal input, and schedule notification. The notification module 1730 may output a notification signal in the form of a video signal through the display unit 1210, may output a notification signal in the form of an audio signal through the sound output unit 1220, and the vibration motor 1230 It is also possible to output a notification signal in the form of a vibration signal through.

14 is a block diagram of a server according to an embodiment.

The server 2000 may include a network interface 2100, a database 2200, and a processor 2300. The network interface 2100 may correspond to the network interface 1500 of the electronic device 1000 illustrated in 14. For example, the network interface 2100 may transmit information on a result of recognizing the received image or object in the image, whether it receives information about an image or an image photographing objects from the electronic device 1000. Can be transferred to.

The database 2200 may correspond to the memory 1700 of the electronic device 1000 illustrated in FIG. 14. For example, the database 2200 includes an image including an object, information on an image, information on at least one guide area displayed on an image or image, and a guide acquired by the electronic device 1000 based on a user input. Information on the contour detected from the region, information on the identification result of the object for each guide region, the contour component for determining the probability that the contour for each guide region corresponds to the object, or information on the identification condition may be further stored.

The processor 2300 typically controls the overall operation of the server 2000. For example, the processor 2300 may generally control the DB 2200 and the network interface 2100 by executing programs stored in the DB 2200 of the server 2000. Also, the processor 2300 may perform some operations of the electronic device 1000 in FIGS. 1 to 14 by executing programs stored in the DB 2100. For example, the processor 2300 receives information on an image including an object from the electronic device, determines at least one guide area corresponding to the object in the received image, and identifies the object from at least one guide area. It is possible to determine the feature information to be performed, and to identify whether or not the object in the image is an animal based on the determined feature information. Also, the processor 2300 may transmit an analysis result of the image to the electronic device 1000.

15 is a diagram illustrating a process of identifying an object in an image by interworking with each other between an electronic device and a server, according to an exemplary embodiment.

In S1602, the electronic device 1000 may acquire an image including nose or nose-related partial objects. According to an embodiment, the electronic device 1000 may acquire an image including an animal image by using at least one camera in the electronic device. In S1604, the electronic device 1000 may transmit an image including nose or nose-related partial objects to the server 2000. According to an embodiment, the electronic device 1000 may transmit an image including an animal or information on an image to the server 2000.

In S1606, the server 2000 may determine at least one guide area corresponding to an object in the image. For example, the server 2000 may determine the second guide areas by determining a first guide area in the image and scaling the determined first guide area according to a preset scale.

In S1608, the server 2000 may determine feature information for identifying an object from the generated at least one guide area. According to an embodiment, the server 2000 removes the guide area that does not satisfy a predetermined probability value that the detected contour corresponds to the object through the primary object identification process, and the probability that the detected contour corresponds to the object. It is also possible to determine feature information only from these high guide areas.

In S1610, the server 2000 may identify the type of object in the image by inputting the extracted feature information into the pre-learned artificial intelligence model. According to another embodiment, the server 2000 detects a first partial contour and a second partial contour from a guide region in which a probability value corresponding to the nose is equal to or greater than a preset threshold among the initially generated guide regions, and The second identification condition may be determined based on the partial contour and coordinate information of the second partial contour. The server 2000 generates feature information by weighting scores for each item of the second identification condition for the first partial contour and the second partial contour based on the coordinate information of the first partial contour and the second partial contour, An object in the guide area may be identified based on the score value indicated by the generated feature information.

In S1612, when the type of the identified object is identified as an animal's nose or partial objects related to the animal's nose, the server 2000 may store at least one pixel value in the image including the object in the database. In S1614, the server 2000 may transmit information on the result of recognizing an object in the image to the electronic device 1000 by analyzing the image. In S1616, the electronic device 1000 may output information on a result of recognizing an object in an image received from the server.

The method according to an embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded in the medium may be specially designed and configured for the present disclosure, or may be known and usable to those skilled in computer software.

In addition, a computer program device including a recording medium storing a program for performing another method may be provided in the above embodiment. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

Although the embodiments of the present disclosure have been described in detail above, the scope of the present disclosure is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concept of the present disclosure defined in the following claims are also included in the present disclosure. It belongs to the scope of rights.

Claims (15)

In an electronic device for identifying an object in an image,
display;
At least one camera;
A memory for storing one or more instructions; And
At least one processor executing the one or more instructions; Including,
The processor executes the one or more instructions,
Obtaining an image including the object,
Determining at least one guide area of different scales corresponding to the object in the acquired image,
Determining feature information for identifying the object from the at least one guide area,
As a result of identifying the type of the object for each of the at least one guide area of the different scale based on the determined characteristic information, the type of the object in the image is identified based on the frequency of the guide area identified as an object of the same type. and,
When the type of the identified object is identified as an animal's nose or partial objects related to the animal's nose, the electronic device stores at least one pixel values in an image including the object. The method of claim 1, wherein the at least one processor
Determining a first guide area corresponding to the object to include a preset reference pixel in the obtained image,
Determining at least one second guide area corresponding to the object by scaling the determined first guide area according to a preset scale,
The electronic device comprising determining feature information for each of the first guide area and the at least one second guide area. The method of claim 2, wherein the at least one processor
The electronic device according to claim 1, wherein the display is controlled so that the first guide area is displayed on the display and the at least one second guide area is not displayed on the display. The method of claim 1, wherein the at least one processor
When the feature information extracted from the at least one guide area is input, the type of the object is identified using artificial intelligence that outputs a probability value for the type of the object. The method of claim 2, wherein the at least one processor
Identifying a type of an object for each of the first guide area and the at least one second guide area,
Among the types of the identified objects, the frequency of the guide area identified as an object of the same type is determined, and
The electronic device, characterized in that to identify the type of the object in the image based on the determined frequency. The method of claim 1, wherein the at least one processor
By applying a kernel to pixel values in the acquired image, noise in the acquired image is removed,
Converting pixel values in the image from which the noise has been removed,
The electronic device, wherein the pixel values determine the at least one guide area in the converted image. The method of claim 6, wherein the at least one processor
Detecting an outline of the object for each of the at least one guide area,
Determine a probability that the detected contour corresponds to the object,
The electronic device, wherein the characteristic information is determined from the at least one guide area including an outline having the determined probability greater than a preset threshold. The method of claim 7, wherein the at least one processor
Determine a first identification condition for identifying the object by using the detected contour components in the contour,
The electronic device comprising: determining a probability that the outline in the at least one guide area corresponds to the object by applying different weights to the determined first identification condition. The method of claim 7, wherein the at least one processor
Dividing the images converted from the pixel values into at least one partial region of a preset size,
Based on the pixel values of the divided partial regions, threshold values are determined for each partial region,
Binarizing pixel values in the partial regions based on the determined threshold values,
An electronic device configured to detect an outline of the object for each of the at least one guide area in the image including the binarized pixel values. The method of claim 8,
The contour components include a height of the detected contour, a width of the contour, an area of the detected contour, an area of an outermost contour among the detected contours, and pixels constituting the contour, and the first identification condition is The area of the detected contour, the coordinate of the detected contour, the ratio of the height of the detected contour and the width of the contour, the area of the outermost contour, and the ratio of the area within the contour excluding the area of the outermost contour It characterized in that it comprises, electronic device. The method of claim 8, wherein the at least one processor
From the at least one guide area including a contour having a probability corresponding to the object greater than a preset threshold, a first partial contour of the animal's nostril and a second partial contour of the animal's nasal midline are detected, ,
Determine coordinate information of the detected first partial contour and coordinate information of the second partial contour,
An electronic device that determines the feature information based on the determined coordinate information of the first partial contour and coordinate information of the second partial contour. The method of claim 11, wherein the at least one processor
Determining a second identification condition with respect to the detected first partial contour and the relative position of the detected second partial contour,
Determine a score value for each item of the second identification condition based on the coordinate information of the first partial contour and the coordinate information of the second partial contour,
An electronic device that determines the feature information based on the determined score value for each item of the second identification condition. The method of claim 12, wherein the second identification condition is
The first partial identification condition regarding whether the largest Y-axis component value among the coordinate information of the second partial contour is less than the largest Y-axis component value among the coordinate information of the first partial contour, the first partial contour and the A second partial identification condition as to whether or not the center line of the second partial contour overlaps, and a third partial identification condition as to whether the total area of the first partial contour is less than or equal to a preset threshold area. Device. The method of claim 4,
The artificial intelligence model includes an input layer, an output layer, and one or more hidden layers between the input layer and the output layer, and the weight of the connection strength between the input layer, the output layer, and the hidden layers is updated. An electronic device comprising a neural network model or a machine learning model that is learned according to an artificial intelligence learning algorithm. In a method for an electronic device to identify an object in an image,
Obtaining an image including the object;
Determining at least one guide area of different scales corresponding to the object in the acquired image;
Determining feature information for identifying the object from the at least one guide area;
Identifying a type of an object in the image using the determined feature information; And
Storing at least one pixel value in an image including the object when the type of the identified object is identified as an animal's nose or partial objects related to the animal's nose; Including,
Identifying the type of the object,
Determining a frequency of a guide area identified as an object of the same type among the types of the identified object as a result of identifying an object for each of the at least one guide area of the different scale; And
Identifying the type of the object in the image based on the determined frequency; The method further comprising.

Images