KR102243466B1 - Method and apparatus for detecting eye of animal in image - Google Patents

Abstract

The present disclosure relates to a method for an electronic device to identify an object in an image. According to one embodiment, the method for an electronic device to identify an object in an image comprises the steps of: obtaining an image including an object; determining at least one guide area corresponding to the object in the obtained image; extracting a feature vector for identifying the object from the at least one guide area; identifying the type of object in the image by inputting the extracted feature vector into an artificial intelligence model; and, when the type of the identified object is identified as an animal's eye or partial objects related to the animal's eye, storing at least one pixel value in the image including the object.

Description

Method and apparatus for detecting the eye area of an animal in an image {METHOD AND APPARATUS FOR DETECTING EYE OF ANIMAL IN IMAGE}

The present disclosure relates to an electronic device and method for identifying an object in an image. More particularly, it relates to an electronic device and method for identifying an object from an image including at least one object.

As information and communication technology develops, technologies for identifying objects included in an image including a plurality of frames are being developed. In particular, technologies are being developed that allow the electronic device to identify an object in an image by itself or to identify a predetermined object set in advance by applying a human recognition method to an electronic device.

In addition, based on recent image recognition technology, biometric information of a person is acquired from an image, and technology using biometric authentication is actively progressing as an authentication method for banking, shopping, and unlocking using the obtained biometric information. Has become. For example, as a representative biometric authentication method, iris recognition through the image of the eye area detected in the image, face recognition through the image of the face area detected in the image, the hand fingerprint image detected in the image is analyzed, and the result of the analysis Technology for performing fingerprint recognition on the basis of is being developed.

However, since authentication technology using biometric information requires high accuracy due to authentication characteristics, there is still a need to develop a technology for accurately authenticating an object through biometric information. In addition, development of biometric authentication technology targeting animals as well as biometric authentication technology targeting humans is required.

Korean Patent Publication No. 10-2018-0109665

According to an embodiment, an electronic device and method for identifying an object in an image may be provided.

In addition, according to an embodiment, an electronic device may be provided in which the electronic device identifies a type of an object in an image and stores at least one pixel value in the image based on the type of the identified object.

According to an embodiment of the present disclosure for achieving the above-described technical problem, a method of identifying an object in an image by an electronic device includes: obtaining an image including the object; Determining at least one guide area corresponding to the object in the acquired image; Extracting a feature vector for identifying the object from the at least one guide area; Identifying a type of object in the image by inputting the extracted feature vector into an artificial intelligence model; And when the type of the identified object is identified as an animal eye or partial objects related to the animal eye, storing at least one pixel value in an image including the object. It may include.

In addition, according to another embodiment of the present disclosure for solving the above technical problem, an electronic device for identifying an object in an image includes a display; At least one camera; A memory for storing one or more instructions; And at least one processor executing the one or more instructions. Including, wherein the processor executes the one or more instructions to obtain an image including the object, determine at least one guide area corresponding to the object in the obtained image, and the at least one guide area A feature vector for identifying the object is extracted from and the extracted feature vector is input to an artificial intelligence artificial intelligence model to identify the type of object in the image, and the identified object type is the animal's eye or the animal. When identified as partial objects related to the eye of the person, at least one pixel value in the image including the object may be stored.

In addition, according to another embodiment of the present disclosure for solving the technical problem, a method for identifying an object in an image by an electronic device, the method comprising: acquiring an image including the object; Determining at least one guide area corresponding to the object in the acquired image; Extracting a feature vector for identifying the object from the at least one guide area; Identifying the type of object in the image by inputting the extracted feature vector into an artificial intelligence artificial intelligence model; And when the type of the identified object is identified as an animal eye or partial objects related to the animal eye, storing at least one pixel value in an image including the object. A computer-readable recording medium storing a program for executing a method on a computer including, may be provided.

According to an embodiment of the present disclosure, it is possible to accurately identify the type of object in an image.

In addition, according to an embodiment, by using a plurality of guide areas according to different scales, it is possible to accurately identify the type of object in the image.

1 is a diagram schematically illustrating a process of identifying an object in an image by an electronic device according to an exemplary embodiment.
2 is a flowchart illustrating a method of identifying an object in an image by an electronic device according to an exemplary embodiment.
3 is a flowchart illustrating a method of determining at least one guide area according to a plurality of scales by an electronic device according to an exemplary embodiment.
4 is a diagram illustrating a method of determining at least one guide area by an electronic device according to an exemplary embodiment.
5 is a flowchart illustrating a method of identifying, by an electronic device, a type of an object in an image based on an object identification result for each guide area, according to an exemplary embodiment.
FIG. 6 is a diagram illustrating a method of identifying a type of an object in an image based on a result of identifying an object for each guide area by an electronic device according to an exemplary embodiment.
7 is a diagram illustrating a method of preprocessing an image in order to remove at least one noise in an image by an electronic device according to an exemplary embodiment.
FIG. 8 is a diagram illustrating a process of extracting a feature vector from some of at least one guide area generated by an electronic device according to an exemplary embodiment.
9 is a diagram for describing a process of detecting an outline of an object from at least one guide area by an electronic device according to an exemplary embodiment.
10 is a flowchart illustrating a detailed method of detecting an outline from an image in at least one binarized guide area by an electronic device according to an exemplary embodiment.
FIG. 11 is a reference diagram for explaining a specific method of detecting an outline from an image in at least one binarized guide area by an electronic device according to an exemplary embodiment.
12 is a diagram illustrating an identification condition used by an electronic device to determine a probability that an object detected in a guide area corresponds to an eye, according to an exemplary embodiment.
13 is a block diagram of an electronic device according to an exemplary embodiment.
14 is a block diagram of an electronic device according to another embodiment.
15 is a block diagram of a server according to an embodiment.
16 is a diagram for describing a process of identifying an object in an image by interworking with each other between an electronic device and a server, according to an embodiment.

The terms used in the present specification will be briefly described, and the present disclosure will be described in detail.

Terms used in the present disclosure have selected general terms that are currently widely used as possible while considering functions in the present disclosure, but this may vary according to the intention or precedent of a technician engaged in the art, the emergence of new technologies, and the like. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning of the terms will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the contents of the present disclosure, not the name of a simple term.

When a part of the specification is said to "include" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary. In addition, terms such as "... unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. .

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present disclosure, and similar reference numerals are attached to similar parts throughout the specification.

1 is a diagram schematically illustrating a process of identifying an object in an image by an electronic device according to an exemplary embodiment.

According to an embodiment, the electronic device 1000 may acquire an image including at least one object, identify an object area within the acquired image, and identify a type of an object represented by the identified object area. According to an embodiment of the present disclosure, the electronic device 1000 may acquire an animal image including an animal by capturing an image including an animal using at least one camera.

According to an embodiment, the object identified by the electronic device 1000 may include a part of body parts of an animal. For example, the object identified by the electronic device 1000 may include at least one of the eye of an animal or partial objects related to the eye of the animal. According to an embodiment, the partial objects according to the present disclosure may include at least one of eyebrows, pupils, whites, and animal hairs positioned around the eye area related to the eyes of the animals. The electronic device 1000 according to the present disclosure detects a partial object region including eyebrows, pupils, and whites as an animal eye region in an animal image or a region for partial objects related to the animal's eyes, and within the detected region. You can identify the type of object or partial object.

According to an embodiment, the electronic device 1000 may be implemented as various types of devices including a communication module capable of communicating with the server 2000. For example, the electronic device 1000 includes a digital camera, a mobile terminal, a smart phone, a laptop computer, a tablet PC, an e-book terminal, a digital broadcasting terminal, personal digital assistants (PDA), and a PMP ( Portable Multimedia Player), navigation, MP3 player, and the like, but are not limited thereto. In addition, according to an embodiment, the electronic device 1000 may be a device that can be worn by a user.

For example, wearable devices include accessory type devices (e.g., watches, rings, wrist bands, ankle bands, necklaces, glasses, contact lenses), head mounted display devices (HMD), textile or clothing integrated devices ( Examples: electronic clothing), a body-attached device (eg, a skin pad), or a living body implantable device (eg, an implantable circuit) may be included, but is not limited thereto.

According to an embodiment, the server 2000 may include other computing devices capable of transmitting and receiving data to and from the electronic device by being connected to the electronic device 1000 through the network 3000. In addition, according to an embodiment, the server device 2000 may be a Wearable Business Management Server (W-BMS) for managing wearable devices. According to an embodiment, the network 3000 includes a local area network (LAN), a wide area network (WAN), a value added network (VAN), a mobile radio communication network, and It is a data communication network in a comprehensive sense that includes a satellite communication network and a mutual combination thereof, and enables each network component shown in FIG. 1 to communicate with each other smoothly, and includes wired Internet, wireless Internet, and mobile wireless communication network. I can.

According to an embodiment, the electronic device 1000 acquires an animal image based on a user's input, displays the acquired animal image, and displays at least one guide area 114 among a plurality of guide areas. It can be displayed on the screen 102. In addition, according to an embodiment, while the user is capturing an animal image, the electronic device 1000 may provide a guide message 112 such as'Please take a picture of the dog's left eye' or'right side of the dog' for convenience. A guide message 116 such as'Please take a picture of your eyes' may be further provided on the screen 104.

The electronic device 1000 according to the present disclosure identifies the type of object photographed in the guide area 114, and when the type of the identified object matches the preset type of object, photographs the body of another animal in the animal image. You can print out a guide message for doing so. According to an embodiment, the electronic device 1000 identifies the type of the object included in the guide area 114 in the image acquired based on the user's input, and the type of the identified object coincides with a preset type of object. In this case, the guide area 114 may be displayed by blinking according to a preset period or by changing the color of the currently displayed guide area 114 to indicate to the user that the image has been accurately captured.

2 is a flowchart illustrating a method of identifying an object in an image by an electronic device according to an exemplary embodiment.

In S210, the electronic device 1000 may acquire an image including an object. For example, the electronic device 1000 may acquire an image using at least one camera included in the electronic device. After obtaining the first image using at least one camera, the electronic device 1000 may obtain a second image of the next frame. That is, the electronic device 1000 may acquire an image including a plurality of images of an object.

In S220, the electronic device 1000 may determine at least one guide area corresponding to an object in the acquired image. For example, the electronic device 1000 may generate a first guide area and generate a plurality of second guide areas by scaling the generated first guide area according to a preset scale. According to an embodiment, the electronic device 1000 may display the first guide area on the screen of the electronic device, and may not display the second guide area on the screen of the electronic device.

According to an embodiment, the electronic device 1000 may determine the first guide area according to a preset position. According to another embodiment, the electronic device 1000 acquires a sensor value according to a user's movement through at least one sensor included in the electronic device, and calculates the coordinates of the first guide area based on the acquired sensor value. You can also decide.

In S230, the electronic device 1000 may extract a feature vector for identifying an object from at least one guide area. For example, the electronic device 1000 may determine a gradient of pixels in an image corresponding to the guide area, and extract a feature vector using the determined gradient. However, the process of extracting the feature vector by the electronic device 1000 is not limited thereto, and the feature vector may be extracted using another feature vector extraction algorithm that extracts the feature vector based on the amount of change in pixel values.

According to an embodiment, the electronic device 1000 may generate a histogram for each partial area in the image by using the gradient of pixels in the image. The electronic device 1000 may extract a feature vector for identifying an object from the guide area by linking bin values of the histogram generated for each partial area.

Also, according to an embodiment, before extracting the feature vector, the electronic device 1000 may change the size of the image to a predetermined size by pre-processing the images to which the feature vector is to be extracted. The electronic device 1000 may extract a feature vector from an image changed to a predetermined size, and identify an object in the image using the extracted feature vector.

According to an embodiment, a feature vector extracted from an image by the electronic device 1000 may be a descriptor for expressing a feature of an image. According to an embodiment, the feature vector extracted from the image by the electronic device 1000 includes feature values according to at least one of a Histogram of Oriented Gradients (HOG) feature, a Scale Invariant Feature Transform (SIFT) feature, a Haar feature, and a Ferns feature. Can include. However, the present disclosure is not limited thereto, and the electronic device according to the present disclosure may use a feature extraction algorithm using other pixel values.

For example, the electronic device 1000 may determine at least one partial region by dividing an image corresponding to the guide region. The electronic device 1000 may determine edges of pixels for each determined partial region, and determine an x-axis component and a y-axis component of the determined edges. In addition, the electronic device 1000 may determine an orientation of an edge using x-axis components and y-axis components of the determined edges for each partial region, and generate a histogram in which the directions of the determined edges are a category. The electronic device 10000 may determine a feature vector by connecting bin values of the histogram generated for each partial region with each other.

In S240, the electronic device 1000 may identify the type of object in the image by inputting the extracted feature vector into an artificial intelligence model. According to an embodiment, in the artificial intelligence model used by the electronic device 1000, when a feature vector is input, the type of the object is defined by a predefined label (e.g., left eye, right eye, nose, eyebrow, mouth). Etc.) Probability values can be output. According to another embodiment, when a feature vector is input, the artificial intelligence model used by the electronic device 1000 may output a label corresponding to the input feature vector with the highest probability as the type of object.

The artificial intelligence model used by the electronic device 1000 may include an artificial neural network model formed in a deep neural network structure having multiple layers, and a machine learning model that is learned according to other artificial intelligence learning algorithms. According to an embodiment, the artificial neural network model used by the electronic device is a convolutional neural network, a deep neural network (DNN), a recurrent neural network (RNN), a bidirectional recurrent deep neural network (BRDNN), and a multilayer perceptron (MLP). ) May be included, but is not limited thereto.

According to an embodiment, the neural network used by the electronic device 1000 may be a multilayer perceptron (MLP). The connection between the perceptrons of the adjacent layer exists, but the connection between the perceptrons of the same layer does not exist. It may be an artificial intelligence in the form of FeedForward. According to another embodiment, the artificial intelligence used by the electronic device 1000 may be a structure in which a fully-connected layer is connected to a CNN structure in which a convolutional layer and a pooling layer are repeatedly used.

In S250, when the type of the identified object is identified as an animal's eye or a partial object related to the animal's eye, the electronic device 1000 may store at least one pixel value in the image including the object. For example, the electronic device 1000 acquires an image of an animal according to a user input in response to an output of a guide message such as'Please take a picture of the left eye', and the type of the object in the acquired image is the left side of the animal. When it is identified by eyes, information on the currently acquired image may be stored in a memory of the electronic device.

According to an embodiment, when the type of the identified object is identified as an animal's eye or a partial object related to the animal's eye, the electronic device 1000 does not store information on the entire image, but stores the identified object. It is also possible to store only information on pixels in the included guide area. In addition, when the type of the identified object is identified as an animal eye or partial objects related to the animal eye, the electronic device 1000 not only stores at least one pixel value in the image including the object, but also stores at least one stored object. The type of the identified object may be further stored by matching the pixel values of.

3 is a flowchart illustrating a method of determining at least one guide area according to a plurality of scales by an electronic device according to an exemplary embodiment.

In S310, the electronic device 1000 may determine a first guide area corresponding to the object to include a preset reference pixel in the image. For example, the electronic device 1000 may determine coordinates of a reference pixel based on the size of the screen of the electronic device, and may determine a first guide area to include a reference pixel positioned at the determined coordinates.

According to another embodiment, the electronic device 1000 includes a current direction of the electronic device and a current position of the electronic device and a location of a photographing target based on a sensing value acquired through at least one sensor included in the electronic device. Position information indicating a relative relationship may be determined, and a first guide area may be determined based on the direction and position information. In addition, according to an embodiment, the electronic device 1000 may determine a first guide area having a rectangular shape having a line segment extending from a position of a reference pixel by a predetermined width and height. According to an embodiment, the first guide region may be formed in a square shape, but is not limited thereto.

In S320, the electronic device 1000 may determine at least one second guide area corresponding to an object by scaling the first guide area according to a preset scale. For example, the electronic device 1000 determines the scaling factor to be 1.1, and applies the determined scaling factor to the horizontal length in the first guide area, thereby increasing the horizontal length by 10% than the horizontal length of the first guide area. The branches may determine the second guide area. Also, the electronic device 1000 may determine a second guide region having a vertical length that is 10% greater than the vertical length of the first guide region by applying a scaling factor of 1.1 to the vertical length of the first guide region.

4 is a diagram illustrating a method of determining at least one guide area by an electronic device according to an exemplary embodiment.

As described above in FIG. 3, after determining the first guide area 416, the electronic device 1000 scales the determined first guide area 416 according to a preset scale, so that at least one of the second guide areas ( 412, 414, 418, 422) can be created. According to an embodiment, the first guide areas 416 may be displayed on the screen of the electronic device, but the second guide areas may not be displayed on the screen of the electronic device. The electronic device 1000 may not display the second guide areas generated based on the first guide area on the screen, but may use only the calculation process for identifying an object in the image.

For example, the electronic device 1000 determines the lower left corner of the first guide area 416 as the coordinates of the reference pixel, and extends from the determined reference pixel position by a predetermined length of the horizontal width 402 The first guide area 416 including the horizontal line segment and the vertical line segment extended by the length of the vertical height 404 may be displayed on the screen. The user of the electronic device can easily determine the area to be photographed based on the screen of the electronic device displayed in the first guide area 416.

In addition, according to an embodiment, the electronic device 1000 determines a scaling factor for increasing or decreasing at least one of a horizontal length or a vertical length of the first guide area, and determines the determined scaling factor as the first guide area. By applying to, it is possible to determine the second guide areas. According to an embodiment, the electronic device 1000 determines two second guide areas 412 and 414 having a larger area than the first guide area, and a second guide area having a narrower area than the first guide area. Two of them 418 and 422 can be determined.

The electronic device 1000 according to the present disclosure may extract a feature vector from each of the first guide area and the at least one second guide area, and identify an object in the image using the extracted feature vector. However, according to another embodiment, the electronic device 1000 according to the present disclosure determines a probability value corresponding to an object for each of the first guide area and the second guide area, and based on whether the determined probability value is greater than a preset threshold. Thus, it is possible to primarily identify objects for the guide areas. The electronic device 1000 according to the present disclosure determines that objects for guide areas having a probability value smaller than a preset threshold are not identified, and artificially only feature vectors extracted from guide areas having a probability value larger than a preset threshold. Enter into the intelligence model.

Therefore, the electronic device 1000 according to the present disclosure does not perform a neural network operation on all guide regions in which the first image is generated, but after first identifying an object according to a preset criterion, the electronic device 1000 is not identified as an object. By not performing a neural network operation on the guide regions, it is possible to increase the operation efficiency of the electronic device. A method of identifying the type of the object in the entire image by the electronic device 1000 based on the object identification value determined for each of the plurality of guide areas will be described in more detail with reference to FIGS. 5 to 6 to be described later.

5 is a flowchart illustrating a method of identifying, by an electronic device, a type of an object in an image based on an object identification result for each guide area, according to an exemplary embodiment.

In S520, the electronic device 1000 identifies a type of an object for each of the first guide area and at least one second guide area. For example, the electronic device 1000 detects a contour of an object for each of the first guide area and the second guide area, and applies different weights to the identification condition determined using the detected contour components. , It is possible to determine a probability that the detected contour for each guide area corresponds to the object. The electronic device 1000 may identify the type of the object for each guide area based on a probability value that the contour detected for each guide area corresponds to the object.

In S540, the electronic device 1000 may determine the frequency of the guide area identified as an object of the same type among the types of the identified object. For example, the electronic device 1000 may calculate the number of guide areas identified as the same type by counting the types of objects identified for each guide area. In S560, the electronic device 1000 identifies the type of the object in the image based on the frequency of the guide area identified as the same type of object. According to an embodiment, the electronic device 1000 may identify the type of the object counted as the largest number among the guide areas as the type of the object in the image.

FIG. 6 is a diagram illustrating a method of identifying a type of an object in an image, based on a result of identifying an object for each guide area, by an electronic device according to an exemplary embodiment.

Referring to FIG. 6, an embodiment in which the electronic device 1000 identifies the type of object in the entire image by using the type of the object identified for each guide area will be described. According to an embodiment, an object corresponding to the first guide area (eg, BOX 0) is identified as a nose, and an object corresponding to the second guide area (eg, Box 1) is identified as a nose, For the remaining guide areas (eg, Box 2, 3, 4), as the probability value is determined to be less than a preset threshold, objects corresponding to the remaining guide areas may not be identified. In this case, the electronic device 1000 may identify the type of the object in the entire image as a nose because the frequency of the guide area identified as a nose is the largest as 2. According to an embodiment, when the electronic device 1000 counts the number of guide areas representing the same type of frequency, the frequency of the unidentified guide area may not be counted.

According to another embodiment, an object corresponding to the first guide area (eg, BOX 0) is identified as a left eye, and an object corresponding to the second guide area (eg, Box 1) is indicated by a nose. It is identified, and the object corresponding to the third guide area (for example, Box 2) is identified as a left eye, and for the remaining guide areas (for example, Box 3, 4), the probability value is determined to be less than a preset threshold. Accordingly, objects corresponding to the remaining guide areas may not be identified. In this case, in the electronic device 1000, the frequency of the guide area identified by the left eye is 2, the frequency of the guide area identified by the nose is 1, and the frequency of the guide area in which the object is not identified. Since is 2, the frequency of the guide area identified by the left eye is compared with the frequency of the guide area in which the object is not identified. However, according to an embodiment, since the electronic device 1000 may not use the frequency of the guide area in which the object is not identified for counting calculations, the electronic device 1000 is a guide identified as a left eye. After determining that the frequency of the region is the highest, the type of the object in the image can be identified with the left eye.

According to another embodiment, an object corresponding to the first guide area (eg, BOX 0) is identified as a left eye, and the object corresponding to the second guide area (eg, Box 1) is identified as a right eye. ), and the object corresponding to the third guide area (e.g. Box 2) is identified as a nose, and the remaining guide areas (e.g., Box 3, 4) have a probability value less than a preset threshold. Accordingly, objects corresponding to the remaining guide areas may not be identified. In this case, the electronic device 1000 has the frequency of the guide identified as the left eye of 1, and the right eye Since the frequency of the identified guide is also 1 and the frequency of the guide area identified by the nose is 1, it may not be possible to determine the frequency of the guide area representing the same type of object among the guide areas. When the electronic device 1000 fails to determine the frequency of the guide area representing the same type of object, the electronic device 1000 may determine that the type of the object in the image is not identified. When the type of the object in the image is not identified, the electronic device 1000 may output a character such as'none' on the screen.

According to another embodiment, an object corresponding to the first guide area (eg, BOX 0) is identified as a left eye, and corresponds to the second, 3, and 4 guide areas (eg, Box 1, 2, 3). All of the objects are identified as noses, and an object corresponding to the fifth guide area (eg, Box 4) may be identified as a right eye. In this case, the electronic device 1000 determines that the frequency of the guide area identified by the nose is 3, the frequency of the guide area identified by the left eye is 1, and the frequency of the guide area identified by the right eye is 1 I can. Accordingly, the electronic device 1000 may identify the type of the object in the image as a nose because the frequency of the guide region identified by the nose is the highest as 3.

According to another embodiment, objects corresponding to the first and second guide areas (eg, BOX 0, 1) are identified by nose, and objects corresponding to the third guide area (eg, Box 2) are all right An object identified as a right eye and corresponding to the fourth and fifth guide areas (eg, Box 3 and 4) may be identified as a left eye. In this case, the electronic device 1000 determines that the frequency of the guide area identified by the nose is 2, the frequency of the guide area identified by the right eye is 1, and the frequency of the guide area identified by the left eye is 2. I can. Accordingly, the electronic device 1000 may determine that the frequency of the guide area currently identified by the nose and the frequency of the guide area identified by the left eye are greater than the frequency of the guide area identified by the right eye. However, since the electronic device 1000 determines that the frequency of the guide area identified by the nose and the frequency of the guide area identified by the left eye is equal to 2, which is determined to be greater than the frequency of the guide area identified by the right eye, the type of object in the image It can be determined that it cannot be identified.

7 is a diagram illustrating a method of preprocessing an image in order to remove at least one noise in an image by an electronic device according to an exemplary embodiment.

According to an embodiment of the present disclosure, the electronic device 1000 may pre-process the image in order to remove noise in the acquired image and improve the accuracy of eye detection. According to an embodiment, in S720, the electronic device 1000 may remove noise in the image by applying a predetermined kernel to pixel values in the image.

According to an embodiment, the electronic device 1000 may remove Gaussian noise in an image by applying a Gaussian kernel indicating a Gaussian distribution. For example, noises following a Gaussian distribution may be included in the image acquired by the electronic device 1000, such as noise due to a characteristic of an image sensor that photographed the image. These noises may deteriorate the quality of an image or an image including a plurality of images. The electronic device 1000 may remove Gaussian noise in the image by applying a Gaussian kernel having a preset size, representing a Gaussian distribution, to an image. According to an embodiment, the electronic device 1000 may divide an image into predetermined partial regions and apply a Gaussian kernel to each of the divided partial regions. However, the present invention is not limited thereto, and the electronic device 1000 may remove noise by using other image filters for removing noise in an image.

In S740, the electronic device 1000 may convert pixel values in the image from which noise has been removed. According to an embodiment, when using a Gaussian kernel, the electronic device 1000 may equalize a histogram of pixel values in an image from which Gaussian noise has been removed. For example, the electronic device 1000 may obtain brightness values of an image for each pixel based on pixel values in the image. The electronic device 1000 may generate a histogram of the brightness values in the image by classifying the brightness values of pixels in the image into bins in which the brightness values for each pixel in the image are classified at a predetermined level.

According to an embodiment, the histogram may include a bin including the number of pixels representing a specific brightness value in an image, and the bins may have a predetermined range for classifying brightness values. The electronic device 1000 may smooth the histogram indicating the distribution of brightness values for the image, so that the brightness values in the image are evenly distributed throughout the image. Also, the electronic device 1000 may improve the contrast of the entire image by smoothing the histogram of the image.

However, according to another embodiment, the electronic device 1000 may equalize the intensity of pixels in the image, not the histogram. In addition, the electronic device 1000 may level pixel values in an image by using a method for leveling other known pixel values in addition to the above-described leveling method.

In S760, the electronic device 1000 may divide the images from which pixel values are converted into at least one partial region, and binarize pixel values in the partial regions using threshold values determined for each divided partial region. For example, the electronic device 1000 may convert the image into a gray scale image and may binarize the image using a preset threshold.

However, the electronic device 1000 according to an embodiment of the present disclosure divides an image into one or more partial regions, and determines a threshold value for each of the divided partial regions, so that a threshold value determined differently for each partial region of the image is determined. By using, the pixel values for each partial area can be binarized. According to an embodiment, when the image is converted to gray scale, the electronic device 1000 may binarize the erection value for each pixel in the partial region using a threshold value determined for each partial region. The electronic device 1000 may more accurately detect a contour corresponding to an object from the binarized image by binarizing the image using different threshold values for each partial region. According to another embodiment, the electronic device 1000 may detect an outline of an object from a guide area including a binary brightness value for each couple area.

In addition, according to an embodiment, when the electronic device 1000 uses the histogram smoothing technique, the electronic device 1000 divides the histogram-smoothed images into at least one partial region, and a threshold determined for each divided partial region. It is also possible to binarize pixel values in partial regions using values.

FIG. 8 is a diagram illustrating a process of extracting a feature vector from some of at least one guide area generated by an electronic device according to an exemplary embodiment.

In S820, the electronic device 1000 may generate at least one guide area in the image. S820 may correspond to a process in which the electronic device 1000 determines a first guide area in FIGS. 3 to 4 and creates second guide areas by scaling the determined first guide area according to a preset scale, so detailed description Will be omitted.

In S840, the electronic device 1000 may detect an outline of an object for each of at least one guide area. For example, the electronic device 1000 may detect a contour corresponding to an object in the image by extracting a feature point from a binarized image and identifying a set of points representing the same image value among the extracted feature points. . According to an embodiment, the electronic device 1000 may detect a plurality of contours corresponding to an object in an image, and may determine outermost contours among the detected contours. According to an embodiment, the electronic device 1000 may detect a max contour among contours detected in an image using a convex hull algorithm. In addition, according to an embodiment, the electronic device 1000 detects a contour from at least one binarized guide region, and then smoothes the boundary of the contour by applying an intermediate value filter to the detected contour. You can also do more. The electronic device 1000 may detect the flattened candidate contour as a contour of an object for each of at least one guide area.

In S860, the electronic device 1000 may determine a probability that the detected contour corresponds to the object. For example, the electronic device 1000 may detect an outermost contour for each of at least one guide area including a binarized image area, and determine a probability that each of the detected outermost contours corresponds to an object.

In more detail, the electronic device 1000 may determine contour components related to an outline of at least one guide area, and determine an identification condition for identifying an object using the determined contour components. The electronic device 1000 may determine a probability that contours in at least one guide area correspond to the object by applying different weights to the determined identification condition. For example, the electronic device 1000 uses the contour components of the contour detected in one guide area, such as the height of the contour, the width of the contour, the area of the contour, the pixels constituting the contour, and the coordinate values of the pixels constituting the contour. , It is possible to determine a coordinate value for a boundary of an ellipse including the pixels. Based on contour components, the electronic device 1000 includes the intensity of pixels in the ellipse formed based on the contour, the area of the detected contour, the height of the detected contour and the ratio of the width of the contour, the area of the contour, and the outermost contour. Identification conditions such as the ratio of the area within the contour excluding the area can be determined.

The electronic device 1000 determines the identification condition as described above from the contour detected in one guide area and the contour components constituting the contour, and then points that the detected contours correspond to the identification condition having different weights. Can be determined. The electronic device 1000 allocates points for each identification condition to the contour detected in the guide area, and then weights the allocated points according to different weights assigned to each identification condition, so that the outline within the guide area corresponds to the object. You can determine the probability of becoming.

In S880, the electronic device 1000 may determine whether probability values corresponding to an object determined for each of at least one guide area are equal to or greater than a preset threshold. In addition, the electronic device 1000 may identify which of the at least one guide area is a guide area including an outline larger than a preset threshold. The electronic device 1000 extracts a feature vector only from a guide region including an outline representing a probability value greater than a preset threshold, and inputs the extracted feature vector into a pre-learned artificial intelligence model, thereby performing the object identification process once. You will perform more.

According to an embodiment, the electronic device 1000 may resize the guide regions to a preset size before extracting a feature vector from guide regions including an outline representing a probability value greater than a preset threshold. . The electronic device 1000 may extract a feature vector from at least one guide area resized to a preset size.

The electronic device 1000 according to the present disclosure determines probability values that contours extracted from at least one guide region in the image correspond to an object, and that the object is not identified for a guide region whose determined probabilities are less than a preset threshold. It is determined, by extracting a feature vector only from a guide region whose probability values are larger than a preset threshold, and inputting the extracted feature vector into the artificial intelligence model, thereby reducing the amount of artificial intelligence computation.

That is, the electronic device 1000 first determines a probability value corresponding to the object with respect to the guide regions in the image, and then uses the feature vector extracted from the guide region having a high probability value corresponding to the object, By performing a neural network operation using an artificial intelligence model only for a guide region having a high probability of being matched, it is possible to quickly and accurately identify the type of object in the image.

9 is a diagram for describing a process of detecting an outline of an object from at least one guide area by an electronic device according to an exemplary embodiment.

In S920, the electronic device 1000 may detect a contour corresponding to an object for each guide area by using the contour detection method corresponding to step S840. In S940, the electronic device 1000 may approximate a contour detected for each guide area. According to an embodiment, the electronic device 1000 may remove some contours from among a plurality of contours in the guide area, and may smooth the boundary of the largest contour. In S960, the electronic device 1000 may identify a boundary of the detected contour for each guide area and generate an ellipse adjacent to the boundary of the identified contour. The electronic device 1000 may determine the intensity of pixels in the ellipse formed based on the contour, and may determine a probability that the contour detected in the guide area corresponds to the object based on the intensity of the pixels in the ellipse.

10 is a flowchart illustrating a detailed method of detecting an outline from an image in at least one binarized guide area by an electronic device according to an exemplary embodiment.

In S1020, the electronic device 1000 may generate a rectangular bounding box including the first guide area in the image including the binarized pixel values. For example, the electronic device 1000 may generate a rectangular bounding box by increasing a predetermined height and width from the first guide area.

In S1040, the electronic device 1000 may determine a candidate contour including the object based on the center point of each side of the bounding box. For example, the electronic device 1000 may identify the coordinates of the vertices of the bounding box, and may identify the coordinates of the center point of the bounding box based on the coordinates of the identified vertices. The electronic device 1000 may determine a candidate contour using two adjacent center points among coordinates of the identified center point. According to an embodiment, the candidate contour line generated by the electronic device 1000 may include at least one of an arbitrary straight line or a curve starting from one of two adjacent center points and ending at the other center point.

In S1060, the electronic device 1000 may adjust the candidate contour using adjustment boxes representing different scores based on a distance away from the candidate contour arranged along the candidate contour. For example, the electronic device 1000 may generate a first adjustment box separated by a first distance on both sides based on the candidate contour line, and generate a second adjustment box separated by a second distance on both sides based on the candidate contour line. have. The electronic device 1000 may allocate a first adjustment score to the first adjustment box and a second adjustment score to the second adjustment box.

The electronic device 1000 may determine scores for partial candidate contours within the currently generated candidate contours by using a plurality of adjustment boxes representing different scores. The electronic device 1000 may adjust the entire candidate contours by adjusting the partial candidate contours based on scores for the partial candidate contours.

According to an embodiment, the electronic device 1000 may flatten the adjusted candidate contour line. According to an embodiment, the electronic device 1000 may flatten the adjusted candidate contour using a preset median filter.

In S1080, the electronic device 1000 may detect the adjusted candidate contour as the contour of the object in the guide area. According to an embodiment, the electronic device 1000 may detect the flattened candidate contour as a contour of an object for each guide area.

FIG. 11 is a reference diagram for explaining a specific method of detecting an outline from an image in at least one binarized guide area by an electronic device according to an exemplary embodiment.

The electronic device 1000 may generate a rectangular bounding box 1112 including a first guide area including a binarized image. The electronic device 1000 may identify the coordinates of the vertices of the bounding box 1112 and identify coordinates of the center points 1102, 1104, 1106, and 1108 of the bounding box based on the coordinates of the identified vertex. According to an embodiment, the electronic device 1000 may generate an arbitrary arc by connecting two adjacent center points among the identified center points. For example, the electronic device 1000 may detect an arbitrary candidate contour 1116 using the center point 1102 and the center point 1104 of the bounding box as a starting point and an end point. According to another embodiment, the electronic device 1000 may detect an arbitrary candidate contour 1114 using the center point 1106 and the center point 1108 of the bounding box as a start point and an end point.

For example, a method in which the electronic device 1000 detects a candidate contour using two adjacent center points of a bounding box will be described in detail. For example, it is assumed that the center point of the bounding box currently identified by the electronic device 1000 is point A (1122) and point B (1124). The electronic device 1000 may generate a straight line connecting the point A and the point B, and identify a center point C 1126 of the generated straight line. The electronic device 1000 may identify a point D 1128 spaced apart from the center point C 1125 by a preset distance so as to be perpendicular to a straight line connecting the points A and B. The electronic device 1000 may detect a primary candidate contour by connecting the point A 1122, the WJA B 1124 and the point D 1128.

The electronic device 1000 according to the present disclosure identifies the identified points A 1122 and D 1128, similar to the method of detecting the first candidate contour using the points A 1122 and B 1124. Can be used to detect a secondary candidate contour between A 1122 and point D 1128. Also, the electronic device 1000 may detect the second-order candidate contour using the point D 1128 and the point B 1124 in a similar manner. The electronic device 1000 may determine an optimal candidate contour for connecting two adjacent center points of the bounding box by repeating the above-described method.

Also, the electronic device 1000 may generate adjustment boxes representing different scores along the determined candidate contour. For example, the electronic device 1000 may generate a first adjustment box 1132 that is separated by a first distance from the candidate contour line, and may generate a second adjustment box 1134 that is separated by a second distance from the candidate contour line. The electronic device 1000 may allocate different adjustment points to the adjustment boxes. According to an embodiment, the electronic device 1000 may allocate a zero adjustment score or a negative adjustment score to areas 1136 other than the first and second adjustment boxes.

The electronic device 1000 identifies coordinates of pixels through which candidate contours pass, and identifies whether the coordinates of the identified pixels belong to the first adjustment box 1132 or the second adjustment box 1134, thereby It is possible to determine the contour score for the contours. When the contour scores determined for the candidate contours are smaller than a preset threshold value, the electronic device 1000 may detect the contour again from the guide region including the binarized image. However, when the contour scores determined for the candidate contours are greater than a preset threshold, the electronic device 1000 may detect the corresponding candidate contour as the contour of the object. That is, the electronic device 1000 may adjust the candidate contours by determining the contour score for the candidate contour using a plurality of adjustment boxes arranged along the candidate contours and comparing the contour score with a preset threshold. The electronic device 1000 may adjust the candidate contour using adjustment boxes representing different scores, and detect the adjusted candidate contour as a final contour of the object in the guide area.

12 is a diagram illustrating an identification condition used by an electronic device to determine a probability that an object detected in a guide area corresponds to an eye, according to an exemplary embodiment.

The electronic device 1000 does not extract feature vectors to be used for artificial intelligence calculations from all guide areas, but determines the probability that the detected contour in the guide area corresponds to the object, and uses only the guide area in which the determined probability is equal to or greater than a preset threshold. So you can identify the object. That is, the electronic device 1000 determines a contour component 1146 constituting an outline in the guide area including the binarized image, a predetermined identification condition 1152 based on the contour component, and determines the determined contour component and the identification condition. Based on, it may be determined whether to extract a feature vector from a specific guide area.

According to an embodiment, prior to a process for the electronic device 1000 to secondarily determine an object by using an artificial intelligence operation, a contour component 1146 necessary for determining a primary object for a plurality of guide areas in an image And identification conditions 1152 may be identified by identification number 1142 and identification number 1148, respectively. The electronic device 1000 matches the identification number 1142 and the contour component 1146, matches the identification number 1148 and the identification condition 1152, and matches the identification number 1142 and the contour component 1146. The matched identification number 1148 and identification condition 1152 may be stored in a memory of the electronic device.

According to an embodiment, the contour component 1146 used by the electronic device 1000 is the height of the contour extracted from the guide area, the width of the contour, the area of the contour, the pixels constituting the contour, and coordinates of the pixels constituting the contour. May include information about. In addition, according to an embodiment, the identification condition 1152 used by the electronic device 1000 is, based on the contour component, the intensity of pixels in the ellipse formed based on the contour, the area of the detected contour, and the coordinates of the detected contour. It may include information, an aspect ratio of the height and width of the detected contour, or an area of the outermost contour and a solidity of an area within the contour excluding an area of the outermost contour.

According to an embodiment, the electronic device 1000 may determine a probability that contours in at least one guide area correspond to an object by applying different weights to the determined identification condition. For example, the electronic device 1000 identifies the intensity values of pixels in the ellipse formed based on the outline detected in the guide area, and then assigns a first score to the ##0001 item according to the intensity of the identified intensity value. can do.

In addition, the electronic device 1000 may allocate a second score to the ##0100 item according to the aspect ratio of the height and width of the contour detected in the guide area. In addition, the electronic device 1000 may allocate a third score to item ##1001 according to a solidity of the area of the outermost contour detected in the guide area and the area within the contour excluding the area of the outermost contour. Likewise, the electronic device 1000 may allocate points according to the degree to which the contours detected from the guide area correspond to items of different identification conditions.

The electronic device 1000 may determine a probability that the contour detected from the guide area corresponds to the object based on a sum of points allocated for each item of each identification condition. In addition, according to an embodiment, the electronic device 1000 does not simply sum the scores corresponding to each identification condition, but determines different weights for each identification condition, and weights the scores for each identification condition according to different weights. By summing, it is also possible to determine the probability that the outline in the guide area corresponds to the object.

13 is a block diagram of an electronic device according to an exemplary embodiment.

14 is a block diagram of an electronic device according to another embodiment.

13, the electronic device 1000 may include a processor 1300, a memory 1700, a camera module 1610, and a display 1210. However, not all of the illustrated components are essential components. The electronic device 1000 may be implemented by more components than the illustrated components, or the electronic device 1000 may be implemented by fewer components.

For example, as illustrated in FIG. 14, the electronic device 1000 according to an exemplary embodiment includes a user input interface 1100 in addition to the processor 1300, the memory 1700, the camera module 1610, and the display 1210. ), an output unit 1200 further including a sound output unit 1220 and a vibration motor 1230, a sensing unit 1400, a network interface 1500, an A/V input unit 1600, a fastening unit (not shown) It may further include.

The user input interface 1100 refers to a means for a user to input a sequence for controlling the electronic device 1000. For example, the user input interface 1100 includes a key pad, a dome switch, and a touch pad (contact capacitive method, pressure resistive film method, infrared sensing method, surface ultrasonic conduction method, red). A method for measuring a powdered tension, a method for measuring a piezo effect, etc.), a jog wheel, a jog switch, etc. may be provided, but the present invention is not limited thereto. The user input interface 1100 may receive a user input sequence for a screen output on the display by the electronic device 1000. In addition, the user input interface 1100 may receive a touch input of a user who touches the display or a key input through a graphic user interface on the display.

The output unit 1200 may output an audio signal, a video signal, or a vibration signal, and the output unit 1200 may include a display unit 1210, an audio output unit 1220, and a vibration motor 1230. have.

The display 1210 includes a screen for displaying and outputting information processed by the electronic device 1000. In addition, the screen may display a guide message for allowing a user to photograph an animal image and at least one guide area. The sound output unit 1220 outputs audio data received from the network interface 1500 or stored in the memory 1700. Also, the sound output unit 1220 outputs sound signals related to functions performed by the electronic device 1000. The vibration motor 1230 may output a vibration signal. For example, the vibration motor 1230 may output a vibration signal corresponding to an output of functions performed by the electronic device 1000.

The processor 1300 generally controls the overall operation of the electronic device 1000. For example, the processor 1300, by executing programs stored in the memory 1700, the user input unit 1100, the output unit 1200, the sensing unit 1400, the communication unit 1500, the A/V input unit 1600 ) And so on. Also, the processor 1300 may perform a function of the electronic device 1000 illustrated in FIGS. 1 to 12 by executing programs stored in the memory 1700.

Specifically, the processor 1300 may obtain an input of a user who touches the screen of the electronic device by controlling the user input unit. According to an embodiment, the processor 1300 may control the microphone to acquire a user's voice. The processor 1300 may acquire images including the object based on a user input, and if the object in the image is not identified, the processor 1300 may obtain an image including the object again based on a user input re-received from the user. I can.

According to an embodiment, the processor 1300 acquires an image including the object by executing the one or more instructions stored in a memory, determines at least one guide area corresponding to the object in the acquired image, and , Extracting a feature vector for identifying the object from the at least one guide area, and inputting the extracted feature vector into an artificial intelligence artificial intelligence model to identify the type of object in the image, and the type of the identified object When is identified as an animal's eye or a partial object related to the animal's eye, at least one pixel value in an image including the object may be stored. According to an embodiment, the artificial intelligence includes an input layer, an output layer, and one or more hidden layers between the input layer and the output layer, and a weight related to the connection strength between the input layer, the output layer, and the hidden layers. It may be a feedforward network in which is updated.

In addition, according to an embodiment, the processor 1300 determines a first guide area corresponding to the object to include a preset reference pixel in the acquired image by executing one or more instructions, and the determined first guide By scaling an area according to a preset scale, at least one second guide area corresponding to the object may be determined, and feature vectors for each of the first guide area and the at least one second guide area may be extracted.

In addition, according to an embodiment, the processor 1300 may control the display so that the first guide area is displayed on the screen of the electronic device, and the at least one second guide area is not displayed on the screen of the electronic device. May be.

According to an embodiment, the processor 1300 determines a gradient of pixels in an image, generates a histogram for each partial region in the image using the determined gradient, and generates a blank of the histogram for each partial region. By linking bin) values, the feature vector can be extracted.

In addition, according to an embodiment, the processor 1300 identifies a type of an object for each of the first guide area and the at least one second guide area, and identifies the type of the identified object as an object of the same type. The frequency number of the guide area to be formed may be determined, and the type of the object in the image may be identified based on the determined frequency number.

In addition, according to an embodiment, the processor 1300 removes Gaussian noise in the acquired image by applying a Gaussian kernel indicating a Gaussian distribution to pixel values in the acquired image, and The histogram of pixel values may be equalized, and the at least one guide area in the image whose histogram is smoothed may be determined.

In addition, according to an embodiment, the processor 1300 detects a contour of the object for each of at least one guide area, determines a probability that the detected contour corresponds to the object, and the determined probability is a preset threshold. The feature vector may be extracted from the at least one guide area including a larger outline.

In addition, according to an embodiment, the processor 1300 determines an identification condition for identifying the object using the detected contour components in the contour, and applies different weights to the determined identification condition, so that the at least one It is possible to determine a probability that the outline in the guide area corresponds to the object.

In addition, according to an embodiment, the processor 1300 divides the histogram-smoothed image into at least one partial region of a preset size, and based on the pixel values of the divided partial regions, the processor 1300 Determine values, binarize pixel values in the partial regions based on the determined threshold values, and detect a contour of the object for each of the at least one guide region within an image including the binarized pixel values. have.

In addition, according to an embodiment, the processor 1300 resizes the sizes of the at least one guide area including an outline having a determined probability greater than a preset threshold, and the characteristic for each of the resized at least one guide area. You can also extract vectors.

In addition, according to an embodiment, the processor 1300 generates a rectangular bounding box including the first guide area in an image including binarized pixel values, and based on the center points of each side of the generated bounding box. A candidate contour including the object is determined, and the candidate contour is adjusted using adjustment boxes representing different scores based on a distance away from the candidate contour and arranged along the determined candidate contour, and the adjusted candidate The outline can be detected as the outline of the object.

In addition, according to an embodiment, the processor 1300 smoothes the adjusted candidate contour using a preset median filter, and calculates the flattened candidate contour with respect to the object for each of the at least one guide area. It can be detected by the outline.

The sensing unit 1400 may detect a state of the electronic device 1000 or a state around the electronic device 1000 and transmit the sensed information to the processor 1300. The sensing unit 1400 includes a magnetic sensor 1410, an acceleration sensor 1420, a temperature/humidity sensor 1430, an infrared sensor 1440, a gyroscope sensor 1450, and a position sensor. (For example, a GPS) 1460, an atmospheric pressure sensor 1470, a proximity sensor 1480, and an RGB sensor 1490 may be included, but are not limited thereto. Since the function of each sensor can be intuitively inferred by a person skilled in the art from its name, a detailed description will be omitted.

The network interface 1500 may include one or more components that allow the electronic device 1000 to communicate with other devices (not shown) and the server 2000. Another device (not shown) may be a computing device such as the electronic device 1000 or a sensing device, but is not limited thereto. For example, the network interface 1500 may include a wireless communication interface 1510, a wired communication interface 1520, and a mobile communication unit 530.

The wireless communication interface 1510 includes a short-range wireless communication unit, a Bluetooth communication unit, a Near Field Communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared data association (IrDA). ) May include a communication unit, a Wi-Fi Direct (WFD) communication unit, etc., but is not limited thereto. The wired communication interface 1520 may connect the server 2000 or the electronic device 1000 by wire.

The mobile communication unit 1530 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include a voice signal, a video call signal, or various types of data according to transmission and reception of text/multimedia messages.

According to an embodiment, the network interface 1500 may transmit information on an image of an object or information on images composed of a plurality of frames to a server under the control of a processor. In addition, the network interface 1500 may further receive information on a result of recognizing an object in the image from the server.

The A/V (Audio/Video) input unit 1600 is for inputting an audio signal or a video signal, and may include a camera 1610 and a microphone 1620. The camera 1610 may obtain an image frame such as a still image or a video through an image sensor in a video call mode or a photographing mode. The image captured through the image sensor may be processed by the processor 1300 or a separate image processing unit (not shown). For example, the camera module 1610 may acquire an animal image multiple times based on a user input.

The microphone 1620 receives an external sound signal and processes it as electrical voice data. For example, the microphone 1620 may receive an acoustic signal from an external device or a user. The microphone 1620 may receive a user's voice input. The microphone 1620 may use various noise removal algorithms for removing noise generated in a process of receiving an external sound signal.

The memory 1700 may store a program for processing and controlling the processor 1300, and may store data input to or output to the electronic device 1000. In addition, the memory 1700 is an image including an object, information on an image, information on at least one guide area displayed on an image or image, and detection from the guide area obtained by the electronic device 1000 based on a user input. Information on the outlined contour, information on the identification result of the object for each guide area, a contour component for determining a probability that the contour for each guide area corresponds to the object, or information on the identification condition may be further stored.

The memory 1700 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), and RAM. (RAM, Random Access Memory) SRAM (Static Random Access Memory), ROM (ROM, Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk And at least one type of storage medium among optical disks.

Programs stored in the memory 1700 may be classified into a plurality of modules according to their functions, for example, a UI module 1710, a touch screen module 1720, a notification module 1730, and the like. .

The UI module 1710 may provide a specialized UI, GUI, etc. that are linked with the electronic device 1000 for each application. The touch screen module 1720 may detect a user's touch gesture on a touch screen and transmit information about the touch gesture to the processor 1300. The touch screen module 1720 according to some embodiments may recognize and analyze a touch code. The touch screen module 1720 may be configured with separate hardware including a controller.

The notification module 1730 may generate a signal to notify the occurrence of an event of the electronic device 1000. Examples of events occurring in the electronic device 1000 include call signal reception, message reception, key signal input, and schedule notification. The notification module 1730 may output a notification signal in the form of a video signal through the display unit 1210, may output a notification signal in the form of an audio signal through the sound output unit 1220, and the vibration motor 1230 It is also possible to output a notification signal in the form of a vibration signal through.

15 is a block diagram of a server according to an embodiment.

The server 2000 may include a network interface 2100, a database 2200, and a processor 2300. The network interface 2100 may correspond to the network interface 1500 of the electronic device 1000 illustrated in 14. For example, the network interface 2100 may transmit information on a result of recognizing the received image or object in the image, whether it receives information about an image or an image photographing objects from the electronic device 1000. Can be transferred to.

The database 2200 may correspond to the memory 1700 of the electronic device 1000 illustrated in FIG. 14. For example, the database 2200 includes an image including an object, information on an image, information on at least one guide area displayed on an image or image, and a guide acquired by the electronic device 1000 based on a user input. Information on the contour detected from the region, information on the identification result of the object for each guide region, the contour component for determining a probability that the contour for each guide region corresponds to the object, or information on the identification condition may be further stored.

The processor 2300 typically controls the overall operation of the server 2000. For example, the processor 2300 may generally control the DB 2200 and the network interface 2100 by executing programs stored in the DB 2200 of the server 2000. Also, the processor 2300 may perform some operations of the electronic device 1000 in FIGS. 1 to 14 by executing programs stored in the DB 2100. For example, the processor 2300 receives information on an image including an object from the electronic device, determines at least one guide area corresponding to the object in the received image, and identifies the object from at least one guide area. It is possible to identify the type of object in the image by extracting the feature vector to be used and inputting the extracted feature vector to the artificial intelligence model stored in the server. Also, the processor 23000 may transmit an analysis result of the image to the electronic device 1000.

16 is a diagram for describing a process of identifying an object in an image by interworking with each other between an electronic device and a server, according to an exemplary embodiment.

In S1602, the electronic device 1000 may acquire an image including an object. According to an embodiment, the electronic device 1000 may acquire an image including an animal image by using at least one camera in the electronic device. In S1604, the electronic device 1000 may transmit an image including an object to the server 2000. According to an embodiment, the electronic device 1000 may transmit an image including an animal or information on an image to the server 2000.

In S1606, the server 2000 may determine at least one guide area corresponding to an object in the image. For example, the server 2000 may determine the second guide areas by determining a first guide area in the image and scaling the determined first guide area according to a preset scale.

In S1608, the server 2000 may extract a feature vector for identifying an object from the generated at least one guide area. According to an embodiment, the server 2000 determines a gradient of pixels in an image or image, generates a histogram for each partial region in the image using the gradient, and a bin value of the histogram generated for each partial region The feature vector can be extracted by connecting.

In S1610, the server 2000 may identify the type of object in the image by inputting the extracted feature vector into the pre-learned artificial intelligence model. In S1612, when the type of the identified object is identified as an animal's eye or a partial object related to the animal's eye, the server 2000 may store at least one pixel value in the image including the object in the database. . In S1614, the server 2000 may transmit information on the result of recognizing an object in the image to the electronic device 1000 by analyzing the image. In S1616, the electronic device 1000 may output information on a result of recognizing an object in an image received from the server.

The method according to an embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded in the medium may be specially designed and configured for the present disclosure, or may be known and usable to those skilled in computer software.

In addition, a computer program device including a recording medium storing a program for performing another method may be provided in the above embodiment. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

Although the embodiments of the present disclosure have been described in detail above, the scope of the present disclosure is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concept of the present disclosure defined in the following claims are also included in the present disclosure. It belongs to the scope of rights.

Claims (15)

In a method for an electronic device to identify an object in an image,
Obtaining an image including the object;
Determining a plurality of guide areas of different scales corresponding to the object in the acquired image;
Extracting a feature vector for identifying the object from the plurality of guide areas;
Identifying the type of object in the image by inputting the extracted feature vector into an artificial intelligence model; And
Storing at least one pixel value in an image including the object when the type of the identified object is identified as an animal eye or partial objects related to the animal eye; Including,
The step of identifying the type of the object is based on the frequency of the guide area identified as an object of the same type as a result of identifying the type of the object for each of the plurality of guide areas of different scales, Identifying a type; The method further comprising. The method of claim 1, wherein determining the plurality of guide areas
Determining a first guide area corresponding to the object to include a preset reference pixel in the obtained image; And
Determining at least one second guide area corresponding to the object by scaling the determined first guide area according to a preset scale; Including more,
The extracting of the feature vector comprises extracting a feature vector for each of the first guide region and the at least one second guide region. The method of claim 2,
Wherein the first guide area is displayed on the screen of the electronic device, and the at least one second guide area is not displayed on the screen of the electronic device. The method of claim 1, wherein extracting the feature vector comprises:
Determining a gradient of pixels in the image;
Extracting the feature vector using the determined gradient; Including, the method. The method of claim 2, wherein the step of identifying the type of object
Identifying a type of an object for each of the first guide area and the at least one second guide area;
Determining a frequency number of a guide area identified as an object of the same type among the types of the identified objects; And
Identifying the type of the object in the image based on the determined frequency; Including, the method. The method of claim 2, wherein the method
Removing noise in the acquired image by applying a kernel to pixel values in the acquired image;
Converting pixel values in the image from which the noise has been removed; And
Determining the plurality of guide regions in the image in which the pixel values are converted; The method further comprising. The method of claim 6, wherein the method
Detecting an outline of the object for each of the plurality of guide areas;
Determining a probability that the detected contour corresponds to the object; And
Extracting the feature vector from a plurality of guide regions including an outline having the determined probability greater than a preset threshold; The method further comprising. The method of claim 7, wherein determining a probability corresponding to the object comprises:
Determining an identification condition for identifying the object using contour components in the detected contour; And
Determining a probability that a contour within the plurality of guide regions corresponds to the object by applying different weights to the determined identification condition; Including, the method. The method of claim 7, wherein the method
Dividing the converted images into at least one partial region of a preset size;
Determining threshold values for each of the subregions to binarize pixel values included in the subregion, based on the pixel values of the divided subregions;
Binarizing pixel values in the partial regions based on the determined threshold values; And
Detecting an outline of the object for each of the plurality of guide areas in the image including the binarized pixel values; The method further comprising. The method of claim 7, wherein extracting the feature vector comprises:
Resizing the sizes of the plurality of guide regions including contours having the determined probability greater than a preset threshold; And
Extracting the feature vector for each of the plurality of resized guide regions; Including, the method. The method of claim 9, wherein detecting the contour
Generating a rectangular bounding box including the first guide area in the image including the binarized pixel values;
Determining a candidate contour including the object based on the center point of each side of the generated bounding box;
Adjusting the candidate contour using adjustment boxes arranged along the determined candidate contour and representing different scores based on a distance away from the candidate contour; And
Detecting the adjusted candidate contour as the contour of the object; Including, the method. The method of claim 11, wherein detecting the contour
Smoothing the adjusted candidate contour line; And
Detecting the flattened candidate contour as a contour of the object for each of the plurality of guide regions; The method further comprising. The method of claim 8,
The contour components include the height of the detected contour, the width of the contour, the area of the detected contour, the area of the outermost contour among the detected contours, and pixels constituting the contour, and the identification condition is the detection Intensity of pixels in the ellipse formed based on the outline, the area of the detected outline, the coordinates of the detected outline, the ratio of the height of the detected outline and the width of the outline, the area of the outermost outline and the outermost It characterized in that it comprises a ratio of the area within the contour excluding the area of the contour. The method of claim 1,
The artificial intelligence model includes an input layer, an output layer, and one or more hidden layers between the input layer and the output layer, and the weight of the connection strength between the input layer, the output layer, and the hidden layers is updated. A method comprising a neural network model or a machine learning model that is trained according to an artificial intelligence learning algorithm. In the electronic device for identifying an object in an image,
display;
At least one camera;
A memory for storing one or more instructions; And
At least one processor executing the one or more instructions; Including,
The processor executes the one or more instructions,
Obtaining an image including the object,
Determining a plurality of guide regions of different scales corresponding to the object in the acquired image,
Extracting a feature vector for identifying the object from the plurality of guide areas,
Identify the type of object in the image by inputting the extracted feature vector into an artificial intelligence model,
When the type of the identified object is identified as an animal eye or partial objects related to the animal eye, storing at least one pixel value in the image including the object,
The processor identifies the type of the object in the image based on the frequency of the guide area identified as an object of the same type as a result of identifying the type of the object for each of the plurality of guide areas of different scales. With electronic devices.

Images