KR102562740B1 - Method and apparatus for identifying eggs of parasites using image normalization - Google Patents

Abstract

An egg classification method is provided, in which an image containing a plurality of parasite eggs is captured through a camera of an egg sorting device, and a processor of the egg classification device determines a plurality of parasite eggs based on the boundary of each parasite egg image in the image. Cropping the individual image, generating a gradient vector for each pixel in a direction in which the brightness of the periphery of each pixel in the individual image of the cropped parasite eggs changes, and weighting the gradient vector for each pixel according to the brightness intensity of each pixel. , and by classifying the gradient vector for each pixel to determine the dominant direction of individual images of cropped parasite eggs, and correcting the direction of individual images of cropped parasite eggs based on the dominant direction, Finally identify which parasite an individual image corresponds to.

Description

Method and apparatus for identifying eggs of parasites using image normalization

The present disclosure relates to methods and devices for efficient microscopy of parasite eggs in feces.

Traditional parasite diagnosis is done by disposing of the feces and identifying the worms under a microscope. In this way, when a person directly inspects parasite eggs with the naked eye, the test result often differs depending on the skill of the speculum person. In particular, among the intestinal parasites, the infection rate for liver flukes and intestinal flukes remains high, but the accuracy of diagnosis is more important than anything else because they have different treatments.

Therefore, there is a need for a method and apparatus that can objectively and accurately perform a speculum examination of parasite eggs regardless of the skill level of the speculum operator.

Regardless of the proficiency of the speculum, it is necessary to have a method and device that can objectively and accurately classify the speculum of parasite eggs and which parasite the eggs correspond to.

An egg classification method according to an embodiment captures an image containing a plurality of parasite eggs through a camera of an egg classification device. The processor of the egg sorting device crops the individual images of the plurality of parasite eggs based on the boundary of each parasite egg image in the image. The processor of the egg sorting device generates a gradient vector for each pixel in a direction in which the brightness of the periphery of each pixel in each image of the cropped parasite egg is changed. The processor of the egg classifier assigns a weight to the gradient vector for each pixel according to the brightness intensity of each pixel. The processor of the egg sorting device classifies the gradient vector for each pixel to determine the dominant direction of individual images of the cropped parasite eggs. The processor of the egg sorter corrects the orientation of individual images of cropped parasite eggs based on the determined dominant orientation. The processor of the egg sorting device finally identifies which parasite the individual image of the calibrated parasite eggs is.

An egg sorting device according to an embodiment includes a camera for capturing an image containing a plurality of parasite eggs, cropping individual images of a plurality of parasite eggs based on the boundary of each parasite egg image in the captured image, A gradient vector for each pixel is generated in the direction in which the brightness of the periphery of each pixel in the individual image of the cropped parasite eggs becomes brighter, a weight is given to the gradient vector for each pixel according to the brightness intensity of each pixel, and the gradient for each pixel The vectors are classified to determine the dominant direction of individual images of cropped parasite eggs, correct the orientation of individual images of cropped parasite eggs based on the dominant direction, and determine which parasite the individual images of corrected parasite eggs are. A processor that performs image processing to identify, and a display to display which parasite the identified parasite is.

Through the method and apparatus according to the present disclosure, the speculum and classification of parasite eggs can be objectively and accurately performed regardless of the skill level of the speculum operator.

1 shows an example of image normalization for measuring an orientation of an image according to an embodiment of the present disclosure.
2 is a diagram illustrating an image inclination according to an embodiment of the present disclosure.
3 is a diagram showing determining a dominant direction within an image according to an embodiment of the present disclosure.
4 is a diagram showing image normalization of parasite eggs according to an embodiment of the present disclosure.
5 shows a normalized egg image according to an embodiment of the present disclosure.
6 is a flowchart for determining the infection type of a parasite by performing normalization according to an embodiment of the present disclosure.
7 is a diagram of performing sample preparation according to an embodiment of the present disclosure.
8 is a diagram showing an example of an abnormal egg image according to an embodiment of the present disclosure.
9 is a flowchart illustrating a method of classifying parasite eggs according to an embodiment of the present disclosure.
10 is a structural diagram of artificial intelligence for classifying parasitic eggs according to an embodiment of the present disclosure.
11 is a block diagram showing an egg sorting device according to an embodiment of the present disclosure.
12 is a flowchart of a method for identifying eggs according to an embodiment of the present disclosure.

Terms used in the present disclosure will be briefly described, and an embodiment of the present disclosure will be described in detail.

The terms used in the present disclosure have been selected from general terms that are currently widely used as much as possible while considering functions in an embodiment of the present disclosure, but they may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technologies, and the like. there is. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding embodiment of the present disclosure. Therefore, terms used in the present disclosure should be defined based on the meaning of the term and the general content of the present disclosure, not simply the name of the term.

When it is said that a certain part "includes" a certain element throughout the present disclosure, it means that it may further include other elements, not excluding other elements unless otherwise stated. In addition, terms such as "unit" and "module" described in this disclosure refer to a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. there is.

Hereinafter, with reference to the accompanying drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily carry out the present disclosure. However, an embodiment of the present disclosure may be implemented in many different forms and is not limited to the embodiment described herein. And in order to clearly describe an embodiment of the present disclosure in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the present disclosure.

The present disclosure relates to a diagnostic method and device for identifying liver flukes and intestinal flukes among intestinal parasites infected by parasitic larvae in feces, particularly humans due to the habit of reproducing freshwater fish. Until now, the identification of eggs of parasites and the diagnosis of parasitic infection are performed through a microscope, which causes a lot of difference in test results depending on the skill of the speculum. In addition, it takes a lot of time to diagnose from low-infected patient samples. In order to compensate for this, by taking images on a microscope and classifying parasite eggs through image processing, not only shortens the time required for diagnosis to determine which parasite is infected, but also supplements/assistes the skill of the diagnoser through precise determination of parasite eggs. can do. In addition, by taking an image of parasite eggs with a microscope and providing an egg-like image through artificial intelligence learning, it not only shortens the time required for diagnosis, but also supplements/assistes the skill of the diagnoser through precise determination of eggs.

1 shows an example of image normalization for measuring an orientation of an image according to an embodiment of the present disclosure.

A portion of the person's face in the original image 110 is slightly tilted from vertical. The egg sorting apparatus according to the present disclosure may generate the cropped image 120 by cropping only the face of a person in the original image 110 . In the cropping image 120, it can be seen that the vertical direction of the face does not coincide with the vertical direction of the coordinate system. At this time, if the tilt angle is θ, rotation coordinates to be rotated to straighten the face image vertically can be calculated as follows.

x _rotated = x * cosθ + y*sinθ

y _rotated = y*cosθ - x*sinθ ... Equation 1

Accordingly, the corrected cropped image 130 is obtained by rotationally correcting the cropped image 120 according to Equation 1 so that the vertical direction of the face matches the vertical direction of the coordinate system. The cropped image 130 corrected in this way may be referred to as a normalized image.

However, in order to generate the corrected cropped image 130, it is important to determine how many degrees the cropped image 120 should be rotated. Therefore, it is important to accurately obtain the image orientation (θ) angle in Equation 1 above. In order to accurately obtain the orientation (orientation, θ) angle, refer to FIG. 2 below.

2 is a diagram illustrating an image inclination according to an embodiment of the present disclosure.

In order to determine the orientation within the image, the image gradient must first be calculated. To calculate the image gradient, a vector field is created by calculating the brightening or darkening direction for each pixel by looking at the brightness of the periphery of each pixel in the image. A vector field can be represented by pixel magnitude and phase.

Referring to FIG. 2 , it can be seen that the gradient in the image is indicated by an arrow. In the case of image 1 (210), the brightness gradually becomes darker as it enters from the outside of the circle, so it has a radiated gradient. In addition, since the brightness change is made constant from the outside to the inside of the circle, it can be seen that the intensity of the pixel is changed in a uniform phase.

Image 2 (220) shows a gradual darkening from the right side of the image to the left side. Accordingly, in image 2 (220), it can be seen that the orientation is a straight line from right to left, and the intensity is uniformly changed from right to left. Of course, in Image 2 220, the image vector is directed from a bright pixel to a dark pixel, but the vector direction may be set from a dark pixel to a bright pixel. In this specification, an image vector may be referred to as a pixel vector, a pixel-by-pixel vector, an image direction vector, or simply a vector, and all terms will be used interchangeably.

According to an embodiment of the present disclosure, in order to accurately determine an orientation in an image, 36 bins of 10 degrees out of 360 degrees are set for an image, and a histogram of an image vector is calculated according to a gradient direction. do. In this case, the direction of the image vector may be determined according to a change in brightness between pixels. At this time, in one embodiment, different weights are assigned to pixels according to brightness intensity (magnitude) of each pixel. In other words, if the brightness intensity difference with neighboring pixels is larger, a larger weight should be given to the vector to determine the overall orientation in the cropped image. In one embodiment, pixels closer to the center of the cropped image may have a higher weight. This is because even within the cropped image, the center image can be considered as a part that has more important information.

By the above method, the angle with the highest value in the histogram is set as the dominant orientation. When the dominant direction is determined, a corrected cropped image may be obtained in a state in which the cropped region from the image is rotated by the dominant direction.

3 is a diagram showing determining a dominant direction within an image according to an embodiment of the present disclosure.

In the case of parasite eggs, unlike blood cells, they have a directional elliptical shape. Therefore, it is more suitable to use artificial intelligence for discrimination of eggs, and discrimination accuracy is higher to determine which parasite eggs are by first performing a normalization step to make all the directions of the ellipses constant.

Referring to FIG. 3 , first, a Gaussian blurred image 310 obtained by Gaussian blurring of an original image is shown. Of course, blurring other than Gaussian blurring - median blurring, bilateral blurring, box blurring, channel blurring - may be used.

Gaussian blurring applies a Gaussian distribution to image processing, and Gaussian smoothing filtering for Gaussian blurring is image filtering with a filter for removing noise generated by a normal distribution or probability distribution. That is, if you analyze the noise of a randomly distributed image, you can see that it has a Gaussian distribution. For Gaussian smoothing filtering, when a kernel following a Gaussian distribution is created and convolution is performed, the center pixel has the largest weight. With , the weights of adjacent pixels are reduced.

By such Gaussian smoothing filtering, the original image may be converted into a Gaussian blur image 310 as shown in FIG. 3 . The Gaussian blurring image 310 may be converted into a gradient intensity image 320 representing gradient strength by calculating an inter-pixel vector, and at this time, a gradient direction image 330 may be determined through the gradient strength image 320. . In order to transform the Gaussian blurring image 310 into the gradient intensity image 320, the gradient strength relative to the neighboring pixels is determined for each pixel using the relative brightness value of the surroundings for each pixel of the image, and the gradient direction is determined according to the determined gradient strength for each pixel. can determine At this time, the slope direction can be organized into 36 bin histograms in units of 10 degrees. Therefore, as described above, when a histogram graph is generated based on the direction vectors indicated in the gradient direction image 330 by dividing 360 degrees into 10-degree bins, the direction bin graph 350 is obtained as shown in FIG. is created That is, if the values of the direction vectors indicated in the gradient direction image 330 are derived and each of them is mapped to the corresponding direction, the vectors with the highest frequency in the direction bin graph 350 are vectors having a value of 3 degrees in the direction, and thus the direction vectors 3 degrees is determined as the dominant orientation. Since vectors have different values depending on the difference in brightness intensity and weight between pixels, even if the size of one vector is accumulated in one bin, it does not simply increase by '1', but different values are accumulated according to the size of the vector. It will be.

When images of parasite eggs are captured in the same way, the dominant direction is determined for each egg by the steps described in FIG. 3 . When the dominance direction of each egg is determined, each egg is cropped and the cropped egg image is rotated by the dominance direction to end the normalization step of the egg image.

4 is a diagram showing image normalization of parasite eggs according to an embodiment of the present disclosure.

Referring to FIG. 4, after smearing parasite eggs, an image examined by an image inspection unit such as a microscope is captured. The non-normalized egg image 410 shows a state that has not yet been normalized in the first original image, Image 420 shows the final normalized egg image by rotating the image in the dominant direction by normalization according to FIG. 3 .

5 shows a normalized egg image according to an embodiment of the present disclosure.

Most of the parasite egg images according to FIG. 5 are normalized with the vertical direction as the dominant direction. The images of the parasite eggs, which were originally arranged in various orientations, are normalized to the dominant orientation, and discrimination is now performed to determine which type of eggs (and which infection is associated with them).

6 is a flowchart for determining the infection type of a parasite by performing normalization according to an embodiment of the present disclosure.

In step 610, the specimen containing the parasite eggs is loaded into the cartridge for speculum examination. Usually, specimens are shipped after keeping human feces in a refrigerated state, pre-treated by water-ether precipitation, and after removing noxious odors, they are loaded into cartridges for speculum examination. For speculum examination, take a drop from the final separated sample solution with a spout and drop it on the slide glass. After covering the cover slide, sample preparation is performed, and the entire area is observed at 100x magnification. discrimination will be performed.

At step 620, sample prep is performed.

Sample preparation refers to a process of preparing a sample by placing a specimen sample on a cartridge, smearing it, and fixing it. The specimen is placed on the cartridge, smeared, and loaded onto the egg sorting device so that it can be examined by an imaging speculum such as a microscope. The egg sorting device is a diagnostic device capable of diagnosing infection through images of parasite eggs. A detailed block diagram of the egg sorting device will be described in detail with reference to FIG. 8 .

The sample preparation will be described in more detail with reference to FIG. 7 .

7 is a diagram of performing sample preparation according to an embodiment of the present disclosure.

According to 710, a sample is loaded onto the cartridge. At this time, the loading position is the lower (left side in the drawing) part of the cartridge. It is important to implement appropriate imaging conditions to accurately photograph parasite eggs. Therefore, according to FIG. 7, a large concave cover glass may be used as the cover glass so that the specimen can be sufficiently provided. In this way, when a large concave cover glass is used, a sample with a capacity of 70 μl, which could be checked by dividing into three slides, can be processed at once. In addition, by attaching a sealing material to the bottom of the cover glass to minimize evaporation of moisture, reproducibility of diagnosis can be greatly improved. 700 shows a cross section of a cartridge loaded with a specimen sample. It is designed in such a way that the cover glass is covered on the slide glass, but the cover glass has a concave shape that blocks the side surface, so that the cartridge can sufficiently accommodate the specimen sample even if the amount of specimen is large. After the specimen 701 is placed on the slide glass 705, a concave-shaped cover glass is covered to prepare a speculum for the specimen 701. In addition, the cover glass 703 and the slide glass 705 are sealed with a sealing material used for nail polish between the cover glass 703 and the slide glass 705 so that the sample does not spill out and does not dry out. Glass can be bonded.

Next, it is necessary to obtain high-resolution, clear images so that image processing and/or artificial intelligence can precisely distinguish between parasite types. In order to secure such a high-resolution and clear image, it is necessary to select an optimal lens to be used in an image inspection unit (not shown). Lenses for image spectroscopy optimize physical and digital image parameters so that optical magnification, including digital magnification, is possible over hundreds of times. According to one embodiment, the lens may be able to magnify 200 times, 400 times, and 500 times.

720 shows an intermediate process in which a sample is smeared in a slide glass after being loaded. And in 730, nail polishing - a kind of sealing - is shown in the state that the smearing is completed.

Returning to FIG. 6 again, the egg image acquisition process of step 630 will be described. The user examines the prepared specimen under a microscope at a magnification of about 100 times. If parasite eggs are found through a visual speculum, the microscope is re-examined at several hundred times magnification to check the parasite egg image and capture it through the camera of the egg identification device. In addition, diagnostic devices such as miLab® can capture images of parasite eggs at hundreds of times magnification without manual reexamination. Accordingly, an egg image may be directly captured at a magnification of several hundred times without a manual reexamination process. In addition, diagnostic devices such as miLab® are equipped with two microscopic speculums of 100x and 400x magnification, and perform rapid search with the 100x magnification in the first step, and 400x in the second step when parasite eggs are found. It is also possible to capture images of parasite eggs by reexamining at a certain magnification. By this capture method, an image containing parasite eggs as shown in FIG. 4 can be obtained. Since the boundary between the parasite eggs and the surrounding image is formed, individual images of the parasite eggs may be cropped based on the boundary of the egg image. Cropping means selecting an image to fit the round or elliptical shape of eggs.

In step 640, the egg sorting device pre-processes the sample image of the parasite egg image.

The sample image means a cropped image. A process of removing abnormal images that are not parasite eggs among cropped images can be referred to as a preprocessing process. The preprocessing process includes excluding abnormal egg images as shown in FIG. 8 from egg identification targets. Referring to FIG. 8 for a moment, an example of an abnormal egg image according to an embodiment of the present disclosure is shown. These abnormal egg images are images in which eggs cannot be identified because the shape of the egg is not intact, a part of the egg is cut off, or a foreign substance is mixed in the image. These abnormal egg images are excluded from egg identification.

Returning to FIG. 6 again, the process of removing the tilting element in step 650 will be described. In the process of removing the italic elements from the parasite egg image, a Scale Invariant Feature Transform (SIFT) technique may be used, but is not limited thereto.

Removal of the tilting element is as described above with reference to FIGS. 2 to 4 .

In one embodiment, the processor of the egg identification device generates a gradient vector for each pixel in a direction in which the peripheral brightness of each pixel in an individual image of the cropped parasite egg becomes brighter or darker, and each pixel according to the brightness intensity of each pixel. A weight may be assigned to the gradient vector for each pixel.

In an embodiment, classifying the gradient vector for each pixel includes accumulating and classifying vector values of vectors in the same direction among the gradient vectors for each pixel. In addition, accumulating the vector values of vectors in the same direction among the gradient vectors for each pixel classifies the gradient vectors according to the vector direction of the vectors in the same direction among the gradient vectors for each pixel, and accumulates the magnitudes of the vectors in the same direction. can be seen as Determining the dominant direction of the individual images of the cropped parasite eggs can be seen as determining the direction of the vector having the largest magnitude among the accumulated vectors in the same direction as the dominant direction.

In one embodiment, in the step of assigning a weight to the gradient of each pixel, a relatively greater weight is assigned when a pixel is located at the center of the cropped image. The reason why the central pixels in an image are given greater weight is because the more central the image, the more likely it is to be an important part of the image.

Through this process, individual parasite egg images according to the dominant direction can be converted into images whose direction is corrected in the dominant direction. An image corrected in the dominant direction in this way may be referred to as a normalized image.

Next, the egg classification process of step 660 will be reviewed.

Refer to FIG. 9 for a detailed look at the parasite egg classification process.

9 is a flowchart illustrating a method of classifying parasite eggs according to an embodiment of the present disclosure.

In one embodiment, the obtained normalized parasite egg image may identify which parasite egg the extracted image corresponds to by utilizing the ResNet-50 neural network technique. Of course, other neural networks other than the ResNet-50 neural network can be used for egg identification. For a moment we refer to Figure 10 to see the structure of the ResNet-50 neural network.

10 shows a ResNet-50 neural network structure according to an embodiment of the present disclosure.

ResNet-50 neural network, which can be called a deep learning algorithm to classify images of parasite eggs, image analysis is divided into 4 stages for proper operation, and ResNet-50 is used to perform a total of 50 convolution layers. can be designed The normalized parasite egg image obtained by the ResNet-50 neural network is classified and the parasite egg is identified.

Parasitic larvae such as liver flukes, intestinal flukes, and roundworms have certain characteristics, such as morphology, color, image texture, and size, depending on the type. Based on at least one of these morphology, color, image texture, and size characteristics, parasite eggs may be classified by the image processing method according to an embodiment of the present disclosure.

Alternatively, parasite eggs may be classified by a deep learning algorithm using a ResNet-50 neural network according to an embodiment of the present disclosure. A processor (not shown) of the egg sorting device compares and analyzes the normalized parasite egg image with previously known parasite eggs. The comparison determines which infection each parasite egg image belongs to. In addition, according to the ResNet-50 neural network, the processor (not shown) of the egg sorting device analyzes the normalized parasite egg image to determine which infection the analyzed and classified parasite egg image belongs to.

Referring back to FIG. 9 , when the normalized egg image is input in step 6601, the processor determines whether the corresponding egg image is left-right asymmetric (step 6603). If the egg image is asymmetric, it is determined whether the size and color are within the range that can be identified as lung flukes (step 6605), and if it is within the range, the parasite egg image is finally determined to be an egg image of lung flukes (step 6607). The characteristics of each parasite egg are shown in Table 1 below.

helminth
egg type Size (length x width) (μm) color shape etc
liver fluke (28~32) x (14~16) light brown Sesame shape With ovules and shoulder projections, protruding wrinkles on egg surface whipworm eggs (45-50) x (20-25) dark tan long eggshell
barrel type Both ends are blocked with mucous plugs lung fluke eggs (80-100) x (45-65) light brown left-right asymmetry Oval broad and flat intestinal fluke eggs (28~32) x (14~16) dark tan U-Oval The wings are not clear,
The shoulder protrusion and the outer surface are smooth without wrinkles. roundworm eggs (60~70) x (30~35) partridge oval The shell surface is thick and bumpy. The surface consists of 3 layers

If the shape of the egg image is not asymmetric in the previous step 6603, or if the size and color are not within the range that can be determined as lung flukes in step 6605, if there is a mucus plug at both ends of the egg image in step 6611, if there is a mucus plug In step 6613, if the size and color are within the range of whipworm eggs, it is determined in step 6615 that the corresponding egg image is an image of whipworm eggs.

In the preceding steps 6611 and 6613, if there is no mucus plug at both ends in the egg image or if it is determined that the size and color are not within the range of whipworm eggs, it is determined in step 6621 whether the surface is bumpy within the egg image, and if the surface is bumpy, step 6623 In step 6627, the size and color are determined, and finally, in step 6627, it is determined that the egg image is an image of a roundworm egg.

In order to identify roundworm eggs, in step 6621, whether or not the surface is uneven and whether the surface is composed of three layers may also be included in the subject of determination (step 6625).

If it is determined as 'No' in steps 6621, 6625, and 6623, it is determined in step 6631 whether the corresponding egg image has an egg operculum and a shoulder protrusion.

If the determination result of step 6631 is 'Yes', it is determined whether there are surface wrinkles in the egg image (step 6633), and if the determination result of step 6633 is 'Yes', the size and size in step 6635 It is determined whether the color is within the range of liver fluke eggs, and if the result of the determination is 'Yes', the corresponding egg image is determined to be an image of liver fluke eggs in step 6637.

If the judgment results of the preceding steps 6631, 6633, and 6635 are all 'No', it is judged whether the size and color of the egg image are within the range of intestinal flukes, and if the judgment result is 'Yes', step 6643 In , the egg image is determined to be an image of an intestinal fluke egg.

Parasite eggs are classified according to the above judgment method.

Returning to FIG. 6 again, in step 670, if parasitic eggs are classified as a result of the previous step 660, the egg sorting device may display the result of diagnosis of whether or not the classified eggs are infected on the display. In other words, on the display of the egg sorting device, it is possible to display what kind of parasitic eggs detected, such as 'liver fluke infection' and 'intestinal fluke detection'.

11 is a block diagram showing an egg sorting device according to an embodiment of the present disclosure.

As shown in FIG. 11, the egg sorting device 2000 according to an embodiment of the present disclosure includes a cartridge loader unit 2100, a processor 2200, an image inspection unit 2300, an image capture unit 2400, It may include an output interface 2500, a user input interface 2600, and a memory 2700. All of the components of the egg sorting device 2000 are not essential, and each component may be added or subtracted according to the design idea of the manufacturer.

Hereinafter, the above components are examined in turn.

The cartridge loader unit 2100 is a unit for loading specimens containing eggs, and can be mounted so that specimens containing parasite eggs are loaded into the cartridge, smeared, and examined by the image speculum unit 2300.

The processor 2200 controls the overall operation of the egg sorting device 2000. The processor 2200 executes the programs stored in the memory 2700, such that the cartridge loader unit 2100, the image inspection unit 2300, the image capture unit 2400, the output interface 2500, the user input interface 2600, and the memory 2700 may be controlled.

According to an embodiment of the present disclosure, the processor 2200 may include an artificial intelligence (AI) processor. The artificial intelligence (AI) processor may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), an existing general-purpose processor (eg CPU or application processor) or a graphics-only processor (eg GPU - Graphic Processing Unit) It may be manufactured as a part of and mounted on the egg sorting device 2000. The processor 2200 may be an artificial intelligence (AI) processor capable of performing ResNet-50 according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, the processor 2200 may include a plurality of processors. Accordingly, the processor 2200 according to an embodiment of the present disclosure includes at least one processor. The processor 2200 may classify eggs according to the image of parasite eggs in the egg sorting device 2000, perform diagnosis according to the classification, and perform control operations according to the program. Result values and result information according to parasite egg classification may be stored in the memory 2700 .

The image speculum unit 2300 is an optical device such as a microscope, and refers to a device capable of enlarging and viewing parasite eggs. The image inspection unit 2300 may be provided together with the egg sorting device 2000 or may be a separately provided optional device. Through the image speculum unit 2300, the smeared specimen placed on the cartridge loader unit 2100 may be magnified and confirmed.

The image capture unit 2400 captures images of parasite eggs identified through the image speculum unit 2300 . The image capture unit 2400 usually consists of a precision camera. The camera of the image capture unit 2400 may be formed of a CMOS sensor, but is not limited thereto. The image containing the parasite eggs captured by the image capture unit 2400 is subjected to image processing-graphic processing by the processor 2200 for egg image classification.

The output interface 2500 is for outputting an audio signal or a video signal, and may include a display unit 2510. Although not shown, the output interface 2500 may optionally include a sound output unit according to a manufacturer's choice.

According to one embodiment of the present disclosure, the egg sorting apparatus 2000 may display information related to the egg sorting apparatus 2000 through the display unit 2510. For example, it is possible to display on the display unit 2510 whether the eggs are lung fluke eggs, whipworm eggs, roundworm eggs, liver fluke eggs, and intestinal fluke eggs along with the image of parasites performed by the egg sorting device 2000.

The display unit 2510 and the touch pad may form a layer structure to form a touch screen. When the display unit 2510 is configured as a touch screen by forming a layer structure of a touch pad, the display unit 2510 may be used as an input device as well as an output device. The display unit 2510 includes a liquid crystal display, a thin film transistor-liquid crystal display, a light-emitting diode (LED), an organic light-emitting diode, At least one of a flexible display, a 3D display, and an electrophoretic display may be included. In addition, two or more display units 2510 may be included depending on the implementation of the egg sorting device 2000.

According to an embodiment of the present disclosure, the output interface 2500 may output, through the display unit 2510, various image speculum-related information as well as a classification result of parasite eggs. According to an embodiment of the present disclosure, the output interface 2500 may display a current power level, an operating mode (eg, image inspection mode, egg classification mode, sleep mode, etc.).

The user input interface 2600 is for receiving an input from a user. The user input interface 2600 includes a key pad, a dome switch, a touch pad (contact capacitance method, pressure resistive film method, infrared sensing method, surface ultrasonic conduction method, integral tension measurement method) , piezo effect method, etc.), a jog wheel, and a jog switch, but may be at least one, but is not limited thereto.

The user input interface 2600 may include a voice recognition module. For example, the egg classification apparatus 2000 may receive an analog signal, a voice signal, through a microphone, and convert the voice part into computer-readable text using an Automatic Speech Recognition (ASR) model. The egg classifier 2000 may obtain the user's utterance intention by interpreting the converted text using a natural language understanding (NLU) model. Here, the ASR model or NLU model may be an artificial intelligence model. The artificial intelligence model can be processed by an artificial intelligence processor designed with a hardware structure specialized for the processing of artificial intelligence models. At this time, the processor 2200 may be a dedicated artificial intelligence processor. AI models can be created through learning. Here, being made through learning means that a basic artificial intelligence model is learned using a plurality of learning data by a learning algorithm, so that a predefined action rule or artificial intelligence model set to perform a desired characteristic (or purpose) is created. means burden. An artificial intelligence model may be composed of a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values, and a neural network operation is performed through an operation between an operation result of a previous layer and a plurality of weight values.

Linguistic understanding is a technology that recognizes and applies/processes human language/text, and includes natural language processing, machine translation, dialog system, question answering, and voice recognition. /Includes Speech Recognition/Synthesis, etc.

The memory 2700 may store programs for processing and control of the processor 2200, and input/output data (e.g., diagnosis information of the egg sorting device 2000, classification information according to images of parasites - lung fluke eggs) An egg image corresponding to a whipworm egg, an egg image corresponding to a whipworm egg, an egg image corresponding to a roundworm egg, an egg image corresponding to a liver fluke egg, and an egg image corresponding to an intestinal fluke egg, etc.) may be stored. The memory 2700 may store an artificial intelligence model.

The memory 2700 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory, etc.), and RAM. (RAM, Random Access Memory) SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk , an optical disk, and at least one type of storage medium. In addition, the control device 2000 may operate a web storage or cloud server that performs a storage function on the Internet.

The communication unit 2800 may include a short-distance communication unit 2810 and a long-distance communication unit 2820. The short-range wireless communication interface (2810) includes a Bluetooth communication unit, a Bluetooth Low Energy (BLE) communication unit, a Near Field Communication interface (WLAN) communication unit, a Zigbee communication unit, and an infrared (IrDA) communication unit. , Infrared Data Association (WFD) communication unit, WFD (Wi-Fi Direct) communication unit, UWB (Ultra Wideband) communication unit, Ant+ communication unit, etc. may be included, but is not limited thereto. The remote communication unit 2820 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the radio signal may include a voice call signal, a video call signal, or various types of data according to text/multimedia message transmission/reception. The remote communication unit 2820 may include a 3G module, 4G module, 5G module, LTE module, NB-IoT module, LTE-M module, etc., but is not limited thereto.

According to an embodiment of the present disclosure, it is possible to communicate with a server or other electric device outside the egg sorting device 2000 and transmit/receive data through the remote communication unit 2820. The communication unit 2800 may be included as an option for the purpose of selling the egg sorting device 2000 or for price competitiveness, or may not be included when communication is not required.

12 is a flowchart of a method for identifying eggs according to an embodiment of the present disclosure.

In step 1210, an image containing a plurality of parasite eggs is captured through the camera of the egg identification device.

In step 1220, the processor of the egg identification device crops individual images of a plurality of parasite eggs based on the boundary of each parasite egg image in the captured image.

In step 1230, the processor of the egg identification device generates a gradient vector for each pixel in a direction in which the brightness of the periphery of each pixel in the individual images of the cropped parasite eggs changes. To generate the gradient vector for each pixel in the captured image, the processor may blur the captured image.

In step 1240, the processor of the egg identification device assigns a weight to the gradient vector for each pixel according to the brightness intensity of each pixel.

In step 1250, the processor of the egg identification device classifies the gradient vector for each pixel to determine the dominant direction of the individual image of the cropped parasite eggs. Classifying the gradient vector for each pixel includes accumulating and classifying vector values of vectors in the same direction among the gradient vectors for each pixel. Accumulating the vector values of the vectors in the same direction among the gradient vectors for each pixel includes classifying the vectors in the same direction among the gradient vectors for each pixel according to the vector directions and accumulating the magnitudes of the vectors in the same direction.

Determining the dominant direction of individual images of cropped parasite eggs means determining the direction of a vector having the largest magnitude among accumulated vectors in the same direction as the dominant direction. Also, when assigning a weight to the gradient of each pixel, the processor may assign a relatively larger weight if the pixel is located at the center of the cropped image.

In step 1260, the processor of the egg identification device corrects the orientation of individual images of cropped parasite eggs based on the determined dominant orientation.

In step 1270, the processor of the egg identification device identifies which parasite the individual image of the calibrated parasite eggs is. When the processor identifies which parasite the individual image of calibrated parasite eggs is, based on at least one of the morphology, color, texture, and size of the individual image of calibrated parasite eggs, identifies which parasite the individual image represents. . In addition, identifying which parasite the individual image represents on the basis of at least one of the morphology, color, texture, and size of the individual image of the corrected parasite egg depends on the egg within the individual image of the corrected parasite egg. Determining whether the parasite egg corresponds to which parasite among lung fluke eggs, whipworm eggs, roundworm eggs, liver fluke eggs, and intestinal fluke eggs by determining at least one of whether the surface is asymmetrical, whether the surface is uneven, whether the surface is wrinkled, the shape of the shoulder protrusion, and whether the egg is open or not. includes doing

Alternatively, in one embodiment of the present disclosure, the processor of the egg identification device passes individual images of parasite eggs through 50 convolution layers divided into 4 stages by the ResNet-50 artificial intelligence model, and the corrected parasites. Individual images of eggs can be analyzed and classified.

The method according to an embodiment of the present disclosure may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer readable medium. Computer readable media may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the present disclosure, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

Some embodiments of the present disclosure may be implemented in the form of a recording medium including instructions executable by a computer, such as program modules executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and includes any information delivery media. In addition, some embodiments of the present disclosure may be implemented as a computer program or computer program product including instructions executable by a computer, such as a computer program executed by a computer.

The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary storage medium' only means that it is a tangible device and does not contain signals (e.g., electromagnetic waves), and this term refers to the case where data is stored semi-permanently in the storage medium and temporary It does not discriminate if it is saved as . For example, a 'non-temporary storage medium' may include a buffer in which data is temporarily stored.

The method according to an embodiment of the present disclosure may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. A computer program product is distributed in the form of a device-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (eg downloaded or uploaded) online, directly between smartphones. In the case of online distribution, at least a part of a computer program product (eg, a downloadable app) is stored on a device-readable storage medium such as a memory of a manufacturer's server, an application store's server, or a relay server. It can be temporarily stored or created temporarily.

Although the embodiments of the present disclosure have been described in detail above, the scope of the present disclosure is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concepts of the present disclosure defined in the following claims are also disclosed. falls within the scope of the

703; cover glass
705; slide glass
2000; Egg Sorting Device
2100; cartridge loader
2200; processor
2300; image speculum
2400; image capture unit
2500; output interface
2510; display
2600; user input interface
2700; Memory
2800; Ministry of Communications
2810; Near Field Communications Department
2820; telecommunications department

Claims (17)

Capturing an image containing a plurality of parasite eggs;
Cropping individual images of the plurality of parasite eggs based on the boundary of each parasite egg image in the image;
Generating a gradient vector for each pixel in a direction in which the brightness of the periphery of each pixel in the individual image of the cropped parasite eggs changes;
assigning a weight to the gradient vector for each pixel according to the brightness intensity of each pixel;
Classifying the gradient vector for each pixel to determine the dominant direction of the individual images of the cropped parasite eggs;
correcting the orientation of individual images of the cropped parasite eggs based on the dominant orientation; and
Parasite egg identification method comprising the step of identifying which parasite the individual image of the corrected parasite eggs corresponds to. According to claim 1,
Classifying the gradient vector for each pixel,
Parasite egg identification method comprising accumulating and classifying vector values of vectors in the same direction among the gradient vectors for each pixel. According to claim 2,
Accumulating the vector values of vectors in the same direction among the gradient vectors for each pixel,
Parasite egg identification method, characterized in that, among the gradient vectors for each pixel, the vectors in the same direction are classified according to the vector direction, and the magnitudes of the vectors in the same direction are accumulated. The method of claim 3, wherein determining the dominant direction of the individual images of the cropped parasite eggs comprises:
And determining the direction of the vector having the largest magnitude among the accumulated vectors in the same direction as the dominant direction. According to claim 2,
The method of identifying parasite eggs, characterized in that the step of assigning a weight to the gradient for each pixel comprises the step of assigning a relatively larger weight when a pixel is located at the center in the cropped image. According to claim 1,
Parasite egg identification method further comprising the step of blurring the captured image to generate a gradient vector for each pixel in the captured image. According to claim 1,
Further comprising the step of excluding the cropped image from the image to be corrected when the cropped individual images of the plurality of parasite eggs deviate from a predetermined elliptical shape, wherein the predetermined elliptical shape has a vertical length of 28 μm (micrometers) to Parasite egg identification method characterized in that the 32um width is 14um to 16um, or the length is 60um to 70um and the width is 30um to 35um in the form of an ellipse. According to claim 1,
The step of identifying which parasite the individual image of the corrected parasite eggs corresponds to,
and identifying which parasite the individual image corresponds to based on at least one of morphology, color, texture, and size of the individual image of the corrected parasite egg identification method. According to claim 8,
Identifying which parasite the individual image corresponds to based on at least one of the morphology, color, texture, and size of the individual image of the corrected parasite larvae,
In the individual image of the corrected parasite eggs, at least one of whether the eggs are asymmetrical, whether the surface is uneven, whether the surface is wrinkled, the shape of the shoulder protrusion, and whether or not the eggs are open is judged to determine whether the parasite eggs are lung fluke eggs, whipworm eggs, or roundworm eggs. Parasitic egg identification method comprising the step of determining whether it corresponds to any one of liver fluke eggs and intestinal fluke eggs. According to claim 1,
The step of identifying which parasite the individual image of the corrected parasite eggs corresponds to,
The corrected individual images of the parasite eggs are analyzed through 50 convolution layers divided into 4 stages by the ResNet-50 artificial intelligence model to analyze the individual images of the parasite eggs. A parasite egg identification method comprising the step of identifying which parasite the image corresponds to. According to claim 1,
Based on the identification of which parasite the corrected individual image of the parasite eggs corresponds to, displaying and displaying which parasite the individual images of the parasite eggs correspond to. method. According to claim 1,
Generating the gradient vector for each pixel in the direction in which the brightness of the periphery of each pixel in the individual image of the cropped parasite eggs changes,
Parasite egg identification method comprising the step of generating a gradient vector for each pixel in a direction in which the brightness of the peripheral portion of each pixel becomes dark. a camera for capturing images containing a plurality of parasite eggs;
Cropping individual images of the plurality of parasite eggs based on the boundary of each parasite egg image in the captured image;
Generating a gradient vector for each pixel in a direction in which the brightness of the periphery of each pixel in the individual image of the cropped parasite eggs becomes brighter,
assigning a weight to the gradient vector for each pixel according to the brightness intensity of each pixel;
Classifying the gradient vector for each pixel to determine the dominant direction of the individual images of the cropped parasite eggs;
Correcting the orientation of individual images of the cropped parasite eggs based on the dominant orientation;
a processor performing image processing to identify which parasite the corrected individual image of parasite eggs corresponds to; and
Parasitic egg identification device comprising a display for displaying which parasite the identified parasite is. According to claim 13,
Parasitic egg identification device further comprising a speculum for examining the plurality of parasite eggs. According to claim 13,
Parasitic egg identification device further comprising a cartridge for smearing and fixing the sample containing the plurality of parasite eggs for image capture. According to claim 13,
Parasitic egg identification device further comprising a slide glass for loading the sample containing the plurality of parasite eggs and a cover glass covering the slide glass. 17. The method of claim 16,
The parasite egg identification device, characterized in that the cover glass is sealed with the slide glass in upper and lower portions so that the sample does not leak.

Images