JP4945045B2 - Multi-neural network image apparatus and method - Google Patents

Abstract

A multi-neural net imaging apparatus (2) and method for classification of image elements, such as biological particles. The multi-net structure utilizes subgroups of particle features to partition the decision space by an attribute or physical characteristic of the particle and/or by individual and group particle classification that includes an unknown category. Preprocessing (6) classifies particles as artifacts based on certain physical characteristics. Post processing (6) enables the use of contextual information either available from other sources or gleaned from the actual decision making process to further process the probability factors and enhance the decisions.

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image device having a plurality of neural nets, a method for adjusting the neural network, and a method for operating such an image device. The device can be used to detect and classify biological particles, and more particularly to detect and classify biological particles from human urine.
[0002]
BACKGROUND OF THE INVENTION
Biological particle analyzers are well known in the art. See, for example, US Pat. No. 4,338,024, assigned to the assignee of the present application. This patent describes a prior art device that uses a computer with a fixed program stored to detect biological particles and classify the detected biological particles.
[0003]
The criteria decision theory used to classify biological particle images is well known and it tends to classify particles by a continuous method of classification. More specifically, for a urine sample containing multiple particle types, the particle image is searched for one or more particle features specific to a particle type and the images are extracted. This process is repeated one particle type at a time for other particles. The problem with this method is that each particle type can exhibit a wide range of values for the retrieved particle feature, and the wide range of values can partially match those of other particle types. There is also a problem of artificial structure (artifacts, artifacts). It does not have clinical significance, eg talc or hair, or is classified for imaging device sensitivity or other problems with the image (eg vague particle boundaries due to incomplete uptake) It is a particle image that cannot be done. Artificial particle images need to be ignored from the analysis in a way that does not adversely affect the overall accuracy of the particle analysis. Thus, it can be difficult to accurately and reliably classify particles in a sample that contains artifacts.
[0004]
In addition, most biological particle classification devices require manual manipulation to accurately classify particles within a sample. Although particle features can be used to separate particle images by particle type, an experienced user is required to verify the results.
[0005]
Neural network computers are also well known. The advantage of a neural network computer is the ability to “learn” from its experience, so theoretically, a neural network computer can have more discriminatory power when trained.
[0006]
There is a need for a biological particle classification method and apparatus for accurate and automated classification of biological particles within a sample, such as a urine sample.
[0007]
Summary of the Invention
In the present invention, a multi-neural network image detection and classification device is disclosed. Multi-neural networks use available information more efficiently. Its need is limited, where it effectively divides the decision space. Thereby, the entire decision still covers all results, but this information allows to be used to make fewer decisions at each stage. In addition, neural networks measure certainty in multiple processing steps to extract images into avoidance classes, eg, artificial structures. In a sense, at each stage of the gauntlet, this multi-neural network is forced to image data to receive a gauntlet that is likely to be placed in the “I don't know” category. Can be considered. This is much more powerful than simply receiving it through a single net. Because, in essence, what is achieved is that multiple adaptations of data to a well-defined template can be made for a more efficient use of that information.
[0008]
The present invention also relates to a large set of particle features and a training method, which involves selecting from many nets and reducing the feature vector size, not just a single pass through the training set. Finally, the present invention provides pre-processing and post-processing where learning information can be included as part of the decision process. Post processing is available from other sources or contextual information collected from the actual decision process can be used to further improve the decision.
[0009]
The present invention is a method for classifying an element in an image that is in one of a plurality of classification classes, the element having a plurality of features. The method includes extracting the plurality of features from the image of the element, determining a classification class for the element, and determining the element based on a plurality of previously determined classification class determinations. Changing the classification class. Element classification class determination includes at least one of the following:
Select and process a first subgroup of extracted features to determine the physical characteristics of the element and extract according to the physical characteristics determined to determine the classification class of the element Selecting and processing a second subgroup of the rendered features;
Depending on the determined classification class group to select and process a third subgroup of the extracted features to determine a group of classification classes of the element and to determine the classification class of the element Sub-step of selecting and processing a fourth sub-group of extracted features.
[0010]
In another aspect of the invention, an imaging device for classifying an element in an image that is in one of a plurality of classification classes, the element having a plurality of features, the device from the image of the element. Means for extracting the plurality of features; means for determining a classification class of the element; and means for changing the determined classification class of the element based on a plurality of previously determined classification class determinations. . The determining means includes at least one of the following:
Means for selecting and processing a first subgroup of extracted features to determine a physical property of the element, and a determined physical property to determine a classification class of the element In response, means for selecting and processing a second subgroup of extracted features;
Means for selecting and processing a third subgroup of extracted features to determine a group of classification classes of the element; and a classification class group determined to determine a classification class of the element And determining means for selecting and processing a fourth subgroup of extracted features.
[0011]
In yet another aspect of the invention, a method for classifying an element in an image that is in one of a plurality of classifications, the element having a plurality of features, the method comprising: Extracting a feature; and determining a classification of the element based on the plurality of features extracted by a first determination criterion, wherein the first determination criterion is an unknown classification of the element A determination step including classification, and when the element is classified as an unknown classification by the first determination criterion, the classification of the element is determined by a second determination criterion different from the first determination criterion And when the element is classified as one of a plurality of classifications according to the first determination criterion, the element is classified according to a third determination criterion different from the first and second determination criterion. Decide That and a step.
[0012]
In yet another aspect of the invention, an imaging device for classifying an element in an image that is in one of a plurality of classification classes, the element having a plurality of features, the apparatus comprising: Extracting the plurality of features from an image; a first processor for determining a classification class for the element; and changing a determined classification class for the element based on a plurality of previously determined classification class determinations. A second processor. The first processor determines the classification class of the element by at least one of the following:
In order to determine the physical characteristics of the element, a first subgroup of extracted features is selected and processed, and according to the determined physical characteristics to determine the classification class of the element Select and process a second subgroup of extracted features;
Select and process a third subgroup of extracted features to determine a group of classification classes of the element, and depending on the determined classification class group to determine the classification class of the element A fourth subgroup of extracted features is selected and processed.
[0013]
Other objects and features of the present invention will become apparent upon consideration of the specification, claims and appended drawings.
[0014]
Detailed Description of the Preferred Embodiment
The present invention provides a method and apparatus for determining the classification of individual particle images in a set of biological particles for the purpose of identifying each individual image and determining the number of images in each given class of particles. Including.
[0015]
Basic method and apparatus
The method is generally shown schematically in FIG. 1 and includes five basic steps:
1) collecting individual images;
2) extracting particle features from each individual image;
3) applying certain pre-processing rules to determine how to classify individual images or how to perform the classification process;
4) Classifying individual images using multiple neural network decision structures;
5) Analyzing a set of decisions or a subset of a set of decisions to determine a change to the total taxonomy of a set or a sub-set or individual image taxonomy.
[0016]
The method of the present invention further includes the step of training the individual neural nets used to make the decision as well as the step of selecting the net used for the final decision making from among the multiple nets created by the training procedure. .
[0017]
There are three main hardware elements used to carry out the present invention: the imaging system 2, the first processor 4 and the second processor 6. These hardware elements are shown schematically in FIG.
[0018]
The imaging system 2 is used to create an image of the field of view of the sample containing the particles of interest. The imaging system 2 preferably has a known flow as described in U.S. Pat. Nos. 4,338,024, 4,393,466, 4,538,299, and 4,612,614. It is a microscope. All of these patents are incorporated herein by reference. A flow microscope produces a continuous range image containing particles as it flows through the flow cell.
[0019]
The first processor 4 analyzes the continuous range of images and separates the particles within individual patches (fragments). A patch extractor (as described in US Pat. Nos. 4,538,299 and 5,625,709, incorporated herein by reference) analyzes the image produced by the imaging system and is of interest Used to define local (patch) containing particles. Each particle boundary is identified and defined and used to extract picture data for each particle from a larger range. Thereby creating a digital pitch image, each containing an image of the individual particles of interest (following significant compression of the data required for processing). The imaging system 2 and the first processor 4 combine to perform the first step (collection of individual images) shown in FIG.
[0020]
The second processor 6 analyzes each particle image to determine a classification method for the particle images. The second processor 6 performs the subsequent four steps shown in FIG. 1 as described below.
[0021]
Boundary enhancement-mask image
To improve particle feature extraction, the particle boundaries are further refined to create a black and white mask image of the particles. This process effectively changes all digital image pixels (background pixels) outside the grain boundary to black pixels and pixels inside the grain boundary to white pixels. The resulting white image of the particles against a black background conveys the shape and size of the particles very clearly, and particle characteristics based only on the shape and size (if the pixel is either white or black) Makes it easy to act for.
[0022]
3A and 3B show the basic steps for converting a particle image into a mask image. First, the Shen-Castan edge detector (as described on pages 29-32 of "Image Processing and Computer Vision Algorithms" of ISBN 0-471-14056-2 by Parker and James R. 1997 John Wiley & Sons. , And incorporated herein by reference) is used to define the edge of the particle of interest, as shown in FIG. 3A. The particle image 10 typically includes images of particles 12 and other particles 14 of interest. The particle image 10 is smoothed, a band-limited Laplace operator image is created, followed by a tone image. A threshold routine is used to detect an edge, whereby a location where the intensity exceeds a predetermined threshold is defined as an edge. The detected edges are connected to each other to result in an edge image 16. It contains lines corresponding to the detected boundaries that outline the various particles.
[0023]
A mask image is created from the edge image 16 in the manner shown in FIG. 3B. The edge image 16 is inverted so that the area line is white and the background is black. The image is then cleaned of all small spots and particles that are too small than you are interested in. A small gap in the boundary line is filled to connect several boundary lines together. The boundary line is widened to increase its width. This extension is on the outer edge of the boundary line because the inner edge defines the actual size of the particle. Pixels that are not joined are cross-linked to create a complete line surrounding the particle. The interior of the boundary is filled with it to create a small sphere that represents the particle. The small sphere is eroded to remove the pixels that formed the boundary line so that it has the correct size. Eventually, the largest microsphere is detected and all remaining microspheres are discarded. The resulting image is a particle mask image, in which white spheres against a black background correspond exactly to the size and shape of the particles of interest.
[0024]
Particle feature extraction
Once the image of the particle of interest is placed in the patch image and the region is limited by creating a white mask image of the particle, the patch and mask image extract the particle features (feature data) from the particle image. Further processing for. In general, particle characteristics numerically describe particle size, shape, texture and color in many different ways that help to accurately classify particle types. Particle features can be grouped into families associated with one of these numerical descriptions and extracted from patch images, mask images, or both.
[0025]
The first family of particle features are all related to the shape of the particles. It helps to identify spherical red and white blood cells, square or rectangular crystals, and elongated feces. The first family of particle features is:
1. Particle region: The number of pixels contained within a particle boundary. Preferably, the particle feature is obtained from a mask image of the particle.
2. Perimeter length: The length of the particle boundary in pixels. Preferably, this is obtained from the particle mask image by making a four-neighbor full circumference image of the mask and counting the number of non-zero pixels.
3. Form factor: spherical representation of particles. This is calculated as the square of the total perimeter divided by the particle region.
4). Area to full circumference: another spherical representation of particles. This is calculated as the particle area divided by the total perimeter.
[0026]
The second family of particle features relates to particle symmetry, and more particularly to the determination of the number of symmetry lines for any given shaped particle. This family of particle features is very distinguishable between feces (typically symmetrical along the long axis) and squamous epithelial cells (SQEPs). Useful. This family of particle features utilizes information obtained from line segments applied at different angular directions to the particle. As shown in FIG. 4, the line segment 20 is drawn through the center of gravity 22 of the mask image 19. For each point along line 20, line segments 24a and 24b perpendicular to it are drawn to extend away from line 20 until they cross the particle boundary, and are of the length of opposing vertical lines 24a and 24b. Differences are calculated and accumulated. This calculation is repeated at each point along the line 20 and all differences are summed and stored as the Symmetry Value of the line 20. In a perfect circle, the symmetry value is zero on every line segment 20. The calculation of the symmetry value is repeated for each angular rotation of the line segment 20, resulting in a plurality of symmetry values, each corresponding to a particular angular direction of one line segment 20. Symmetry values are normalized by the particle region values and stored in a list of symmetry values ordered from low to high.
[0027]
The second family of particle features is:
5). Minimum object: The smallest symmetry value in the ordered list that represents the maximum symmetry exhibited by the particle at some value of rotation.
6. 20% symmetry: Symmetric values that make up 20% of the ordered list of symmetry values.
7.50% symmetry: Symmetric values that make up 50% of the ordered list of symmetry values.
8. 80% symmetry: Symmetric values that make up 80% of the ordered list of symmetry values.
9. Maximum symmetry: The highest symmetry value in the ordered list that represents the smallest object represented by the particle at some value of rotation.
10. Average symmetry: The average value of symmetry values.
11. Standard deviation symmetry: Standard deviation of the symmetric value.
[0028]
The third family of particle features relates to particle skeletalization. It creates one or more line segments that characterize the size and shape of the particles. These particle characteristics are ideal for identifying analytes that have multiple components within a cluster, such as budding yeast, mycelium yeast, and leukocyte clumps. These analytes have a skeleton with multiple branches, which makes it easy to distinguish from an analyte with a single branch skeleton. The creation of skeletal images is well known in the art of image processing and is described in Parker and James R., 1997 John Wiley & Sons, ISBN 0-471-14056-2 “Algorithms for Image Processing and Computer Versions”, 176- It is disclosed on page 210. Skeletalization involves crushing each part of the particle boundary towards the center in a direction perpendicular to it. For example, as shown in FIGS. 5A to 5D, a complete circle is collapsed to one point, a crescent is collapsed to a curve, a figure 8 is collapsed to two straight lines, and a cell with a depression is collapsed to a curve. The preferred embodiment utilizes two skeletal algorithms, ZSH and BZS. ZSH is a Zhang-Suen thinning algorithm that uses Holt transform with stair movement. BZS is a Zhang-Suen thinning algorithm using Holt transform. Parker's Figure 5.11 (page 182) shows the difference in results when these algorithms are applied, along with the C code for each algorithm.
[0029]
The third family of particle features is:
12 ZSH skeleton size: The size of the skeleton is preferably determined by counting the number of pixels forming the skeleton.
13. ZSH standardized skeletal size: The skeletal size is normalized by the size of the particle and is determined by dividing the skeletal size by the particle region.
14 BZS skeleton size: The size of the skeleton is preferably determined by counting the number of pixels forming the skeleton. The perfect circle skeleton size is 1, and that of the crescent would be the length of the curve.
15. BZS standardized skeleton size: The skeleton size is normalized by the size of the particle and is determined by dividing the skeleton size by the particle region.
[0030]
The fourth family of particle features relates to measuring the shape of the particle using a radius length that fits the particle, and the displacement value sequence of these values. In particular, the centroid is preferably defined inside the particle using a mask image, and multiple radii emanating from the centroid at different angles extend outward toward the particle boundary. The radius lengths are collected in a list of radius values, and the list is categorized from low to high values. A% displacement value in an ordered list of values represents a value having a list position corresponding to a% from the list base. For example, a list's 30% displacement value is a value placed 30% above the list's base, and the list has 70% of its value above it. Thus, in the 10-value ordered list, the 30% displacement value is the seventh value from the top of the list, and the 50% displacement value is the middle value of the list.
[0031]
The fourth family of particle features is:
16. 25% radius value: A value corresponding to a 25% displacement value in the list of radius values.
17. 50% radius value: A value corresponding to a 50% displacement value in the list of radius values.
18. 75% radius value: A value corresponding to a 75% displacement value in the list of radius values.
19. Minimum average ratio: The ratio between the minimum radius value and the average deformation value.
20. Maximum average ratio: The ratio of the maximum radius value to the average radius value.
21. Average radius value: Average of radius values.
22. Standard deviation radius value: The standard deviation of the radius value.
[0032]
The fifth family of particle features measures the intensity of the particle image. The light absorption properties of different analytes are significantly different. For example, a crystal generally has a refractive power and can actually collect light in one spot so that its interior is brighter than the background. However, colored white blood cells are typically substantially darker than the background. The average intensity is shown as the total light absorption of the particles, but the standard deviation of the intensity measures the uniformity of the absorption characteristics of the particles. In order to measure the intensity, the particles are preferably separated by using a mask image to mask the patch image of the particles. Thus, only the remaining pixels (within the mask) are pixels contained within the particle region. This family of particle features includes:
23. Average pixel value: The average pixel value of all pixels inside the particle boundary.
24. Pixel value standard deviation: The standard deviation of the pixel values of the pixels inside the particle boundary.
[0033]
The sixth family of particle features uses the particle's Fourier transform to measure the particle's radial distribution. The Fourier transform depends on the size, shape and texture (ie fine grain structure) of the particles. In addition to adding texture, the Fourier transform amplitude is independent of the position of the particle, and particle rotation is directly reflected as the rotation of the transform. Finding a cluster of energy in one revolution is an indication of the linear state of the particle (ie, the particle has a linear portion). This discovery helps distinguish between particles like crystals for red blood cells. The Fourier transform of the particle patch image is preferably calculated using the well known Fast Fourier Transform (FFT) algorithm with a 128 × 128 pixel window. The following particle features are then calculated:
25. FFT average intensity of rotated 128 pixel lines: a matrix list of average pixel values along the 128 pixel lines as a function of rotation angle. This is calculated by placing a radius line of length 128 pixels during the transformation and rotating the radius line through a 180 ° arc by increasing N °. For each increase of N °, the average of the pixel values along the radius line is calculated. The average pixel value of N ° increase is accumulated in the queue as the average intensity with the corresponding angle increase.
26. FFT max / min 128 pixel angle difference: The difference between the angle values corresponding to the highest and lowest average intensity values accumulated in the queue.
27. FFT 128 pixel average intensity standard deviation: the standard deviation of the average intensity value accumulated in the queue.
28. FFT average intensity of rotated 64 pixel lines of rotated 128 pixel lines: Same as FFT average intensity, but instead of using 64 pixel long radius lines.
29. FFT max / min 64 pixel angle difference: Same as FFT max / min 128 pixel angle difference, but instead of using 64 pixel long radius lines.
30. FFT 64 pixel average intensity standard deviation: Same as FFT 128 pixel average intensity standard deviation, but instead of using 64 pixel long radius lines.
31. FFT average intensity of rotated 32 pixel lines: Same as FFT average intensity of rotated 128 pixel lines, but instead of using 32 pixel long radius lines.
32. FFT max / min 32 pixel angular difference: Same as max / min 128 pixel angular difference, but instead of using 32 pixel long radius lines.
33. FFT 32 pixel average intensity standard deviation: Same as FFT 128 pixel average intensity standard deviation, but instead of using 32 pixel long radius lines.
[0034]
Additional FFT particle features that are all related to standard deviation values based on turning radius lines that change length are as follows:
34. FFT 128 pixel average intensity standard deviation type: An accumulated queue list of standard deviations of average pixel values along 128 pixel lines for different rotations. This is calculated by placing a radius line of length 128 pixels during the transformation and rotating the radius line through a 180 ° arc by increasing N °. For each increment of N °, the standard deviation value of the pixel values on the line is calculated. All N ° increase standard deviation values are classified from low to high and accumulated in a queue.
35. FFT 128 pixel minimum radius standard deviation: The minimum radius standard deviation value retrieved from the sorted queue list of standard deviation values.
36. FFT 128 pixel maximum radius standard deviation: the maximum radius standard deviation value retrieved from the sorted queue list of standard deviation values.
37. FFT128 pixel 25% displacement value radius standard deviation: Radius standard deviation value from the queue corresponding to the 25% displacement value of the value accumulated in the queue.
38. FFT 128 pixels 50% displacement value radius standard deviation: Radius standard deviation value from the queue corresponding to the 50% displacement value of the value accumulated in the queue.
39. FFT 128 pixels 75% displacement value radius standard deviation: Radius standard deviation value from the queue corresponding to the 75% displacement value of the value accumulated in the queue.
40. FFT 128 pixel minimum to average radius standard deviation ratio: Ratio of minimum and average radius standard deviation values accumulated in the queue.
41. FFT 128 pixel maximum to average radius standard deviation ratio: Ratio of the maximum and average radius standard deviation values accumulated in the queue.
42. FFT 128 pixel mean radius standard deviation: The mean radius standard deviation value of the values stored in the queue.
43. Radius standard deviation FFT 128 pixel standard deviation: All standard deviations of the radial standard deviation accumulated in the queue.
44. FFT 64 pixel average intensity standard deviation type: Same as FFT 128 pixel average intensity standard deviation type, but instead of using 64 pixel long radius lines.
45. FFT 64 pixel minimum radius standard deviation: Same as FFT 128 pixel minimum radius standard deviation, but instead of using 64 pixel long radius lines.
46. FFT 64 pixel maximum radius standard deviation: Same as FFT 128 pixel maximum radius standard deviation, but instead of using 64 pixel long radius line.
47. FFT pixel 25% displacement value radius standard deviation: Same as FFT 128 pixel 25% displacement value radius standard deviation, but instead of using 64 pixel long radius line.
48. FFT pixel 50% displacement value radius standard deviation: Same as FFT 128 pixel 50% displacement value radius standard deviation, but instead of using 64 pixel long radius line.
49. FFT pixel 75% displacement value radius standard deviation: Same as FFT 128 pixel 75% displacement value radius standard deviation, but instead of using a 64 pixel long radius line.
50. FFT 64 pixel minimum to average radius standard deviation ratio: Same as FFT 128 pixel minimum to average radius standard deviation ratio, but instead of using 64 pixel long radius lines.
51. FFT 64 pixel maximum to average radius standard deviation ratio: Same as FFT 128 pixel maximum to average radius standard deviation ratio, but instead of using 64 pixel long radius lines.
52. FFT 64 pixel average radius standard deviation: Same as FFT 128 pixel average radius standard deviation, but instead of using 64 pixel long radius lines.
53. Radius standard deviation FFT 64 pixel standard deviation: Same as the radial standard deviation FFT 128 pixel standard deviation, but instead of using a 64 pixel long radius line.
54. FFT 32 pixel average intensity standard deviation type: Same as FFT 128 pixel average intensity standard deviation type, but instead of using 32 pixel long radius lines.
55. FFT 32 pixel minimum radius standard deviation: Same as FFT 128 pixel minimum radius standard deviation, but instead of using 32 pixel long radius lines.
56. FFT 32 pixel maximum radius standard deviation: Same as FFT 128 pixel maximum radius standard deviation, but instead of using 32 pixel long radius line.
57. FFT 32 pixel 25% displacement value radius standard deviation: Same as FFT 128 pixel 25% displacement value radius standard deviation, but instead of using 32 pixel long radius line.
58. FFT 32 pixel 50% displacement value radius standard deviation: Same as FFT 128 pixel 50% displacement value radius standard deviation, but instead of using 32 pixel long radius line.
59. FFT 32 pixel 75% displacement value radius standard deviation: Same as FFT 128 pixel 75% displacement value radius standard deviation, but instead of using 32 pixel long radius line.
60. FFT 32 pixel minimum to average radius standard deviation ratio: Same as FFT 128 pixel minimum to average radius standard deviation ratio, but instead of using 32 pixel long radius lines.
61. FFT 32 pixel maximum to average radius standard deviation ratio: Same as FFT 128 pixel maximum to average radius standard deviation ratio, but instead of using a 32 pixel long radius line.
62. FFT 32 pixel average radius standard deviation: Same as FFT 128 pixel average radius standard deviation, but instead of using 32 pixel long radius lines.
63. Radius standard deviation FFT 32 pixel standard deviation: Same as the radial standard deviation FFT 128 pixel standard deviation, but instead of using a 32 pixel long radius line.
[0035]
More FFT particle features are used, all of which relate to the mean value based on the rotated radius line varying in length:
64. FFT 128 pixel average intensity type: A sorted queue list of average pixel values along 128 pixel lines for different rotations. This is calculated by placing a radius line of length 128 pixels during the transformation and rotating the radius line through a 180 ° arc by increasing N °. For each increase in N °, the average value of the pixels on that line is calculated. The average pixel values for all N ° increments are sorted from low to high and accumulated in the queue.
65. FFT 128 pixel minimum average: Minimum radius average retrieved from the sorted queue list of averages.
66. FFT 128 pixel maximum radius value: Maximum radius average value retrieved from the average categorized queue.
67. FFT 128 pixel 25% displacement radius average: the radius average from the queue corresponding to the 25% displacement of the average accumulated in the queue.
68. FFT 128 pixel 50% displacement value radius mean value: the radius mean value from the queue corresponding to the 50% displacement value of the mean value accumulated in the queue.
69. FFT 128 pixel 75% displacement radius average: the radius average from the queue corresponding to the 75% displacement of the average accumulated in the queue.
70. FFT 128 pixel minimum to average radius average ratio: the ratio of the minimum and average radius average accumulated in the queue.
71. Ratio of FFT128 pixel maximum to average radius average: Ratio of maximum to average radius average stored in the queue.
72. FFT 128 pixel mean radius standard deviation: The mean radius standard deviation value of the mean value accumulated in the queue.
73. Average FFT 128 pixel standard deviation: All standard deviations of the average radius value accumulated in the queue.
[0036]
The seventh family of particle features uses gray scales of image intensity and color histogram displacement values. They provide additional information about intensity changes within the particle boundary. In particular, gray scale, red, green and blue histogram displacement values provide intensity characteristics in different spectral bands. Furthermore, the color used in particle analysis results in some particles absorbing certain colors, such as green, while others exhibit refractive properties at certain wavelengths. Thus, using all particle features allows one to identify colored particles such as white blood cells that absorb green and crystals that refract yellow light.
[0037]
Histograms, cumulative histograms and displacement calculation are disclosed in US Pat. No. 5,343,538. It is incorporated herein by reference. The particle image is typically captured using a CCD camera that separates the image into three color components. The preferred embodiment uses an RGB camera that captures the red, green and blue components of the particle image separately. The following particle features are calculated based on the grayscale, red, green and blue components of the image:
74. Grayscale pixel intensity: A sorted queue list of grayscale pixel intensities inside the particle boundary. The gray scale value is the sum of the three color components. For each pixel inside the particle boundary (as masked by the mask image), a grayscale pixel value is added to the grayscale queue, which is classified (eg, from low to high).
75. Minimum grayscale image intensity: The minimum grayscale pixel value accumulated in the queue.
76.25% grayscale intensity: A value corresponding to a 25% displacement value of a grayscale pixel value accumulated in the queue.
77. 50% grayscale intensity: A value corresponding to a 50% displacement value of a grayscale pixel value accumulated in a queue.
78.75% grayscale intensity: A value corresponding to a 75% displacement value of a grayscale pixel value accumulated in a queue.
79. Maximum grayscale image intensity: The maximum grayscale pixel value accumulated in the queue.
80. Red Pixel Intensity: A sorted queue list of red pixel intensities inside the particle boundary. The particle image is transformed so that only the red component of each pixel value remains. For each pixel inside the particle boundary (as masked by the mask image), a red pixel value is added to the red matrix, which is then classified from low to high.
81. Minimum red image intensity: The minimum red pixel value accumulated in the queue.
82.25% red intensity: A value corresponding to the 25% displacement value of the red pixel value accumulated in the queue.
83.50% red intensity: A value corresponding to the 50% displacement value of the red pixel value accumulated in the queue.
84.75% red intensity: A value corresponding to the 75% displacement value of the red pixel value accumulated in the queue.
85. Maximum red image intensity: The maximum red pixel value accumulated in the queue.
86. Green Pixel Intensity: A sorted queue list of green pixel intensities inside the particle boundary. The particle image is transformed so that only the green component of the pixel value remains. For each pixel inside the particle boundary (as masked by the mask image), a green pixel value is added to the green queue, which is then classified from low to high.
87. Minimum green image intensity: The minimum green pixel value accumulated in the queue.
88.25% green intensity: A value corresponding to a 25% displacement value of the green pixel value accumulated in the queue.
89. 50% green intensity: A value corresponding to a 50% displacement value of the green pixel value accumulated in the queue.
90.75% green intensity: A value corresponding to a 75% displacement value of the green pixel value accumulated in the queue.
91. Maximum green image intensity: The maximum green pixel value accumulated in the queue.
92. Blue pixel intensity: A sorted queue list of blue pixel intensity inside the particle boundary. The particle image is transformed so that only the blue component of the pixel value remains. For each pixel inside the particle boundary (as masked by the mask image), a blue pixel value is added to the blue matrix, which is then classified from low to high.
93. Minimum blue image intensity: The minimum blue pixel value accumulated in the queue.
94.25% blue intensity: A value corresponding to a 25% displacement value of the blue pixel value accumulated in the queue.
95.50% blue intensity: A value corresponding to the 50% displacement value of the blue pixel value accumulated in the queue.
96.75% blue intensity: A value corresponding to the 75% displacement value of the blue pixel value accumulated in the queue.
97. Maximum blue image intensity: The maximum blue pixel value accumulated in the queue.
[0038]
The eighth family of particle features uses concentric circles and rings to further characterize changes in FFT grading. A central circle is defined on the center of the FFT, with seven rings (in the form of washers) that continuously increase in diameter coaxial with the central circle outside the central circle. The first ring has an inner diameter equal to the outer diameter of the central circle, an outer diameter equal to the inner diameter of the second ring, and so on. The following particle features are calculated from the center circle and 7 rings in the FFT:
98. Center circle average value: Average value of FFT size inside the center circle.
99. Center circle standard deviation: Standard deviation of the FFT size inside the center circle.
100. Ring average value for the center circle: the ratio of the average value of the FFT size inside the first ring to that of the center circle.
101. Ring standard deviation with respect to the center circle: Ratio of the standard deviation of the FFT size inside the first ring to that of the center circle.
102. Ring average value for a circle: the ratio of the average FFT magnitude inside the first ring to that of a circle defined by the outer diameter of that ring.
103. Ring standard deviation for a circle: The ratio of the standard deviation of the FFT magnitude inside the first ring to that of a circle defined by the outer diameter of that ring.
104. Average of rings relative to the ring: that of the ring or center circle having the next smallest diameter after the average FFT size inside the first ring (in the case of the first ring it would be the center circle) To ratio.
105. Standard deviation of the ring relative to the ring: that of the ring or center circle with the next smallest diameter of the FFT size standard deviation inside the first ring (in the case of the first ring it would be the center circle) To ratio.
106-111: Same as features 100-104 except that the second ring is used instead of the first ring.
112-117: Same as features 100-104 except that a third ring is used instead of the first ring.
118-123: Same as features 100-104 except that a fourth ring is used instead of the first ring.
123-129: Same as features 100-104 except that a fifth ring is used instead of the first ring.
130-135: Same as features 100-104 except that a sixth ring is used instead of the first ring.
136-141: Same as features 100-104, except that a seventh ring is used instead of the first ring.
154 to 197 are the same as 98 to 141 except that they are applied to the FFT of the particle image FFT.
[0039]
The last family of particle features has sides equal to 11%, 22%, 33%, 44%, 55%, and 66% of the FFT window size (eg, 128) to further characterize changes in the FFT grading. Use concentric squares. It is affected by the size, shape and texture of the original analyte image. There are two well known texture analysis algorithms that characterize the texture of an FFT. The first is referred to as Vector Dispersion, which is described in Parker pages 165-168, which includes fitting to a plane to teach the region using normals and is incorporated by reference. . The second, referred to as the Surface Curvature Metric, includes fitting a polynomial to that region and is described in Parker pages 168-171, incorporated by reference. The following particle features are calculated from windows of different sizes during the FFT:
142-147: Application of vector distribution algorithm to 11%, 22%, 33%, 44%, 55%, and 66% FFT windows, respectively.
148-153: Application of surface curvature distance function algorithm to 11%, 22%, 33%, 44%, 55%, and 66% FFT windows, respectively.
[0040]
Processing and judgment
Once the aforementioned particle characteristics are calculated, processing rules are applied to determine whether a particle classification or all of the particles in the entire sample are processed. Preferred embodiments capture a low power objective (eg, 10 ×) for performing a low power field (LPF) scan with a larger field of view to capture larger particles, and capture finer details of smaller particles In order to perform a high power field (HPF) scan with greater sensitivity, a particle image is acquired using a high power objective (eg, 40 ×).
[0041]
The system of the present invention utilizes separate multi-neural network decision structures to classify particles captured in LPF and HPF scans. Since most particles of interest are one of the LPF or HPF scans but appear in both, another decision structure minimizes the number of particles of interest that each structure must classify.
[0042]
Neural network structure
FIG. 8 shows the basic neural network structure used for all neural nets in LPF and HPF scans. Each net is an input X corresponding to one of the aforementioned particle features selected for use in the net.₁-X_dContains an input layer with Each input is a plurality of neurons Z in a hidden layer₁-Z_jAre connected to each one. These hidden layer neurons Z₁-Z_jEach sums all values received from the input layer in a weighting manner, so that the actual weight of each neuron can be assigned individually. Each hidden layer neuron Z₁-Z_jAlso applies a nonlinear function to the weighted sum. Each hidden layer neuron Z₁-Z_jOutput from the second (output) layer Y of the neuron₁-Y_kAre supplied to each one. Output layer neuron Y₁-Y_kEach also sums the inputs received from the hidden layers in a weighting method and applies a non-linear function to the weighted sum. The output layer neurons provide the output of the net, so the number of these output neurons corresponds to the number of decision classes that the net makes. The number of inputs is equal to the number of particle features selected for input to the net.
[0043]
As will be described later, each net is “trained” to produce accurate results. For each decision made, only the particle features that are appropriate for the determination of the net are selected for input to the net. The training procedure involves changing various weights for the neurons until a satisfactory result is achieved from the net as a whole. In the preferred embodiment, the various nets were trained using Product Version 5.30 NeuralWorks. It is manufactured by Carnegie, Pa's Neural Ware, in particular the extended delta bar delta propagation algorithm. The non-linear function used for all neurons of all nets in the preferred embodiment is a hyperbolic tangent function, where the input range is limited to -0.8 to +0.8 to avoid low slope regions. It is done.
[0044]
LPF scan process
The LPF scanning process is shown in FIG. 6A and begins by obtaining the next particle image (analyte) using a low power objective. Then, a neural net classification method is executed. It includes the process of applying a neural network cascade structure to the analyte image, as shown in FIG. 6B. Each neural network takes a selected subgroup of the calculated 198 particle features described above and calculates a classification probability factor ranging from 0 to 1 where the particles meet the net features. The cascade structure of the net helps to improve the accuracy of each neural net result downstream. This is because, if a particle is classified as having a pre-screened operation because it has or does not have a certain property, each net can essentially be designed more accurately. For system efficiency, all 198 particle features are preferably calculated for each particle image, and then the neural net classification process of FIG. 6B is applied.
[0045]
The first neural net applied to the particle image is an AMOR Classifier Net, which determines whether the particle is amorphous. For the preferred embodiment, this net includes 42 inputs for a selected subset of the 198 particle features described above, 20 neurons in the hidden layer, and 2 neurons in the output layer. . The column referred to as “LPF AMOR2” in the tables of FIGS. 9A-9C shows the number of the above 42 particle features selected for use in this net. The first and second outputs of this net each correspond to the probability of whether the particle is amorphous. The one with the higher probability constitutes the net decision. If the net determines that the particle is amorphous, the particle analysis ends.
[0046]
If it is determined that the particle is not amorphous, the SQEP / CAST / OTHER classification net is applied. It determines whether the particle is a squamous cell (SQEP), a stool cell (CAST), or another type of cell. For the preferred embodiment, this net includes 48 inputs for a selected subset of the 198 particle features described above, 20 neurons in the hidden layer, and 3 neurons in the output layer. . The column labeled “LPF CAST / SQEP / OTHER3” in the tables of FIGS. 9A-9C shows the number of the above 48 particle features selected for use in this net. The first, second, and third outputs of this net correspond to the probability that the particle is feces, SQEP, or another particle type, respectively. Any of the highest probabilities constitutes a net decision.
[0047]
If the particle is determined to be a fecal cell, a CAST Classifier Net is applied. It determines whether the particles are leukocyte clamps (WBCC), amorphous fecal cells (HYAL), or unclassified feces such as pathological fecal cells (UNCC). For the preferred embodiment, this net includes 36 inputs for a selected subset of the 198 particle features described above, 10 neurons in the hidden layer, and 3 neurons in the output layer. . The column labeled “LPF CAST3” in the tables of FIGS. 9A-9C shows the number of the above 36 particle features selected for use in this net. The first, second and third outputs of this net correspond to the probability that the particle is WBCC, HYAL or UNCC. Whichever has the highest probability constitutes a net decision.
[0048]
If it is determined that the particle is a squamous cell, that determination is left alone.
[0049]
If it is determined that the particle is another type of cell, the OTHER Classifier Net is applied. It determines whether the particles are non-squamous cells (NSE), such as liver epithelial cells or transitional epithelial cells, unclassified crystals (UNCX), yeast (YEAST), or mucus (MUCS). For the preferred embodiment, this net includes 46 inputs for a selected subset of the 198 particle features described above, 20 neurons in the hidden layer, and 4 neurons in the output layer. The column referred to as “LPF OTHER4” in the tables of FIGS. 9A-9C shows the number of the above 46 particle features selected for use in this net. The first, second, third and fourth outputs of this net correspond to the probability that the particle is NSE, UNCX, YEAST or MUCS. Whichever has the highest probability constitutes a net decision.
[0050]
Returning to FIG. 6A, once the neural net classification determines the particle type, an avoidance rule technique (ART by Abstention Rule) is applied to determine whether the particle should be classified as an artificial structure. . This is because there is no net that gives a classification probability factor high enough to guarantee particle classification. The avoidance rule technique applied by the preferred embodiment is as follows: if the final classification by net structure is HYAL and the CAST probability is less than 0.98 in the SQEP / CAST / Other net The particles are reclassified as artificial structures. Similarly, if the final classification by net structure is UNCC and the CAST probability is less than 0.95 in the SQEP / CAST / Other net, the particle is reclassified as an artificial structure.
[0051]
The next step shown in FIG. 6A applies to particles that withstand the technique by avoidance rules. If the particles are classified by the net structure as UNCC, HYAL or SQEP, the classification is accepted unconditionally. If the particle is classified as another type of particle, a particle uptake test is applied to determine whether the particle should be classified as an artificial structure. The partial capture test determined whether the particle boundary hits one or more particle image patch boundaries, so only a portion of the particle image was captured by the patch image. The partial capture test of the preferred embodiment basically looks at the pixels that form the boundary of the patch to ensure that they represent background pixels. This is done by collecting cumulative intensity histograms on the patch boundaries and calculating the lower and upper limits of these intensities. The lower limit of the preferred embodiment is the greater of the third value from the bottom of the histogram or the 2% value from the bottom of the histogram, whichever is greater. The upper limit is the larger of the third value from the top of the histogram or the value of 2% from the top of the histogram. A patch image is considered partially captured if the lower limit (eg, pixel intensity having a range from 0 to 255) is less than 185. Similarly, if a patch has an upper limit of 250 or less and a lower limit of less than 200 (this is to handle the case where the circular light in the particle image teaches the patch image boundary), it appears to be a partial capture. . All particles that will survive the partial uptake test maintain their classification and the LPF scanning process is complete.
[0052]
In the preferred embodiment, the partial uptake test is also used as one of the particle features used by some of the neural nets. The feature value is 1 if the particle boundary hits one or more particle image patch boundaries, and 0 otherwise. This particle feature is assigned “0” in FIGS.
[0053]
HPF scanning process
The HPF scanning process is shown in FIG. 7A and begins by obtaining the next particle image (analyte) using a high power objective. Two pre-processing artifact classification steps are performed before submitting the particles to neural net classification. The first pretreatment step begins by defining five size boxes (HPF1-HPF5), corresponding to the smallest box in which each of the particles can fit perfectly. In the preferred embodiment, the smallest box HPF5 is 12 × 12 pixels and the largest box HPF1 is 50 × 50 pixels. All particles corresponding to the HPF5 box are classified as artificial structures and removed from further consideration. This is because these particles are too small for an accurate classification that gives a determination of the system.
[0054]
The second preprocessing step finds all remaining particles corresponding to HPF3 or HPF4. It has a cell area of less than 50 square pixels, is neither long nor thin, and classifies them as artificial structures. This second preprocessing step combines the size and aspect ratio criteria. It removes smaller particles that tend to be round. Once a particle corresponding to a cell area of less than 50 square pixels is isolated in an HPF3 or HPF4 box, each such particle will be artificial if it meets one of the following two criteria: Classified as a structure. First, if the square around the particle separated by the particle area is less than 20, the particle is neither long nor thin and is classified as an artificial structure. Second, if the ratio of the eigenvalues of the variance matrix of X and Y moments (also called the stretch value) is less than 20, the particles are not long or thin and are classified as artificial structures.
[0055]
The particle image that has survived the two pre-processing steps described above comprises the neural network cascade structure shown in FIG. 7B. Each neural network takes a selected subgroup of the above calculated 198 particle features and calculates a classification probability factor ranging from 0 to 1 where the particles meet the criteria of the net. As with the net cascade structure, this helps to improve the accuracy of each neural net result downstream, and preferably all of the 198 particle features are for each particle image before the HPF scan begins. Is calculated.
[0056]
The first neural net applied to the particle image is an AMOR classification net, which determines whether the particle is amorphous. For the preferred embodiment, this net includes 50 inputs for a selected subset of the 198 particle features described above, 10 neurons in the hidden layer, and 2 neurons in the output layer. . The column referred to as “LPF AMOR2” in the tables of FIGS. 9A-9C shows the number of the above 50 particle features selected for use in this net. The first and second outputs of this net each correspond to the probability of whether the particle is amorphous. The one with the higher probability constitutes the net decision. If the net determines that the particle is amorphous, the particle analysis ends.
[0057]
If it is determined that the particle is not amorphous, a Round / Not Round classification net is applied. It determines whether the particle shape exhibits a predetermined amount of a circle. For the preferred embodiment, this net includes 39 inputs for a selected subset of the 198 particle features described above, 20 neurons in the hidden layer, and 2 neurons in the output layer. . The column referred to as “HPF ROUND / NOT ROUND2” in the tables of FIGS. 9A-9C shows the number of the above 39 particle features selected for use in this net. The first and second outputs of this net correspond to the probability that the particle is “round” or not. Either of the highest probabilities constitutes a net decision.
[0058]
If the particle is determined to be “round”, a Round Cells Classifier Net is applied. It determines whether the particles are red blood cells (RBC), white blood cells (WBC), non-squamous cells (NSE) such as liver epithelial cells or transitional epithelial cells, or yeast (YEAST). For the preferred embodiment, this net includes 18 inputs for a selected subset of the 198 particle features described above, 3 neurons in the hidden layer, and 3 neurons in the output layer. . The column referred to as “HPF Round 4” in the tables of FIGS. 9A-9C shows the number of the above 18 particle features selected for use in this net. The first, second, third and fourth outputs of this net correspond to the probability that the particle is RBC, WBC, NSE or YEAST. Whichever has the highest probability constitutes a net decision.
[0059]
If it is determined that the particle is not “round”, a Not Round classification net is applied. It applies the Round Cells Classifier Net. It consists of non-squamous cells (NSE) such as red blood cells (RBC), white blood cells (WBC), liver epithelial cells or transitional epithelial cells, unclassified crystals (UNCX), yeast (YEAST), sperm (SPRM) Or it is determined whether it is bacteria (BACT). For the preferred embodiment, this net includes 100 inputs for a selected subset of the 198 particle features described above, 20 neurons in the hidden layer, and 7 neurons in the output layer. . The column referred to as “HPF NOT Round7” in the tables of FIGS. 9A-9C shows the number of the above 100 particle features selected for use in this net. The output of 7 in this net corresponds to the probability that the particle is RBC, WBC, NSE, UNCX, YEAST, SPRM or BACT. Any of the highest probabilities constitutes a net decision.
[0060]
Returning to FIG. 7A, once the neural net classification determines the particle type, an avoidance rule technique is applied to determine whether the particle should be classified as an artificial structure. This is because there is no net that gives a classification probability factor high enough to guarantee particle classification. The avoidance rule technique applied by the preferred embodiment reclassifies the four particle types as artificial structures if certain criteria are met. First, if the final classification by net structure is yeast and the YEST probability is less than 0.9 in the Not Round Cells classification net, the particle is reclassified as an artificial structure. Second, if the final classification by the net structure is NSE and the NSE probability is less than 0.9 in the Round Cells classification net or the round probability is less than 0.9 in the Round / Not Round classification net, The particles are reclassified as artificial structures. Third, if the final classification by the net structure is not round NSE and the NSE probability is less than 0.9 in the Not Round Cells classification net, the particle is reclassified as an artificial structure. Fourth, if the final classification by net structure is UNCX and UNCX probability is less than 0.9 in Round Cells classification net or round probability is less than 0.9 in Round / Not Round classification net, The particles are reclassified as artificial structures.
[0061]
The next step shown in FIG. 7A is a partial uptake test, which applies to all particles that withstand the technique with avoidance rules. The partial uptake test determines whether a particle boundary should be classified as an artificial structure. This is because the particle boundary hits one or more particle image patch boundaries, and therefore only a portion of the particle image is captured by the patch image. Similar to the LPF scan, the partial capture test of the preferred embodiment basically looks at the pixels that form the boundary of the patch to ensure that they represent the background pixels. This is done by collecting cumulative intensity histograms on the patch boundaries and calculating the lower and upper limits of these intensities. The lower limit of the preferred embodiment is the greater of the third value from the bottom of the histogram or the 2% value from the bottom of the histogram, whichever is greater. The upper limit is the larger of the third value from the top of the histogram or the value of 2% from the top of the histogram. A patch image is considered partially captured if the lower limit (eg, pixel intensity having a range from 0 to 255) is less than 185. Similarly, a patch is considered partial capture if the upper limit is less than 250 and the lower limit is less than 200 to handle the case where the circular light of the particle image teaches the patch image boundary.
[0062]
All particles that withstand partial uptake tests maintain their classification. All particles that appear to be partial uptake are further subject to the technique of partial uptake rules, which reclassifies such particles as artificial structures if any of the following six criteria are met.
1. If it corresponds to the HPF1 size box.
2. If it was not classified as either RBC, WBC, BYST or UNCX.
3. If it was classified as an RBC, it would correspond to an HPF2 size box, or if it had a stretch value greater than 5.0.
4). If it was classified as WBC and had a stretch value of 5.0 or greater.
5). If it is classified as UNCX and has a stretch value greater than 10.0.
6). If it was classified as BYST and had a stretch value of 20.0 or greater.
If none of these six criteria is met by the particle image, neural net classification is allowed to be valid even if the beginning is considered partial acquisition, and the HPF scan process is complete. These six rules were chosen to continue partial classification decisions when partial capture does not distort the neural net decision process, but to remove particles where partial capture leads to wrong decisions. .
[0063]
In order to best determine which features should be used for each of the above mentioned neural nets, the feature values input to any given neural network are changed one by one from a smaller amount and the neural network output This is recorded. The feature with the greatest effect on the output of the neural network should be used.
[0064]
Post-processing decision
Once all particle images are classified by partial type, a post-determination process is performed to further increase the accuracy of the classification results. This process takes into account the set of results and removes the classification results that are not considered reliable as a whole.
[0065]
A user-configurable concentration threshold is a type of post-determination process that establishes a noise level threshold for all results. These thresholds can be set by the user. If the neural net classification image concentration is less than the threshold, all particles in the category are reclassified as artifacts. For example, if an HPF scan finds only a few RBCs of all samples, these are incorrect results and these particles are reclassified as artifacts.
[0066]
Excessive amorphous detection is another post-decision process that throws away suspicious results if too many particles are classified as amorphous. In the preferred embodiment, if there are 10 or more non- irregular HPF patches and more than 60% of them are classified as irregular by the neural network, the results of all samples are discarded as unreliable.
[0067]
The preferred embodiment also includes a clear filter of many LPF errors, which discards inconsistent or suspicious results. Unlike HPF particles, LPF artifacts have all shapes and sizes. In many cases, it is not possible to discriminate LPF artifacts from true clinically important analytes with system decisions. In order to reduce false realities for LPF artifacts, many filters are used to look at the aggregate decisions made by the net, throwing away those results that just don't make sense. For example, if the HPF WBC count is less than 9, then all LPF WBCCs should be reclassified as artifacts because leukocyte clamps are probably not present unless enough white blood cells are found. Furthermore, the detection of just a few particles of a certain type should be ignored because it is unlikely that there will be such a small number of these particles. In preferred embodiments, the system comprises 3 or more LPF UNCX detection particles, or 2 or more LPF NSE detection particles, or 3 or more LPF MUC detection particles, or 2 or more HPF SPRM detection particles, or 3 or more LPF YEAST. The detection particles must be found. If these thresholds are not met, each particle is reclassified as an artificial structure. In addition, there must be at least two HPF BYST detection particles in order to receive any LPF YEAST detection particles.
[0068]
Neural network training and selection
Each neural network is trained using a pre-classified image training set. In addition to the training set, a second smaller set of pre-classified images is reserved as a test set. In the preferred embodiment, the commercial program “NeuralWare” published by NeuralWorks is used to perform training. The training ends when the average error of the test set is minimized.
[0069]
This process is repeated for multiple starting seeds and net structures (ie, hidden layers and number of elements in each layer). The final choice is based on satisfying constraints on errors between specific classes as well as the overall average error rate. For example, it is undesirable to distinguish squamous cells from pathological feces. This is because squamous epithelial cells usually occur in female urine specimens, whereas pathological feces present an abnormal situation. Therefore, the preferred embodiment helps nets with SQEP at UNCC error rates less than 0.03 at the expense of more misclassification of UNCC as SQEP. This situation somewhat reduces the sensitivity of UNCC detection, but minimizes false realities in female specimens. It disables the system with a sufficiently high frequency of occurrence, since a high percentage of female urine samples are called abnormal. Therefore, it is desirable to construct this selection criterion for net training using a method that takes into account the cost of a particular error rate in selecting the “optimal” net, as well as minimizing the front-back difference rate.
[0070]
As can be seen from the foregoing, the method and apparatus of the present invention differs from the prior art in the following aspects. In the prior art, at each processing stage, the particles are classified with the remaining unclassified particles taking into account artifacts or unknowns. In order to minimize particle classification as an artifact or unknown, the range of values for classification in the etch stage is large.
[0071]
In contrast, the range of values for classification in the etch stage of the present invention is narrow, resulting in only particles with a greater probability of being so classified, the remainder for further processing with respect to the pretreatment stage. Classification in The multi-net structure of the present invention can be used to divide a decision space by particle attributes or physical properties (eg, its circular shape) and / or by dividing individual and group particle classifications including unknown categories. Use subgroups. This divided the decision space. It creates a probability factor in each decision and uses the available information more effectively. The need is limited, allowing this information to be used to produce the same number of overall decisions but with fewer possible outcomes at each stage. Pre-processing and post-processing allow learning information to be included as part of the decision process. Post-processing allows the use of contextual information, either available from other sources or collected from the actual decision process, to further process the probability factor and improve the decision. The use of neural network certainty measures the image to avoid classes, ie, artificial structures, at a number of processing power stages. In a sense, this multi-neural approach can be seen as forcing image data to run a gauntlet that is likely to be placed in the “I don't know” category at each stage of the gauntlet. This is much more powerful than simply running through a single net. Because, in essence, what is achieved is multiple adaptations of data to a template that is much better defined than a single template can, which allows for efficient use of information. Another way of thinking about this is that data in different subspaces is analyzed, requiring that subspace or other characteristics that it falls from the feature, in a way that fits perfectly or well enough. Is done. The training method of the present invention involves selecting from many nets and reducing the feature vector size rather than just one pass through the training set. A high number of features themselves increases the accuracy of the system, each focusing on a specific set of physical properties.
[0072]
It is to be understood that the invention is not limited to the embodiments shown herein above, but includes any and all modifications within the scope of the appended claims. Therefore, although the invention will be described with respect to the classification of images of biological samples, it will also be understood to include image analysis for any image that has features that can be extracted and used to classify the images. I want. For example, the present invention can be used for face recognition. Features are the shape, size, position and dimensions of eyes, nose, mouth, etc., or more general features such as face shape and size, so that facial images can be identified and classified into pre-determined categories Can be extracted to identify and classify.
[Brief description of the drawings]
FIG. 1 is a flow diagram illustrating a method of the present invention.
FIG. 2 is a schematic view of the apparatus of the present invention.
FIGS. 3A and 3B are flow diagrams illustrating boundary enhancement of the present invention.
FIG. 4 is a diagram illustrating symmetrical feature extraction of the present invention.
5A-5D are diagrams showing skeletonization of various shapes.
FIG. 6A is a flowchart showing LPF scan processing of the present invention.
FIG. 6B is a flowchart of neural network classification used in the LPF scan processing of the present invention.
FIG. 7A is a flowchart showing HPF scan processing of the present invention.
FIG. 7B is a flowchart of neural network classification used in the HPF scan processing of the present invention.
FIG. 8 is a schematic diagram of a neural network used in the present invention.
FIGS. 9A-9C are tables showing particle features used in various neural networks in the LPF and HPF scan processing of the present invention. FIGS.

Claims (10)

Detecting biological particles contained in individual human urine samples, the biological particles being characterized by the shape, symmetry, skeletonization, size, light absorption characteristics, and texture of the particles An imaging device that classifies into a plurality of different final classification classes, including non-squamous epithelial cells, unclassified crystals, mucous membranes, sperm, yeast, budding yeast, leukocyte aggregates, and leukocyte classes:
Means for extracting a plurality of features from a particle image;
Means for selecting and processing the first subgroup of extracted features to make a first quantitative determination of assignment of particles to classes;
Means for selecting and processing a second subgroup of the extracted features to determine a final classification class to which the particles belong, the second subgroup depending on the first subgroup Selected, means;
A means of counting the number of particles in each final classification class;
If the final classification is non-squamous cells, unclassified crystals, mucous membranes, sperm, or yeast, the predetermined classification class count threshold for the selected final classification class and the respective number counted And if the final classification class is a leukocyte aggregate, the threshold of the predetermined classification class count for another final classification class that is a white blood cell and the other final classification class Means for accepting or rejecting the final classification of the particles counted in the selected final classification class based on the comparison with the counted number;
Imaging device. The means for selecting and processing the first subgroup is adapted to determine any one of a plurality of groups of classification classes to which the particles belong;
Means for selecting and processing the second subgroup is adapted to select a second subgroup based on the group of the determined classification class;
The image apparatus according to claim 1. The means for selecting and processing the first subgroup and the means for selecting and processing the second subgroup are such that each decision includes assigning a probability factor to classify the particles. Including means for changing the determined classification class to an artificial classification if one or more of the used probability factors does not exceed a predetermined threshold;
The image apparatus according to claim 1 or 2. Having artificial structure means for determining whether the particle is an artificial structure based on predetermined physical properties of the particle;
The image device according to claim 1, 2 or 3. The artificial structure means determines if a particle image boundary intersects a boundary of an image patch containing the particle image; and if the particle image boundary and the image patch boundary are determined to intersect Means for changing the determined classification class into an artificial structure classification;
The image device according to claim 4. Means for selecting and processing the first subgroup and means for selecting and processing the second subgroup comprise a plurality of neural networks;
The image device according to claim 1. Comprising means for training the neural network by selecting and processing sub-groups of extracted features using a training set of known particles together with a test set of particles,
The means for training the neural network iteratively trains the neural network until the correct rate of determination of the classification class of the test set of particles reaches a predetermined value;
The image device according to claim 6. The means for extracting is
Means for defining a first line segment across the centroid of the image of the particle;
Means for defining a second line segment and a third line segment that extend perpendicularly away from the first line segment in opposite directions at a point along the first line segment;
Means for using the difference between the lengths of the second line segment and the third line segment to calculate the extracted symmetry feature of the image of the particles;
The image device according to claim 1. The means for extracting comprises means for squashing the boundaries of the image of the particles in an orthogonal direction to form one or more line segments;
The image device according to claim 1. At least one of the plurality of extracted features is a measurement of a spatial distribution of the image of particles, and at least another of the plurality of extracted features is a measurement of a spatial frequency domain of the image of particles. Is characterized by:
The image device according to claim 1.

Images