JP3338625B2 - Object identification device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an improvement of an object identifying apparatus for identifying an amorphous object from a biological microscope image or a fluorescence microscope image of a sample containing a particle component such as blood, urine, and a pathological specimen.

[0002]

2. Description of the Related Art An apparatus of this type is disclosed in Japanese Patent Laid-Open No. 6-13 / 1994.
No. 8417, JP-A-8-29140, JP-A-8-30
No. 4390, for example. Each of these apparatuses detects a power spectrum by performing a two-dimensional Fourier transform on an image obtained by an imaging unit, and identifies an irregular-shaped object based on the power spectrum.

Some of these apparatuses extract a part of an image obtained by an image pickup means and use the extracted part as an image to be analyzed. For example, one leukocyte is first detected, and then the nucleus portion of the leukocyte is extracted. According to this, leukocytes can be identified for each type. Conventionally, such extraction is performed by using RGB of a color image obtained by an imaging unit.
(Red, Green, Blue) concentration information was directly used.

[0004]

As described above, conventionally, when a specific portion of an irregular-shaped object is extracted, the RGB density information of a color image is directly used. For this reason, when the brightness of the irregular-shaped object fluctuates due to a light source, staining conditions, or the like, an image area other than the image area to be extracted may be extracted, or a part of the image area to be extracted may not be extracted. I was

It is an object of the present invention to stably extract an image area to be analyzed in image extraction for identifying an irregular object, even if the brightness of the irregular object to be imaged varies. Is to be able to do it.

[0006]

In the present invention SUMMARY OF THE INVENTION, RGB imaging means for imaging a material containing amorphous object to be identified as a color image, the color information from the RGB density information of a color image the imaging means has captured A hue detection unit that detects by HIS conversion, and a part to be analyzed in the image captured by the imaging unit based on the hue information detected by the hue detection unit or based on the RGB density information and the hue information. Extracting means for extracting, two-dimensional Fourier transform means for two-dimensionally Fourier-transforming the image of the analysis target portion extracted by the extracting means, and an image of the analysis target part based on the conversion result of the two-dimensional Fourier transform means. Power spectrum detecting means for detecting a power spectrum; and a power spectrum based on the power spectrum detected by the power spectrum detecting means. There are the a configuration having a identifying means for identifying irregular object.

In the apparatus configured as described above, an image to be analyzed is extracted based on hue information. For this reason,
Compared with a device that directly extracts an analysis target image from RGB density information, an analysis target image can be stably extracted even if the brightness of the target object fluctuates due to a light source or staining conditions.

In the apparatus having the above configuration, if the hue detecting means is an analog arithmetic circuit, the hue can be detected quickly with an extremely simple configuration.

Further, if the device having the above configuration is a device for identifying blood cells, it is possible to accurately identify blood cells.

In the apparatus having the above structure, if the two-dimensional Fourier transform means is means for performing a two-dimensional Fourier transform by calculation using digital data of an extracted image, the hardware of the apparatus is simplified. .

On the other hand, two-dimensional Fourier transform means is
If an optical device that performs optical Fourier transform is used, two-dimensional Fourier transform can be performed quickly. further,
In the present invention, in an object identification device for extracting an analysis target portion from a captured color image and identifying an amorphous object, an imaging unit for imaging a substance containing the amorphous object to be identified as a color image, and the imaging unit includes: Converts hue information from RGB density information of a captured color image to RGB-HIS conversion
A hue detection means for detecting the hue information detected by the hue detection means, or an extraction part for extracting a portion to be analyzed from the image picked up by the imaging means based on the RGB density information and the hue information Means .

[0012]

FIG. 1 is a block diagram of a leukocyte identification apparatus according to the present invention. The microscope 1 has a sample stage on which a sample is placed. The sample stage is movable by a sample stage moving means 2 on a plane orthogonal to the optical axis of the objective lens of the microscope 1. The identification target object position detecting means 3 is means for detecting that the identification target object has reached a specific position. The sample stage control means 4 includes the identification object position detection means 3
Is a means for controlling the sample stage moving means 2 based on the detection signal. The focus detecting means 5 is a means for detecting whether the microscope 1 is in focus with respect to the sample, and the focus adjusting means 6
Is a means for changing the position of the sample with respect to the microscope 1. The automatic focus control means 7 is a means for controlling the focus adjustment means 6 based on the detection signal of the focus detection means 5 to focus the microscope 1 on the sample on the sample stage.

The image pickup means 8 is a means for picking up a still color image at a certain point in time from a microscope image of the sample on the sample table, and uses, for example, a CCD camera. Memory 9
Stores an image captured by the image capturing means 8.

The hue detecting means 10 is a means for detecting the hue of each pixel of the image picked up by the image pick-up means 8. The specific configuration of this means will be described later.

The first image preprocessing means 11 selects a pixel having a hue value set based on the image stored in the memory 9 and the hue detected by the hue detection means 10, and selects the set hue. This is a means for extracting an image composed of pixels having a value.

The auxiliary parameter detecting means 12 is a means for detecting the color and area of the image extracted by the first image preprocessing means 11. The second image pre-processing means 13 is means for enlarging or enhancing when the image extracted by the first image pre-processing means 11 is small or not clear.

The two-dimensional Fourier transform means 14 is means for performing a two-dimensional Fourier transform of the image supplied from the second image preprocessing means 13. This means is disclosed in JP-A-8-291.
For example, optical means may be used as described in JP-A No. 40, or a computer may be used as described in JP-A No. 8-304390. The power spectrum detecting means 15 is a means for obtaining a power spectrum based on the conversion result of the two-dimensional Fourier transform means 14.

The identification judging means 16 identifies the image extracted by the first image preprocessing means 11 based on the power spectrum obtained by the power spectrum detecting means 15 and the color and area data detected by the auxiliary parameter detecting means 12. Is a means for performing The identification determining means 16 has a learning function, and is connected to a learning table 17 for storing a learning result.

The control section 18 controls the entire apparatus, and includes an automatic focus control section 7, a sample stage control section 4, a first image preprocessing section 11, a second image preprocessing section 13, It controls the two-dimensional Fourier transform means 14.

The hue detecting means 10 has a specific configuration as shown in FIG. This means uses RGB-HIS conversion (conversion from RGB density into hue H, lightness I, and saturation S) defined by Haydn. Haydn's definition is as follows. H is assumed to be normalized to 1 ≦ H ≦ 4, and when R, G ≧ B, H = (GB−B) / (R + G−2B) +3 (1) When G, B ≧ R , H = (B−R) / (G + B−2R) +1 (2) When R and B ≧ G, H = (R−G) / (B + R−2G) +2 (3) . In the above equations (1) to (3), the constant term is set to 0, the equation is expanded, and the division term is eliminated to define as follows. H1 = H-3, H2 = H-1, H3 = H
When R, G ≧ B, H1 (R + G−2B) = (GB) (4) When G, B ≧ R, H2 (G + B−2R) = (BR) (5) When R, B≥G, H3 (B + R-2G) = (RG) (6)

The apparatus is used when it is known in advance that the pixels of the object to be identified satisfy R, B ≧ G. Therefore, equation (6) is used. It is also assumed that the image to be extracted is known to have a hue equal to or greater than a certain value H0.

In FIG. 2, an analog arithmetic unit 21 is a circuit for calculating B + R-2G from the R, G, and B densities of each pixel provided from the imaging means 8, and an analog arithmetic unit 22
Is a circuit for calculating R-G from the density of R, G, B of each pixel provided from the imaging means 8. The voltage divider 23 is a voltage divider that can change the degree of voltage division by a voltage division setting signal provided from the control unit 18 and is a circuit that multiplies (B + R-2G), which is the output of the analog operation unit 21, by H3. Comparator 2
Reference numeral 4 denotes a circuit for comparing the output H3 (B + R-2G) of the voltage divider 23 with the output (RG) of the analog arithmetic unit 22 and outputting the result. Therefore, the comparator 24 compares the left side and the right side of the equation (6). Where H3 = H
It is set to 0-2. Therefore, the comparator 24
If it is determined that H3 (B + R-2G)> (RG), it is understood that the hue of the pixel is H0 or more.

The hue detecting means 10 may be constituted by an analog arithmetic circuit as shown in FIG. In this case, in Equations (4) to (6), the following equation in which the subtraction term on the right side is shifted to the left side is the basis. When R, G≥B, H1 (R + G-2B)-(GB) = 0 (7) When G, B≥R, H2 (G + B-2R)-(BR) = 0 ( 8) When R, B≥G, H3 (B + R-2G)-(RG) = 0 (9)

As described above, this apparatus is used for
The pixel of the object to be identified is a case where it is known in advance that R, B ≧ G. Therefore, equation (9) is used. It is also assumed that the image to be extracted is known to have a hue equal to or greater than a certain value H0.

In FIG. 3, an analog operation unit 21, an analog operation unit 22 and a voltage divider 23 are the same as the circuit shown in FIG. The difference calculator 25 is a circuit that calculates the difference between the output of the voltage divider 23 and the output of the analog calculator 22. Comparator 2
Reference numeral 7 denotes a circuit for comparing the voltage 0 V of the reference voltage power supply 26 with the output of the subtractor 25.

The difference calculator 25 outputs the output H3 (
B + R−2G) and the output (R−
G), the comparator 27 calculates the difference and the reference voltage 0
Compare with V. Therefore, the comparator 27 compares the left side and the right side of Expression (9). Here, H3 = H0-
It is set to 2. Therefore, the comparator 27 determines that H3
If it is determined that (B + R-2G)-(RG)> 0, it is understood that the hue of the pixel is H0 or more.

Next, the operation of the apparatus having the above configuration will be described with reference to the flowchart of FIG. First, the operator places the sample on the sample stage of the microscope 1 (S1). Here, the sample is stained blood, and the amorphous object to be identified is white blood cells. Next, the control unit 18 causes the automatic focus control unit 7 to control the focus position. Thereby, the automatic focus control means 7 operates the focus adjustment means 6 based on the focus position detected by the focus detection means 5 so that the lens of the microscope 1 is focused on the sample on the sample stage (S2). ). Next, the controller 18 instructs the sample stage controller 4 to start moving the sample stage (S3). As a result, the sample stage moves, and the identification target object position detecting means 3 detects when one identification target object reaches a specific position. The sample stage control means 4 determines whether or not the identification object has been detected based on the detection result of the identification object position detection means 3 (S4), and stops the movement of the sample stage if there is this detection (S5). If there is no such detection, the process returns to step S3.

Next, the image pickup means 8 picks up a microscope image (S6). That is, the imaging means 8 captures a microscope still image of one detected object to be identified and outputs it to the memory 9 and the hue detection means 10. The memory 9 stores this. The hue detecting means 10 detects a hue for each pixel of the given image (S7). For example, if the hue detecting means 10 has the configuration shown in FIG. 2 or FIG. 3, it is detected whether the hue of each pixel is larger or smaller than the hue value H0.
Next, the first preprocessing unit 11 extracts an image based on the hue detection result provided from the hue detection unit 10 or based on RGB density information and hue information (S
8). This image is an image to be analyzed, for example, an image of a nucleus of a white blood cell.

Next, the auxiliary parameter detecting means 12
The color and area of the image extracted by the image preprocessing means 11 are detected (S9). Next, the second image preprocessing means 13
It is determined whether the image extracted by the image preprocessing means 11 is small or not clear, that is, whether enlargement or enhancement is necessary (S10), and if necessary, enlargement or enhancement is performed (S11).

Next, the two-dimensional Fourier transform means 14 outputs the second
The two-dimensional Fourier transform of the image provided from the image preprocessing means 13 is performed (S12). In step S10,
If it is determined that the enlargement or emphasis processing is unnecessary, the processing in step S11 is omitted. Next, the power spectrum detecting means 15 is a two-dimensional Fourier transforming means 14.
(S1)
3).

Next, the identification determining means 16 sets the learning table 1
7, the image extracted by the first image preprocessing unit 11 is identified based on the power spectrum obtained by the power spectrum detection unit 15 and the color and area data detected by the auxiliary parameter detection unit 12. (S14). Here, leukocytes are identified as, for example, lymphocytes, neutrophils, eosinophils, basophils and the like. The learning result of the identification determining means 16 is stored in the learning table 17.

The sample stage control means 4 stores the process of moving the sample stage, thereby determining whether all the identification target objects, that is, leukocytes have been identified (S15), and all the identification target objects are identified. If it is determined that the processing has been completed, the processing of the present apparatus is terminated; otherwise, the processing returns to step S3.

According to the present apparatus, the identification determining means 16 performs the identification based on not only the power spectrum but also the color and area data, so that more accurate identification can be performed. Further, according to the present apparatus, when the extracted image is small, the image is enlarged, and when the extracted image is unclear, the image is emphasized, so that more accurate identification can be performed.

In the present apparatus, the embodiment in the case of R, B ≧ G is shown. However, depending on the staining method, the condition that R, B ≧ G may not be satisfied at a site to be extracted. In such a case, based on the size relation of R, G, and B,
Equations (4), (5) or (7), (8) may be selected and used.

In the present apparatus, the hue detecting means 10 specifically uses H of RGB-HIS conversion defined by Haydn, but instead of this, the following RGB is used.
-H of HIS conversion may be used. (A) RGB-HIS conversion by Raines definition R, G, B, H are 0 <R <1, 0 <G <1, 0 <B <
H = arctan {(GB) (2R-GB)} (B) RGB-HIS based on HSI hexagonal pyramid color model
Transformation Consider a cube that touches three axes in an RGB orthogonal coordinate system. The main diagonal axis of the RGB cube is a lightness axis I, and
Let max {R, G, B} and Imin = min {R, G, B}.
At this time, r, g, and b are expressed as follows: r = (Imax-R) / (Imax-Imin) g = (Imax-G) / (Imax-Imin) b = (Imax-B) / (Imax-Imin) By definition, when Imax = 0, H = undefined When Imax ≠ 0 and R = Imax, H = (π / 3) ×
(Bg) When Imaxmax0 and G = Imax, H = (π / 3) ×
(2 + r−b) When Imax ≠ 0 and B = Imax, H = (π / 3) ×
(4 + gr) However, when H <0, 2π is added to the value of H. (C) RGB-HIS conversion RGB-HIS conversion using a bi-SI hexagonal pyramid color model Imax = max {R, G, B}, Imin = min {R, G, B}
And the lightness I is set to I = (Imax + Imin) / 2.
At this time, r, g, and b are expressed as follows: r = (Imax-R) / (Imax-Imin) g = (Imax-G) / (Imax-Imin) b = (Imax-B) / (Imax-Imin) By definition, when Imax = Imin, H = indefinite. When Imax ≠ Imin, and when R = Imax, H = (π / 3).
× (b−g) When Imax ≠ Imin and G = Imax, H = (π / 3)
× (2 + r−b) When Imax ≠ Imin and B = Imax, H = (π / 3)
× (4 + gr) However, when H <0, 2π is added to the value of H.

The hue detecting means 10 shown in FIGS. 2 and 3 is for determining whether a set value of one hue is larger or smaller.
If another set of circuits for processing the output in the same manner is provided, and the voltage dividing degrees of the voltage dividers 23 of the respective circuits are set differently, the upper and lower limits of the hue can be set. Can be detected.

In the present apparatus, the hue detecting means 10
Although an analog arithmetic circuit is used, this may be a digital computer.

[0038]

According to the first aspect of the present invention, the RGB density information of a color image is converted into hue information by RGB-HIS conversion to extract an image of a portion to be analyzed. . Such a hue is independent of the brightness, so that even if the brightness of the irregular-shaped object to be identified fluctuates due to the light source or the staining conditions, it is possible to stably extract the image of the analysis target portion. Therefore, according to the present invention, the identification accuracy can be improved. Also, as in the conventional case, the R
When the analysis target portion is directly extracted from the GB density information, it is necessary to determine the optimal extraction condition for each of the three parameters. However, since only one parameter is required, the optimal extraction condition is simply determined. be able to.

According to the second aspect of the present invention, since the hue detecting means is an analog arithmetic circuit, the hue can be detected quickly with an extremely simple configuration.

According to the third aspect of the present invention, blood cells can be accurately identified.

According to the fourth aspect of the present invention, since the Fourier transform means is a means for performing a Fourier transform by calculation using digital data of an extracted image, the hardware of the apparatus is simplified. .

According to the fifth aspect of the present invention, since the Fourier transform means is an optical device for performing an optical Fourier transform, it can quickly perform a Fourier transform. According to the invention described in claim 6, even if the brightness changes due to the light source or the staining conditions, the analysis target portion can be stably extracted, and the identification accuracy can be improved.

[Brief description of the drawings]

FIG. 1 is a diagram showing an overall configuration of an apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing a specific example of the hue detecting means 10 shown in FIG.

FIG. 3 is a diagram showing another specific example of the hue detection unit 10 shown in FIG.

FIG. 4 is a flowchart for explaining the operation of the apparatus shown in FIG. 1;

[Explanation of symbols]

 Reference Signs List 1 microscope 8 imaging means 10 hue detection means 11 first image preprocessing means (extraction means) 14 two-dimensional Fourier transform means 15 power spectrum detection means 16 identification determination means

Continued on the front page (72) Inventor Yasuhiro Takemura 585 Tomicho, Funabashi-shi, Chiba Sumitomo Osaka Cement Co., Ltd. (72) Inventor Toshiharu Takei 585 Tomihara-cho, Funabashi-shi, Chiba Sumitomo Osaka Cement Co., Ltd. Inside the laboratory (72) Inventor Yutaka Nagai 1-31-4 Nishi-Ochiai, Shinjuku-ku, Tokyo Nihon Kohden Kogyo Co., Ltd. (72) Inventor Shigeko Kato 1-31-4, Nishi-Ochiai, Shinjuku-ku, Tokyo In Kogyo Co., Ltd. (56) References JP-A-8-148571 (JP, A) JP-A-8-29140 (JP, A) JP-A-8-304390 (JP, A) JP-A-8-297099 (JP, A) , A) JP-A-6-138417 (JP, A) EP 713086 (EP, A1) (58) Fields investigated (Int. Cl. ⁷ , DB name) G01N 21/84 G01J 3/46 G01N 21/27 G01N 21/64 G01N 33/49

Claims (6)

(57) [Claims] An image pickup means for picking up a substance containing an amorphous object to be identified as a color image, and hue detection for detecting hue information from RGB density information of the color image picked up by the image pickup means by RGB-HIS conversion. Means for extracting a part to be analyzed from the image taken by the imaging means based on the hue information detected by the hue detection means or based on the RGB density information and the hue information; Image of the analysis target part extracted by the
Two-dimensional Fourier transform means for performing two-dimensional Fourier transform; power spectrum detecting means for detecting a power spectrum of the image of the analysis target portion based on the conversion result of the two-dimensional Fourier transform means; and power detected by the power spectrum detecting means An identification means for identifying the irregular-shaped object based on a spectrum. 2. The object identification apparatus according to claim 1, wherein the hue detection means is an analog operation circuit. 3. The object identifying apparatus according to claim 1, wherein the irregular-shaped object to be identified is a blood cell. 4. The object identification apparatus according to claim 1, wherein the two-dimensional Fourier transform means is a means for performing a Fourier transform by calculation using digital data of the extracted image. 5. The object identification apparatus according to claim 1, wherein the two-dimensional Fourier transform means is an optical device that performs an optical Fourier transform. 6. An object identifying apparatus for extracting a portion to be analyzed from a captured color image and identifying an amorphous object, an imaging unit for imaging a substance containing the amorphous object to be identified as a color image, and the imaging unit. Hue detecting means for detecting hue information from RGB density information of a color image captured by RGB-HIS conversion , based on the hue information detected by the hue detecting means, or based on the RGB density information and the hue information Extracting means for extracting a portion to be analyzed from the image captured by the image capturing means.