JPH09311102A - Flow type particle image analyzing method and device thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain the optimum focusing position of a particle image without carrying out complicated processing, in a flow type particle image analizing device, for adjusting the focus. SOLUTION: A particle image characteristic parameter related to the focus adjustment from a characteristic extraction circuit 27, and a particle kind necessary for the adjustment of a focus out of the results of a particle classification/ discrimination of a discrimination decision circuit 28 are taken out, and the amount of dislocation from the focusing position is calculated by a focal position judging part 60. In the focal position judging part 60, the average value of the characteristic parameters of a plurality of particles is calculated by a parameter average processing part 61, and the amount of dislocation from the focusing position is obtained by a dislocation calculating part 62. The dislocation calculating part 62 for obtaining the dislocation from the focusing position is made up of a neural network.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a particle image analysis method and apparatus for taking an image of particles suspended in a flowing liquid and analyzing the particles, and more particularly to a flow type method for automatically focusing a particle image. The present invention relates to a particle image analysis method and apparatus.

[0002]

2. Description of the Related Art In the technical field of conventional flow-type particle image analysis, a liquid sample in which particles are suspended is flown into a flow cell, and static particle images in continuously flowing sample samples are picked up and each static image is taken. Particle classification and analysis based on particle images are disclosed in Japanese Patent Publication No. 57-500995 and Japanese Patent Laid-Open No. 63-500995.
It is described in Japanese Patent Application Laid-Open No. 94156, Japanese Patent Application Laid-Open No. 4-72544, and the like. In these prior arts, it is stated that a flat sample flow in which the shape of the sample flow in the flow-type particle image analyzer is made extremely thin in the optical axis direction of the image pickup system is required. The need for a flat flow with a small thickness is because a clear image cannot be obtained unless the sample flow thickness is within the depth of focus of the microscope objective for capturing the particle image.

A method of automatically adjusting the focus of a particle image in a flow type particle image analyzer is described in JP-A-4-81640. The autofocusing method described here quantifies the image sharpness of the entire particle image and adjusts the position of the flow cell or objective lens so that the value of this evaluation parameter is maximized. Focus adjustment requires a process of calculating the value of the evaluation parameter each time while moving the position of the objective lens or the flow cell, and an algorithm for finding the maximum value and setting the objective lens at that position. .

[0004]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention
In the focus adjustment with No. 40, the evaluation parameter for finding the in-focus position can be easily calculated, but then the flow cell or the objective lens is moved once to maximize this evaluation parameter and the position is adjusted. It is necessary to re-calculate the evaluation parameters in and compare each value with the previous value to determine step by step. Then, in the next step, it must be decided in which direction the objective lens or flow cell should be moved. It may be difficult to find the maximum value near the in-focus position. This is because the value of the evaluation parameter changes gradually near the in-focus position. Therefore, there is a problem that the closer to the focus point, the longer the focus adjustment processing time including the mechanical system. In normal automatic focus adjustment, it is impossible to know how much the state at the start of focus adjustment deviates from the in-focus position. Therefore, the in-focus position must be found through the above-described processing.

Japanese Patent Publication No. 42-14096 and Japanese Patent Publication No. 58-41486
The issue also describes parameters for evaluating the sharpness of the entire image. Since all of these observe changes in sharpness in one image unit, it is necessary to see changes in sharpness in the same image. However, the flow-type particle image analysis device has a structural problem that the same image cannot be reproduced. That is, since the sample particles are constantly flowing, the same image cannot be captured, it is difficult to see the change in the evaluation parameter in the same image, and the evaluation parameter may vary from image to image. .

An object of the present invention is to provide a method and an apparatus for easily knowing an optimum focusing position for capturing a particle image in a flow type particle image analyzing apparatus without performing complicated processing.

[0007]

[Means for Solving the Problems] A sample liquid in which particles are suspended is surrounded by a cleansing liquid and flowed in a flow cell, and the sample liquid is irradiated with a light beam to image the particles in the sample liquid, and an image of the captured particle is captured. The present invention relates to a flow type particle image analysis method and apparatus for performing image analysis to classify particles, and more particularly to a focus adjustment of an optical system, which has the following features.

(1) It is characterized in that it has means for detecting how much it deviates from the current in-focus position by using one or a plurality of particle characteristic parameters used for particle classification.

(2) In 1), in order to detect the degree of deviation from the in-focus position, the focus position with respect to the value of one or a plurality of particle characteristic parameters when the focus positions are sequentially shifted in advance. It is characterized by having a first neural network for learning the relationship of deviation.

In (3) 2), when one or a plurality of particle characteristic parameters fluctuate because the sample flow thickness or the particle shape is not constant, each of the particle characteristic parameters of a plurality of particles is changed. It is characterized in that the learning of the first neural network is performed based on the average value of.

In (4) 3), when learning the relationship between the amount of deviation from the in-focus position and the particle image characteristic parameter, the internal characteristics are measured based on the data obtained by measuring the particle characteristic parameter at a position other than the measurement point. It is characterized by adding interpolated and extrapolated values.

In (5) 2-4), the first neural network learned about the particle feature parameter and the amount of deviation from the focus position is used, and the same one used for learning one or a plurality of particle images in the sample. It is characterized by using the particle feature parameter to obtain information on how much the current focus position is deviated.

In (6) 5), the average value of a plurality of particle images is used as the value of one or a plurality of particle characteristic parameters used for detecting how much the current focus position is deviated. , The deviation from the in-focus position is detected.

(7) In 2), the neural network output indicating the deviation from the in-focus position is configured to match the actual geometric deviation amount from the actual in-focus position. .

(8) In 5), when the output of the neural network is not sufficient for the discrete output, a means for interpolating the data from the output values of a plurality of points to more accurately calculate the shift amount of the in-focus position is provided. Is characterized by.

(9) Using standard particles of uniform size,
It is characterized in that the deviation from the focus position of each particle is measured to obtain information about the sample flow.

In (10) 9), among the information of the sample flow, information about the thickness of the sample flow is obtained.

(11) 10) is characterized in that, among the sample flow information, variation information regarding the thickness of the sample flow is obtained.

In (12) 1 to 11), in order to obtain information on the deviation from the in-focus position, texture information of the particle image or a characteristic parameter for extracting high frequency components of the image is used as the particle image feature amount. Characterize.

(13) 12) is characterized in that information such as density dispersion, density contrast, density differentiation is used as the texture information.

In (14) 1 to 13), a sample in which standard particles of uniform size are suspended is used as the particles for detecting the deviation from the in-focus position.

(15) In the standard particle sample in (14), a continuous particle in which a plurality of particles are connected, an impurity particle not related to focus adjustment, and one standard particle are classified and identified, and And a particle classification / identification process for removing impurity particles.

(16) In the standard particle sample according to 15), a continuous particle in which a plurality of particles are connected, an impurity particle not related to focus adjustment, and a single standard particle are classified and identified, and And a second neural network for removing impurity particles.

In (17) 1 to 13), a second neural network for identifying a specific particle in the measurement sample is provided as a particle for detecting the deviation from the in-focus position, and the particle feature of the specific particle is provided. It is characterized in that the deviation from the in-focus position is detected by using the first neural network based on the parameters.

(18) 16-17) is characterized in that the second neural network relating to particle classification and the first neural network for detecting the deviation from the in-focus position are combined and configured as one neural network. And

(19) 1 to 18) is provided with means for adjusting the optical system to the in-focus position based on the deviation information from the in-focus position.

(20) 19) is characterized in that it has means for adjusting the optical system to the in-focus position by one operation based on the deviation information from the in-focus position.

In (21) 1 to 18), the amount of deviation from the in-focus position is displayed by using an external output means based on the information on the deviation from the in-focus position.

(22) At the stage of learning the relationship between the deviation from the in-focus position and the particle feature parameter in 2), learning of the first neural network is performed for each flow type particle image analyzer. Characterize.

(23) In 2), the normalization constant of each flow type particle image analyzer, that is, the average of the particle image characteristic parameters used to obtain the deviation information from the in-focus position at the in-focus position. It is characterized by normalizing the measured particle feature parameter with a value.

(24) A flow-type particle image analyzer having a plurality of measurement modes is characterized in that it has means for measuring the amount of deviation from the in-focus position for each measurement mode.

In (25) 24), the deviation amount from the in-focus position is measured for each of a plurality of measurement modes, and the deviation amount from the in-focus position is measured for each measurement mode during actual sample measurement. On the basis of this, it is characterized in that the focus adjustment conditions are switched.

Next, the present invention will be described more specifically.

When the sample liquid in which the particles are suspended is surrounded by a cleansing liquid and flowed in a flow cell, in particular, in a flow type particle image analysis method of capturing a still image of particles by pulsed light irradiation and analyzing the particles in the image. , In order to get a clear grain image, the focus adjustment must be done correctly. Generally, these flow type particle image analysis methods are required to perform image processing for classifying and identifying individual particles, feature extraction for each particle, and classification / identification processing using feature parameters in real time. Classification of these particles
Some of the particle feature parameters used for identification can be diverted to obtain information about the deviation from the focus position. As the characteristic parameter to be used, one or a plurality of parameters corresponding to the defocus information are used. These parameters need not have a linear relationship with the defocus information.

The neural network method is used as one of the means for obtaining the information on the deviation from the in-focus position from the particle image characteristic parameter. It is assumed that the neural network has a three-layer structure including an input layer, an intermediate layer and an output layer. When using a neural network, a learning step is necessary to determine the internal structure of the network in advance. The learning of the neural network uses the well-known backpropagation method. A plurality of particle image feature parameters are input as inputs, and a plurality of output terminals representing deviations from the in-focus position are prepared as outputs. As a condition, the deviation from the in-focus position is gradually changed and the learning is repeated. The shift amounts from the in-focus point do not have to be taken at equal intervals, but it is more convenient to express the shift amounts later by collecting data at equal intervals.

The neural network representing the deviation from the focus position where the internal structure is determined by learning will be referred to as the first neural network here.
Usually, the shape of the sample particles used for learning is not constant, and the sample flow has a certain size, so that the particle image feature parameters obtained from the particle images may have large fluctuations. Therefore, one state
Calculate the average value of the feature parameters for multiple particle images so that the (deviation from the in-focus position) can be determined,
The learning of the first neural network is preferably executed based on the average value of the characteristic parameters. By using the average value, the learning time of the neural network can be greatly reduced, and the measurement result of the amount of deviation from the in-focus position is stable. In this case, a process of normalizing each particle characteristic parameter with the particle characteristic parameter at the in-focus position may be added.

As described above, when the average value of the characteristic parameters is used, the fluctuation of the data becomes small. Further, it becomes possible to interpolate or extrapolate the value for the deviation from the in-focus position other than the measurement points by a spline function or the like, and the learning data can be increased even if there are few measurement points. Using the first neural network learned in this way, using the same particle feature parameter used for learning one or a plurality of particle images in the sample, how much is deviated from the current in-focus position You can know the information.

However, as described at the time of learning, due to the problem of sample flow thickness and the difference in particle size,
The particle characteristic parameter changes, and as a result, the amount of deviation from the in-focus position changes. In such a case, it is possible to know the amount of deviation from the average focus position of the measurement sample by calculating the average value of multiple feature parameters and inputting them to the neural network as in the case of learning. You can The output of the neural network that shows the amount of deviation from the in-focus position is determined by the amount of deviation from the in-focus position during learning, so it is possible to correspond to a geometrical distance in the strict sense, Information on the step feed amount and the rotation angle of the pulse motor may be used. In addition, the output result of the neural network can only obtain information on the discrete focal positions for the number of outputs, but if this is not enough, it is more accurate by interpolating the data from the output values of multiple points. It is possible to calculate the amount of deviation from the in-focus point. Conversely, using standard particles of uniform size,
By examining the amount of deviation from the in-focus position for each particle, it is possible to know the deviation of the sample flow from the average in-focus position in particle units. As a result, it is possible to obtain information on the sample flow, especially information in the thickness direction. In addition, by observing this time change, it is possible to observe the fluctuation of the sample flow in the optical axis direction.

In order to obtain the information about the deviation from the in-focus position, it is preferable to use the texture information of the particle image or the characteristic parameter for extracting the high frequency component of the image as the particle image characteristic parameter. As the texture information, information such as density dispersion, density contrast, and density differentiation is particularly suitable. Further, it is best to use, as sample particles for detecting the deviation from the in-focus position, those in which standard particles of uniform size are suspended.

As the particles used for focus adjustment, polystyrene particles of uniform size which are commonly used as standard particles are used. Since these standard particles have a large refractive index with respect to water, an optical image of the particles is generally seen as a bright image or a dim image before and after the in-focus position due to the lens effect. This change is reflected in the density information and the texture information among the above-mentioned particle characteristic parameters. It is confirmed by the change of these parameter values before and after focusing. When measuring the deviation from the in-focus position based on a blood sample, red blood cells of relatively uniform size are suitable. Red blood cells are also suitable for urine samples.

In the above-mentioned standard particles or red blood cells, a plurality of particles may be connected and image-processed as if they were one particle. In addition, it is necessary to remove unnecessary particles existing in the sample that are not related to focus adjustment. For this reason, before calculating the deviation from the in-focus position, 1
It is advisable to perform an operation to classify and identify individual particles, concatenated particles, and unnecessary particles. In the blood sample, one red blood cell, a plurality of connected red blood cells, white blood cells, etc. are classified and identified, and only one red blood cell particle is taken out. After performing this operation, it is led to the input of the first neural network for calculating the deviation information from the in-focus position. It is better to use averaged values for the parameters. In order to classify and identify white blood cells and connected particles from connected particles, unnecessary particles, or a blood sample, particle classification / identification processing of the part originally used for particle image analysis may be used. A second neural network may be used to classify and identify white blood cells and connected particles from the above-mentioned connected particles, unnecessary particles, or a blood sample. In practice, the first and second neural networks may be combined into one and configured to be regarded as a single neural network.

If the amount of deviation from the in-focus position can be measured using the method described above, the imaging optical system composed of the microscope objective lens can be adjusted based on this value. Moreover, since the amount of deviation from the in-focus position can be directly obtained, the optical system can be adjusted to the in-focus position by one measurement. Focusing automation can be performed.

Normally, even if each characteristic parameter does not have a linear relationship with the amount of deviation from the in-focus position, the output result of the first neural network can correctly correspond to the amount of deviation from the in-focus position. . When the focus adjustment is not automated, the amount of deviation from the in-focus position can be displayed using the external output means. The focus position can be manually adjusted while observing the display result. In this case, it is not necessary to monitor the actual particle image separately. When the step of learning the relationship between the amount of deviation from the in-focus position and the particle image characteristic parameter is executed for each one of the flow type particle image analysis devices, there is an advantage that the focus adjustment is optimized for each device.

Further, since it takes time to execute the neural network learning exactly one by one up to this point, the average value of the characteristic parameters at the focus position in each flow type particle image analyzer is measured, This value is regarded as the device constant of each device, and the characteristic parameters measured at the time of actual measurement are normalized by the device constant of each device and input to the first neural network for calculating the deviation from the in-focus position. You can also do it. In this case, a strictly correct result is shown at the in-focus position, but in some cases, the characteristics of the characteristic parameters of individual devices may change due to factors such as the imaging system and variations in emission intensity. In some cases, the lag information may not be reflected.

Generally, some flow-type particle image analyzers have a plurality of measurement modes. That is, the flow rate of the sample flow may be changed, or the size of the sample flow may be changed in order to increase the measurement volume. In this way, when the conditions of the sample flow are changed, the center position of the sample flow in each measurement mode may shift. In such a case, it is necessary to measure the amount of deviation from the in-focus position for each measurement mode. When the same standard particle is used in a plurality of measurement modes and the deviation amount from the in-focus position is known, the same first neural network can be used. If the deviation amount from the in-focus position is measured for each of the plurality of measurement modes, it is possible to deal with it by switching the deviation amount in each measurement mode during actual sample measurement.

Since the sample flow is relatively stable, it is not absolutely necessary to perform focus adjustment before the actual sample measurement. Before starting the measurement, it is better to prepare sample particles in which standard particles of uniform size are suspended and check the focus adjustment state. After the focus state is adjusted, a regular measurement sample is flown and the particle image analysis processing is executed. Also,
If the measurement sample contains a measurement target that has a relatively small variation in the value of a characteristic parameter such as size, use the characteristic parameter of the particle to check the focus adjustment state in parallel with the actual sample measurement. It is possible. As described above, since there is a thickness factor of the sample flow, in order to check the focus state, it is necessary to evaluate with an average value of a plurality of particles, but if a sufficient number of particles can be secured,
The focus adjustment state can be checked for each sample measurement, and the objective lens can be moved to the in-focus state.

[0047]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a schematic explanatory view showing the structure of a flow type particle image analysis apparatus according to an embodiment of the present invention, and FIG. 2 is a block diagram showing the structure of a focus position determination unit in the embodiment of FIG. FIG. 3 shows a change in the average value of the density dispersion (red component of the image), which is one of the particle image characteristic parameters used for measuring the deviation amount from the in-focus position, with respect to the deviation amount from the in-focus position. It is a graph. FIG. 4 shows the output values of the neural network at each output terminal of the neural network when the objective lens is moved. FIG. 5 is an explanatory diagram of a process of interpolating and calculating a deviation amount from a more correct focus position from the output value of the neural network.

In FIG. 1, 100 is a flow cell and 10 is a flow cell.
1 is an image pickup means, 102 is a particle detection means, 103 is a particle analysis means, 1 is a flash lamp, 1a is a flash lamp drive circuit, 2 is a field lens, 3 is a microscope condenser lens, 5 is a microscope objective lens, and 6 is a connection lens. Image position, 7 is a projection lens, 8 is a TV camera, 11 is a field stop, 12 is an aperture stop, 19 is a minute reflector, 15 is a semiconductor laser, 16 is a collimator lens, 17 is a cylindrical lens, 18 is a reflector, 20 Is a beam splitter, 2
1 is an aperture, 22 is a light detection circuit, 23 is a flash lamp lighting control circuit, 24 is an AD converter, 25 is an image memory,
26 is an image processing control circuit, 27 is a feature extraction circuit, 28 is an identification circuit, 29 is a central control unit, 40 is a particle number analysis unit, 5
0 is a display unit. Reference numeral 60 is a focus position determination unit, and 70 is an objective lens drive unit.

In FIG. 1, the flow-type particle image analysis apparatus comprises a flow cell 100 to which a sample liquid in which particles are suspended is supplied, an image pickup means 101 for picking up an image of the sample liquid in the flow cell 100, and the picked-up image. Particle detection means 102 for detecting particles from the above and particle analysis means 103 for analyzing the detected particles are provided.

The flow cell 100 is supplied with the sample liquid 110 and the sheath liquid surrounding the sample liquid 110.
10 forms a stream wrapped in sheath fluid.

The flow of the sample liquid 110 is a stable steady flow having a flat cross section perpendicular to the microscope optical axis 9 of the image pickup means 101, that is, a so-called sheath flow, and the center of the flow cell 100 is above the paper. Flows downward from. The flow velocity of the sample liquid 110 is controlled by the central controller 29.

The image pickup means 101 has a function as a microscope, and makes the flash lamp 1 which is a pulse light source, the flash lamp drive circuit 1a for causing the flash lamp 1 to emit light, and the pulse light flux from the flash lamp 1 parallel. The field lens 2, the microscope condenser lens 3 for focusing the parallel pulsed light flux 10 from the field lens 2 on the sample liquid flow in the flow cell 100, and the pulsed light flux irradiated on the sample liquid flow in the flow cell 100 are condensed and combined. A microscope objective lens 5 for forming an image at the image position 6, a TV camera 8 for converting a particle image at the image forming position 6 projected through the projection lens 7 into an image data signal which is an electric signal by a so-called interlace method.
A field stop 11 and an aperture stop 12 that limit the width of the pulsed light flux 10 are provided. As the TV camera 8, a CCD camera or the like with a small afterimage is used.

The particle detecting means 102 includes a semiconductor laser 15 which is a detection light source for emitting a laser beam as a detection beam, and a laser beam 14 which is a parallel laser beam from the semiconductor laser 15.
Collimator lens 16 and
Is located between the cylindrical lens 17 that focuses only one direction of the laser beam 14 from the mirror, the reflecting mirror 18 that reflects the laser beam 14 from the cylindrical lens 17, the microscope condenser lens 3 and the flow cell 100,
The laser beam 14 from the reflecting mirror 18 is fed to the sample liquid 11
It is composed of a micro-reflecting mirror 19 which leads to the vicinity of the upstream side of the image capturing area in the flow of zero.

Further, a microscope objective lens 5 for condensing scattered light of the laser light flux 14 due to particles in the sample liquid (this microscope objective lens 5 is also used as the microscope objective lens 5 for image formation of the image pickup means 101). ), A beam splitter 20 for reflecting the scattered light collected by the microscope objective lens 5, and a stop 21 for the scattered light from the beam splitter 20.
A light detection circuit 22 that receives light via the sensor and outputs an electric signal based on its intensity, and a flash lamp lighting control circuit 23 that operates the flash lamp drive circuit 1a based on the electric signal from the light detection circuit 22 are provided. ing.

The particle analysis means 103 converts the image data signal transferred from the TV camera 8 into a digital signal A.
An image memory 25 that stores data based on a signal from the D converter 24 and the AD converter 24 at a predetermined address, an image processing control circuit 26 that controls writing and reading of data in the image memory, and an image memory 25
A feature extraction circuit 27 that performs image processing based on a signal from the device and performs particle number and classification, an identification circuit 28, and a central control unit 29.
And a particle number analysis unit 40 that determines the number of particles in the sample liquid 110, and a display unit 50 that displays the number of particles and the classification as a result of image processing.

The central control unit 29 includes a photographing condition of the TV camera 8, a sample liquid flow condition of the flow cell 100, control of the image processing control circuit 26, storage of the image processing result from the identification circuit 28, and a particle number analyzing unit 40. Is transmitted / received, displayed on the display unit 50, and the like.

As shown in FIG. 2, the focus position determination unit 60 includes a parameter averaging processing unit 61, a deviation calculation unit 62 from the in-focus position, and a deviation amount conversion unit 63. The output of the deviation amount conversion unit 63 is It is sent to the objective lens driving unit 70, and the microscope objective lens 5 is moved back and forth in the optical axis direction to adjust the optical system to a focused state. The input signal of the parameter average processing unit 61 is a particle feature parameter of the feature extraction circuit 27, in particular,
A characteristic parameter suitable for obtaining the focus position information is selected. The particle classification result of the identification circuit 28 is used to average the parameters for a particular particle type.

The operation of the flow type particle image analysis apparatus configured as described above will be described below. First, the operations of the image pickup means 101 and the particle detection means 102 will be described. The semiconductor laser 15 constantly oscillates continuously, and always observes that particles in the sample liquid 110 pass through the detection region. The laser light flux 14 from the semiconductor laser 15 is
The collimator lens 16 converts the laser beam into parallel laser beams, and the cylindrical lens 17 focuses the beam 14 in only one direction. The laser beam 14 is reflected by the reflecting mirror 18 and the minute reflecting mirror 19 and is irradiated onto the flow of the sample liquid 110 in the flow cell 100. This irradiation position is set by the cylindrical lens 17 by the laser beam 14
Is a particle detection position at which is focused, and is a position near the upstream side of the image capturing region on the sample liquid flow.

When the particles to be analyzed cross the laser beam 14, the laser beam 14 is scattered. The scattered light of the laser beam is reflected by the beam splitter 20, received by the photodetector circuit 22, and converted into an electric signal based on its intensity. Further, the output of the photodetection circuit 22 is sent to the particle number analysis unit 40, and signals having a predetermined level and a predetermined pulse width or more are regarded as detected particles, and the result is transmitted to the flash lamp lighting control circuit 23. When the particles are detected, it is assumed that the particles to be image-processed have passed, and the detection signal is transmitted to the flash lamp emission control circuit 23. The predetermined delay time is determined by the distance between the particle detection position and the image capturing area, the flow velocity of the sample liquid 110, and the like.

When the flash light emission signal is transmitted to the flash lamp drive circuit 1a, the flash lamp drive circuit 1a causes the flash lamp 1 to emit light. The pulsed light emitted from the flash lamp 1 travels on the optical axis 9 of the microscope, becomes parallel light by the field lens 2, is focused by the microscope condenser lens 3, and is irradiated onto the flow of the sample solution 110 in the flow cell 100. The width of the pulsed light flux 10 is limited by the field stop 11 and the aperture stop 12. Sample liquid 11 in the flow cell 100
The pulsed light flux applied to the stream of 0 is condensed by the microscope objective lens 5 and forms an image at the image forming position 6. The image at the image forming position 6 is projected onto the image pickup surface of the TV camera 8 by the projection lens 7 and converted into an image data signal by the interlace method. In this way, the static particle image of the particles in the sample liquid 110 is captured. The image capturing condition of the TV camera 8 is preset in the central control unit 29, and the image capturing operation of the TV camera 8 is controlled by this image capturing condition.

Next, the operation of the particle analysis means 103 will be described. The image data signal captured by the TV camera 8 and converted by the interlace system is the AD converter 2
Converted to digital signal in 4 and the data based on this is
It is controlled by the image processing control circuit 26 and stored in a predetermined address of the image memory 25. The data stored in the image memory 25 is read out under the control of the image processing control circuit 26, input to the feature extraction circuit 27 and the identification circuit 28 to be subjected to image processing, and the result is stored in the central control unit 29. It The contents stored in the central control unit 29 are the particle identification feature parameters used for particle classification and the particle classification result data.

The classification and identification processing of particles is automatically performed by the pattern recognition processing which is usually performed. This image processing result, measurement conditions, and information on the number of images that have been image processed,
It is sent from the central control unit 29 to the particle number analysis unit 40. In the particle number analysis unit 40, the particle detection signal from the central control unit 29 and the photodetection circuit 22 and the image processing control circuit 2
Based on the control signal from 6, the correspondence between the detected particles and the particle classification result is investigated, and the final classification and classification result of the particle image is summarized.

Next, a method of measuring the amount of deviation from the in-focus position will be described. Here, the measurement of the amount of deviation from the in-focus position using a neural network will be described.
First, the learning process of this neural network will be described.

In the present embodiment, in order to check the in-focus state, focus position information focusing on the entire TV camera image is not used, but individual particle images, particularly particle image analyzers, are originally used for particle classification / identification. The particle image feature parameters are used. It is not necessary to use all feature parameters used for particle classification. The focus is on particle image parameters and particle image texture information, which are greatly affected by deviations from the in-focus position of the image. Specifically, the texture information about the particle image such as the density distribution of the particle image, the density contrast, and the density differential is suitable.

In order to obtain the in-focus position information, it is necessary to use particles that meet the conditions as much as possible. Here, latex standard particles made of polystyrene having a uniform size were used. Since standard particles have a large refractive index for water,
Due to the lens effect, an optical image of a particle is generally seen as a dim image in which the particle itself is bright or dim before and after the in-focus position. This change is reflected in the density information and the texture information among the above-mentioned particle characteristic parameters. It is confirmed by the change of these parameter values before and after focusing.

FIG. 3 shows the deviation amount from the in-focus position with respect to the average value of the density dispersion (red component of the image) which is one of the particle image characteristic parameters used for measuring the deviation amount from the in-focus position. Shows the changes to. The particle size used is 10 μm. In the figure, the actual measurement points are five points 1, 4, 7, 10, and 13 of the objective lens position, and the other points are the interpolation results by the cubic spline function.
Further, the position 7 is the in-focus position, and one scale on the horizontal axis is 1 μm.

The learning data is collected as follows. The objective lens is fixed to a plurality of points, and normal particle classification / identification processing is executed at that position. Individual particle classification results and particle feature parameters are collected. Of the particle classification results, only particles effective for focus adjustment are extracted, and the average value of the characteristic parameters is calculated by the parameter average processing unit 61. The average value of the plurality of particle characteristic parameters and the positional information relationship of the objective lens are learned in the neural network which constitutes the deviation calculation unit 62 from the in-focus position. The input layer of the neural network is the number of particle feature parameters to be used, the output layer is the number of settings for moving the objective lens,
When data is interpolated, the number including it and the number of intermediate layers are appropriate. The learning process of the neural network is
Often run on a separate workstation. Further, the learning process may be performed inside the central control unit 29. Learning generally uses the well-known backpropagation method. When the learning of the neural network is completed, the weighting coefficient of each part forming the network is set in the deviation calculation part 62 from the in-focus position.

Calculation of the deviation from the actual in-focus position proceeds in the same way as in the learning process. The same standard particles as those used in the learning process are prepared for focus adjustment, and particle image analysis processing is executed. Particle classification / identification processing specially made for focus adjustment may be executed. This is because it is necessary to distinguish / exclude a sample in which a plurality of standard particles are concatenated and unnecessary particles unrelated to focus adjustment in a sample in which standard particles are suspended. Of the particle classification results, only particles effective for focus adjustment are extracted, and the average value of the characteristic parameters is calculated by the parameter average processing unit 61. In this case, the result of the feature extraction circuit 27 is used as the particle feature parameter, and the result of the identification circuit 28 is used as the particle classification / identification result. The result of the parameter averaging processing unit 61 with respect to the characteristic parameter effective for focus adjustment is led to the input layer of the neural network that constitutes the in-focus position shift calculation unit 62. From the output layer of the neural network, positional information of the deviation from the in-focus position can be obtained from the output layer. There are multiple output layers of the neural network,
The position indicating the maximum output value is the deviation information from the in-focus position. As an example, in the case of FIG. 3, the position of the objective lens is learned at 13 points, and the step is 1 μm. FIG. 4 shows the output value of each neural network output end with respect to the objective lens position, using the result of actual learning using a plurality of characteristic parameters.

In FIG. 4, with respect to the objective lens position,
The positional relationship showing the maximum value of the output terminal of the neural network is linear. Moreover, the result shows only a small neural network output before and after one peak, and it can be seen that the detection sensitivity of the in-focus position is very high. This is because learning of the neural network is performed based on the average values of the plurality of particle characteristic parameters, and the actual calculation of the in-focus position also uses the average value of the characteristic parameters. The particle feature parameters used are not necessarily linear with respect to changes in the position of the objective lens, but the neural network results are linear. Therefore, at the time of learning, if the positional relationship of the objective lens is made to correspond to a strict geometrical distance, it is possible to know how much the objective lens deviates from the actual in-focus position.

Since the output result of the neural network is shown by the value of the calculation result at a plurality of output points, the process of determining the maximum value of the output from this result or converting it into the actual geometrical dimension, the objective lens The shift amount conversion unit 63 executes the process of determining the driving condition. Also, the neural network results show only discrete results,
In the case of an intermediate position, output values appear at multiple points. In this case, it is preferable to include in the shift amount conversion 63 a process of calculating a shift amount from a more accurate in-focus position using these results. Interpolation calculation of the intermediate value may be calculated as follows if it is a simple linear interpolation.

P = p (n) + N (n + 1) * (p (n + 1)-
p (n)) / (N (n) + N (n + 1)) where p (n) and p (n + 1) are the positions of the objective lens and N
(n) and N (n + 1) are output values of the neural network at that point. FIG. 5 shows an example of the neural network output value at each output end of the neural network at a general objective lens position and the calculation result of the above equation. The calculation result is 8.178, and if the in-focus position is 7, the shift amount from the in-focus position is determined to be 1.178 μm.

Since the calculation result of the shift amount conversion unit 63 directly indicates the shift from the in-focus position, the objective lens drive unit 70 is used to focus the microscope objective lens 5 by the shift amount corresponding to the shift amount. By moving back and forth in the position direction,
Focus can be adjusted. In this operation, the amount of movement to the in-focus position can be known directly by a neural network calculation, so that the focus can be adjusted by one operation. The conventional method has a big difference from having to repeatedly execute the operation of finding the optimum value.

The output of the shift amount conversion unit 63 is set so as to correspond to the number of steps of the pulse motor or the movement amount is output so as to correspond to the rotation angle in accordance with the driving method of the objective lens driving unit 70. You can also choose to do so. In the flow-type particle image analyzer, the sample flow is fairly stable once the focus adjustment is performed, so it is not necessary to perform the focus adjustment frequently. It is sufficient that the standard particles as described above are made to flow once as the sample liquid at the start of the measurement, the amount of deviation from the in-focus position is measured, and the focus adjustment processing is performed.

It should be noted that the particle feature parameter for measuring the deviation from the in-focus position was not premised on using a normalized value, but in reality, the particle feature parameter was normalized by the feature parameter at the in-focus position. Neural network learning and shift amount calculation may be performed using the calculated values. In this case, the normalization process is added to the parameter average processing unit. It may be divided by the feature amount of the in-focus position, or may be further normalized so that it becomes exactly 0 at the in-focus position. Normalization can shorten the learning convergence time of the neural network.

As the characteristic parameters to be used, one or a plurality of parameters corresponding to the defocus information are used. These parameters need not have a linear relationship with the defocus information. By using a plurality of devices, even if there are fluctuation factors for each device, these fluctuations can be absorbed. By using standard particles of uniform size as much as possible and calculating the amount of deviation from each in-focus position without performing the above-described averaging operation, the deviation of the sample flow from the average in-focus position can be calculated in particle units. Can be found at. As a result, it is possible to obtain information on the sample flow, especially information in the thickness direction.
In addition, by observing this time change, it is possible to observe the fluctuation of the sample flow in the optical axis direction. Further, when measuring the deviation from the in-focus position based on a blood sample, erythrocytes of relatively uniform size can be used instead of the standard particles.

In the case of the standard particles or red blood cells described above, a plurality of particles may be connected and image-processed as if they were one particle. In addition, there is a need to remove unwanted particles present in the sample that are not related to focusing. For standard particles, it is advisable to perform an operation of classifying and identifying one particle, a concatenated particle, and an unnecessary particle. In the blood sample, one red blood cell, a plurality of connected red blood cells, white blood cells, etc. are classified and identified, and only one red blood cell particle is taken out. After performing this operation, it is led to the input of the focus position determination unit 60 which calculates the deviation information from the in-focus position. In order to classify and identify leukocytes and connected particles from connected particles, unnecessary particles, or a blood sample, an identification circuit 28 that performs particle classification / identification processing of a portion originally used for particle image analysis may be used.

In order to classify and identify leukocytes and connected particles from the above-mentioned connected particles, unnecessary particles, or blood samples,
The identification circuit 28 may be configured by the second neural network. In practice, the first and second neural networks may be combined into one and configured to be regarded as a single neural network. When the focus adjustment is not automated, the amount of deviation from the in-focus position can be displayed using the external output means. The focus position can be manually adjusted while observing the display result. In this case, it is not necessary to monitor the actual particle image separately. When the step of learning the relationship between the amount of deviation from the in-focus position and the particle image characteristic parameter is executed for each one of the flow type particle image analysis devices, there is an advantage that the focus adjustment is optimized for each device.

Further, since it takes time to execute the neural network learning exactly one by one up to this point, the average value of the characteristic parameters at the focus position in each flow type particle image analyzer is measured, This value is regarded as the device constant of each device, and the characteristic parameter measured at the time of actual measurement is normalized by the device constant of each device, and is input to the neural network for calculating the deviation amount from the in-focus position. good.

The flow type particle image analysis device may have a plurality of measurement modes. That is, the flow rate of the sample flow may be changed, or the size of the sample flow may be changed in order to increase the measurement volume. In such a case, if the conditions of the sample flow are changed, the center position of the sample flow in each measurement mode may shift. In such a case, it is necessary to measure the amount of deviation from the in-focus position for each measurement mode. When the same standard particle is used in a plurality of measurement modes to measure the deviation amount from the in-focus position, the same first neural network can be used. For multiple measurement modes,
If the shift amount from the in-focus position is measured, it can be dealt with by switching the shift amount for each measurement mode during actual sample measurement.

[0081]

According to the present invention, the following effects can be obtained.

(1) When a neural network is used, the amount of deviation from the in-focus position can be directly measured.

(2) Since the particle characteristic parameters originally possessed by the flow-type particle image analysis apparatus are used, it is not necessary to newly prepare hardware for parameter extraction for focus adjustment.

(3) Since the used particle image characteristic parameter is calculated based on the average value of a plurality of particles, the influence of the variation due to the thickness of the sample flow can be ignored.

(4) Similarly, even if the size of the standard particles used for focus adjustment varies, the effect can be reduced by the averaging process.

(5) Since the objective lens can be controlled by using the deviation from the in-focus position, focus adjustment can be performed with one adjustment. Unlike the conventional hill climbing method, it is not necessary to repeatedly perform the objective lens adjustment operation.

(6) For focus adjustment, a specific particle type (one particle, red blood cell, etc.) is selected and processed,
The adjustment operation is stable.

(7) When there are a plurality of measurement modes with different sample flow conditions, focus adjustment information and focus adjustment can be performed for each.

(8) Information in the thickness direction of the sample flow can be reflected as necessary.

[Brief description of drawings]

FIG. 1 is a schematic explanatory view showing the configuration of a flow type particle image analysis apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a focus position determination unit in FIG.

FIG. 3 shows a change of an average value of density dispersion (red component of an image), which is one of the particle image characteristic parameters used for measuring the deviation amount from the in-focus position, with respect to the deviation amount from the in-focus position. Graph.

FIG. 4 is a three-dimensional graph showing neural network output values at each output terminal of the neural network when the objective lens is moved.

FIG. 5 is an explanatory diagram of a process of interpolating a shift amount from a more correct in-focus position from an output value of a neural network.

[Explanation of symbols]

1 ... Flash lamp, 5 ... Microscope objective lens, 6 ... Image forming position, 8 ... TV camera, 15 ... Semiconductor laser, 20 ...
Beam splitter, 22 ... Photodetection circuit, 25 ... Image memory, 26 ... Image processing control circuit, 27 ... Feature extraction circuit, 2
8 ... Identification circuit, 29 ... Central control unit, 60 ... Focus position determination unit, 61 ... Parameter averaging unit, 62 ... Deviation calculation unit from focusing position, 63 ... Deviation amount conversion unit, 70 ... Objective lens drive unit, 100 ... Flow cell, 110 ... Sample liquid.

Claims (38)

[Claims] 1. A sample liquid in which particles are suspended is surrounded by a sheath liquid, flowed in a flow cell, and the sample liquid is irradiated with light rays to image the particles in the sample liquid, and the imaged particle image is analyzed. Then, in the flow-type particle image analysis method for classifying particles, the amount of deviation from the in-focus position is measured by using one or more particle characteristic parameters used for particle analysis in order to adjust the focus of the optical system. A flow-type particle image analysis method comprising the steps of: 2. The flow type particle image analysis method according to claim 1, wherein the step of measuring the amount of deviation from the in-focus position is one or a plurality of particle characteristic parameters when the focus positions are sequentially shifted in advance. A flow-type particle image analysis method, comprising: a first neural network that learns the relationship of the shift of the focus position with respect to the value of. 3. The flow type particle image analysis method according to claim 1 or 2, wherein the first neural network is trained based on an average value of a plurality of particle characteristic parameters. Particle image analysis method. 4. The flow type particle image analysis method according to claim 3, wherein when learning the relationship between the amount of deviation from the in-focus position and the particle image feature parameter, the particle feature at a position other than the measurement point is measured. A flow-type particle image analysis method comprising a step of adding an interpolated and extrapolated value based on data obtained by measuring a parameter. 5. The flow-type particle image analysis method according to claim 2, wherein the first neural network learned about the particle feature parameter and the amount of deviation from the in-focus position is used, and the particles in the sample are sampled. A flow-type particle image analysis method comprising the step of measuring the amount of deviation from the current in-focus position using the same particle feature parameter used in one or more of the above learnings of the image. 6. The flow type particle image analysis method according to claim 5, wherein a plurality of particles are used as a value of one or a plurality of particle characteristic parameters used for measuring the amount of deviation from the current focus position. A flow-type particle image analysis method comprising a step of measuring an amount of deviation from a focused position using an average value of images. 7. The flow type particle image analysis method according to claim 2, wherein the output of the first neural network for measuring the amount of deviation from the in-focus position is an actual geometrical position from the actual in-focus position. A flow-type particle image analysis method characterized in that the flow-type particle image analysis method is configured so as to match the shift amount. 8. The flow type particle image analysis method according to claim 5, wherein when the output of the first neural network is not sufficient for discrete output, data is interpolated from output values of a plurality of points to obtain a focus position. A flow-type particle image analysis method comprising the step of more accurately calculating the amount of deviation of the particle size. 9. The flow type particle image analysis method according to claim 5 or 8, wherein standard particles of uniform size are used,
A flow-type particle image analysis method, characterized in that the amount of deviation from the in-focus position for each individual particle is measured, and as a result, information regarding the sample flow is obtained. 10. The flow-type particle image analysis method according to claim 9, wherein among the sample flow information, information about the thickness of the sample flow is obtained. 11. The flow-type particle image analysis method according to claim 10, wherein, of the sample flow information, variation information regarding the thickness of the sample flow is obtained. 12. The flow type particle image analysis method according to claim 1, wherein in order to obtain information on a deviation from a focused position, the particle image feature amount is texture information or an image of the particle image. A flow type particle image analysis method characterized by using a characteristic parameter for extracting a high frequency component of. 13. The flow type particle image analysis method according to claim 12, wherein the texture information is density dispersion,
A flow-type particle image analysis method characterized by using information of density contrast and density differential. 14. The flow-type particle image analysis method according to claim 1, wherein a sample in which standard particles of uniform size are suspended is used as a particle for detecting the amount of deviation from the in-focus position. A flow-type particle image analysis method characterized by being used. 15. The flow-type particle image analysis method according to claim 14, wherein a plurality of particles in a standard particle sample are connected to each other, an impurity particle not related to focus adjustment,
And a flow type particle image analysis method characterized by having a particle classification / identification process for classifying and discriminating one standard particle and removing the above-mentioned connected particles and impurity particles. 16. The flow type particle image analysis method according to claim 15, wherein a plurality of particles in a standard particle sample are connected to each other, an impurity particle not related to focus adjustment,
And a flow type particle image analysis method characterized by having a second neural network as a particle classification process for classifying and identifying one standard particle and removing the concatenated particles and the impurity particles. 17. A flow-type particle image analysis method according to claim 1, wherein a specific particle in an actual measurement sample is identified as a particle for detecting a deviation from a focused position. With a neural network of
A flow-type particle image analysis method characterized by detecting a deviation from a focus position using a first neural network based on particle characteristic parameters of the specific particle. 18. The flow-type particle image analysis method according to claim 16, wherein a second neural network for particle classification and a first neural network for detecting a deviation from a focus position are combined. Then 1
A flow-type particle image analysis method characterized by being configured as two neural networks. 19. The flow-type particle image analysis method according to claim 1, further comprising the step of adjusting the optical system to the in-focus position based on the obtained deviation information from the in-focus position. A flow type particle image analysis method characterized by the above. 20. The flow-type particle image analysis method according to claim 19, wherein 1 based on the deviation information from the in-focus position.
A flow-type particle image analysis method, which comprises a step of adjusting an optical system to a focused position by a single operation. 21. The flow type particle image analysis method according to claim 20, wherein the amount of deviation from the in-focus position is displayed using an external output means based on the information on the deviation from the in-focus position. Flow type particle image analysis method. 22. The flow-type particle image analysis method according to claim 2, wherein the step of learning the relationship between the amount of deviation from the in-focus position and the particle feature parameter is performed for each flow-type particle image analysis apparatus. A flow-type particle image analysis method characterized by performing learning of a first neural network. 23. The flow-type particle image analysis method according to claim 2, wherein a normalization constant for each flow-type particle image analysis device, that is, a particle image used to obtain deviation information from a focus position. A flow-type particle image analysis method, characterized in that a measured particle feature parameter is normalized by an average value of the feature parameters at a focus position. 24. The flow type particle image analysis method according to claim 2, further comprising a step of providing a plurality of measurement modes and measuring a deviation amount from a focus position for each measurement mode. Flow type particle image analysis method. 25. The flow type particle image analysis method according to claim 24, wherein the amount of deviation from the in-focus position is measured for each of a plurality of measurement modes, and at the time of actual sample measurement,
A flow-type particle image analysis method, characterized in that focus adjustment conditions are switched based on the amount of deviation from the in-focus position for each measurement mode. 26. A sample liquid in which particles are suspended is surrounded by a cleansing liquid, flowed in a flow cell, and the sample liquid is irradiated with light rays to image the particles in the sample liquid, and the imaged particle image is analyzed. Then, in a flow-type particle image analyzer that classifies particles, the amount of deviation from the in-focus position is measured using one or more particle feature parameters used for particle analysis, in order to adjust the focus of the optical system. A flow-type particle image analysis device having means for 27. The flow type particle image analyzer according to claim 26, wherein one or a plurality of particle characteristic parameters when the focus positions are sequentially shifted in advance as means for measuring the amount of deviation from the focus position. A first neural network that learns the relationship of the shift of the focus position with respect to the value of, the means for calculating the average value of each of one or more particle characteristic parameters, and the first value based on the average value of the characteristic parameters. A flow-type particle image analysis device having means for learning the neural network of 1. 28. The flow-type particle image analysis device according to claim 26 or 27, wherein the first neural network learned about the particle feature parameter and the amount of deviation from the focus position is used to measure 1 of particle images in the sample. A flow-type particle image analysis device having means for measuring the amount of deviation from the current in-focus position using the same particle feature parameter used in the learning of a plurality of particles. 29. The flow type particle image analyzer according to claim 28, wherein a plurality of particles are used as the value of one or a plurality of particle characteristic parameters used for measuring the amount of deviation from the current in-focus position. A flow-type particle image analysis device comprising means for calculating an average value of an image and means for measuring an amount of deviation from a focus position based on a particle characteristic parameter based on the average value. 30. In the flow-type particle image analyzer according to claim 28, if the output of the first neural network is not sufficient for the discrete output, data are interpolated from the output values of a plurality of points to obtain the in-focus position. A flow-type particle image analysis apparatus having means for more accurately calculating the deviation amount of 31. The flow-type particle image analyzer according to claim 28, further comprising means for measuring the amount of deviation from the in-focus position with respect to each particle by using standard particles of uniform size. A flow-type particle image analysis device, characterized in that it has means for obtaining information about the sample flow, in particular, the thickness of the sample flow or the thickness variation information of the sample flow. 32. The flow type particle image analyzer according to claim 26, wherein a plurality of particles in a standard particle sample are connected to each other, impurity particles not related to focus adjustment, and A flow-type particle image analysis device comprising a unit for classifying and identifying one standard particle, and a particle classifying / identifying processing unit for removing the above-mentioned connected particles and impurity particles. 33. A particle classification / identification unit for identifying a specific particle in a measurement sample as a particle for detecting a deviation from a focus position in the flow type particle image analysis apparatus according to claim 26. And a means for detecting the deviation from the in-focus position by using the first neural network based on the particle characteristic parameter of the specific particle. 34. The flow type particle image analyzer according to any one of claims 26 to 33, further comprising means for adjusting the optical system to the in-focus position based on the obtained deviation information from the in-focus position. A flow type particle image analysis device characterized by the above. 35. The flow type particle image analyzer according to claim 34, wherein 1 is set based on the deviation information from the in-focus position.
A flow-type particle image analysis device having means for adjusting an optical system to a focused position by one operation. 36. The flow type particle image analyzing and analyzing apparatus according to claim 35, further comprising means for displaying a deviation amount from the in-focus position by using an external output means based on the deviation information from the in-focus position. A flow type particle image analysis device characterized by the above. 37. A flow-type particle image analyzer according to claim 26, particularly a flow-type particle image analyzer having a plurality of measurement modes, for measuring a deviation amount from a focus position for each measurement mode. A flow-type particle image analysis device comprising: 38. A flow type particle image analyzer according to claim 37, wherein means for measuring the amount of deviation from the in-focus position for each of a plurality of measurement modes, and for each measurement mode at the time of actual sample measurement. 2. A flow-type particle image analysis apparatus having means for switching focus adjustment conditions based on the amount of deviation from the in-focus position.