KR101995764B1 - Cytometry apparatus and method - Google Patents

Abstract

A method and apparatus for distinguishing a blood cell, comprising: a division unit for dividing an image of a blood cell image into a plurality of regions and selecting each blood cell image; extracting first classification feature data from the divided hemocyte images by a residual network method A second extracting unit for extracting second classification feature data in a random forest system in divided blood cells and nucleus and cytoplasmic image in the divided unit, and a second extracting unit for extracting second candidate feature data based on the first and second feature data, And extracting the classification feature data from the captured blood cell image in the residual network format and the random forest format, and extracting the extracted classification feature data from the captured blood cell image, It is possible to accurately distinguish the type of blood cells.

Description

[0001] CYTOMETRY APPARATUS AND METHOD [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a blood cell differentiation apparatus and method, and more particularly, to a blood cell differentiation apparatus and method for distinguishing blood cell types using a blood cell image of a blood sample.

An image analyzer is a device that identifies the type of blood cells by imaging and analyzing blood cells contained in a blood sample smeared with blood.

The blood cells include white blood cells (WBCs), red blood cells (RBCs), and platelets.

Among these, white blood cells can be divided into granulocytes, monocytes, and lymphocytes (LY), including neutrophils (NE), eosinophils (EO), and basophils have.

Actually, leukocytes may exist in various forms depending on the patient's condition, and the hematocrit counting the number of each type is very important for diagnosing the condition of the patient, diagnosis and tracking of the disease.

Therefore, various methods are being developed to improve the accuracy of the image analyzer's discrimination results.

The applicant of the present invention has filed a patent application for a hemocyte imaging apparatus, a hemocyte differentiation system, and a hemocyte image processing method disclosed in the following patent documents 1 to 3.

On the other hand, unlike a cell analyzer having a hemocyte analyzing performance limited to two to five types, recently, six types or more and up to 14 hemocytes are analyzed.

Table 1 is a table showing the classification system of wall blood differentiation, and FIG. 1 is an image of each white blood cell shown in Table 1.

Mnemonic WBC type One NE Neutrophil (seg & band) 2 ME Metamyelocyte 3 MY Myelocyte 4 PR Promyelocyte 5 LY Lymphocyte 6 LA Lymphocyte, Abnormal 7 LR Lymphocyte, Abnormal 8 MO Monocyte 9 EO Eosinophil 10 BA Basophil 11 BL Blast 12 PC Plasma cell 13 NR NRBC (Erythroblast) 14 AR Cell not reportable

As shown in Table 1 and FIG. 1, it is possible to subdivide into 14 types, including the above-mentioned five kinds of normal blood cells, 8 kinds of abnormal blood cells and cases in which classification is impossible.

Korean Patent Registration No. 10-1741766 (issued on May 31, 2017) Korean Patent Registration No. 10-1741765 (issued on May 31, 2017) Korean Patent Registration No. 10-1741764 (issued on May 31, 2017)

Therefore, it is required to develop a technique that can accurately discriminate 14 kinds of white blood cells including 6 types, especially 5 kinds of normal white blood cells and 9 kinds of abnormal white blood cells, using the blood cell image captured by the image analyzer.

SUMMARY OF THE INVENTION An object of the present invention is to provide a blood cell differentiation apparatus and method capable of precisely discriminating blood cells using the captured blood cell image.

Another object of the present invention is to provide a blood cell differentiation apparatus and method for distinguishing blood cells using white blood cells, leukocytes, nuclei, and cytoplasmic images classified in the captured blood cell image.

According to an aspect of the present invention, there is provided a blood cell differentiation apparatus comprising: a division unit for dividing an image of a blood cell image into a plurality of regions and selecting each blood cell image; A second extracting unit for extracting second classification feature data in a random forest manner from the blood cells and nucleus and cytoplasmic images divided by the division unit, And a determination unit for extracting characteristics of the blood cell candidate based on the second classification feature data, determining the type of the blood cell, and outputting the result.

According to another aspect of the present invention, there is provided a blood cell differentiation method comprising the steps of: (a) dividing an image of a blood cell image picked up using a divided part and selecting each blood cell image; (b) Extracting the first classification feature data from the divided blood cell images using the residual network method, (c) extracting the second classification feature data from the divided blood cells and the nuclear and cytoplasmic images using a random forest method using the second extraction unit, Extracting a feature of the blood cell candidate based on the first and second classification feature data using the determination unit, and determining and outputting the type of the blood cell candidate.

As described above, according to the blood cell differentiation apparatus and method according to the present invention, the classification feature data is extracted from the captured blood cell image by the residual network method and the random forest method, and the blood cell image is classified So that the kind of blood cells can be accurately discriminated.

In addition, according to the present invention, each classification feature data extracted in the residual network method and the random forest method are independently connected and input to the pulley connected layer module, and the candidate feature is extracted based on the random forest coefficient value, By discriminating the kind, it is possible to obtain an effect that the accuracy of the discrimination result can be improved.

1 is a view showing an image of each white blood cell,
2 and 3 are views for explaining the CNN scheme and the residual network scheme, respectively,
4 is a configuration diagram of a blood cell differentiating apparatus according to a preferred embodiment of the present invention,
FIG. 5 is a detailed block diagram of the blood cell differentiating device shown in FIG. 4,
FIG. 6 is a process diagram for explaining a blood cell differentiation method of the blood cell differentiation apparatus according to a preferred embodiment of the present invention.

The present invention distinguishes blood cells using classification feature data extracted by a residual network method and classification feature data extracted by a random forest method for an image of a blood cell image, Can be improved.

The residual network scheme may be configured by adding a shortcut connection that is directly connected to the output of a convolutional neural network (CNN).

2 and 3 are views for explaining the CNN scheme and the residual network scheme, respectively.

As shown in FIG. 2, the CNN receives an input x, outputs an output H (x) via two weighted layers, and obtains optimal H (x) through learning.

Here, if two weighted layers are learned to obtain the difference H (x) -x between the output and the input, then H (x) = F (x) + x.

As shown in Fig. 3, the short cut is a structure in which two weighted layers are directly connected without parameters, so that there is no difference except that addition is added from the viewpoint of calculation amount.

Therefore, the residual network learns to obtain H (x) -x, and if it is optimal, F (x) = 0, the learning direction is predetermined and the pre-conditioning function do.

Learning from the direction in which F (x) becomes almost zero enables learning of a small movement of F (x), that is, residual, as it becomes possible to detect a small fluctuation of input. (residual learning).

Also, since the same x as the input is directly connected to the output, there is no influence on the number of parameters, and there is no increase in computation through a short-cut connection except for the addition, and input and output are connected Therefore, the effect of simplifying the forward path and the backward path can be obtained.

Accordingly, the residual network can be easily optimized for the deep network, and the accuracy can be improved due to the increased depth.

On the other hand, residual network has limitations in debugging due to feature extraction such as color or shape.

Accordingly, in addition to the residual network method, the present invention can improve the accuracy of the blood cell discrimination result by discriminating the blood cells using the random forest method in which the features are directly designed and inputted and the classification feature data is extracted according to the inputted characteristics .

In the random forest method, n is selected randomly from the main training data set, n pieces are selected, and d pieces of data characteristic values are selected in the selected n pieces of data samples to allow decision trees to be learned The process is repeated k times, and it is predicted using k decision trees. The average of the predicted results or the most predicted results are selected as the final predicted values.

In this random forest, the decision trees generated according to given learning data are very different, so it is difficult to generalize and use them, and the disadvantages of decision tree learning with large performance and fluctuation of learning results can be solved.

4 and 5, a blood cell differentiation apparatus and method according to a preferred embodiment of the present invention will be described in detail.

In this embodiment, a method of discriminating 14 kinds of leukocytes is described, but it should be noted that the present invention is not necessarily limited to this, and it is also applicable to a method of discriminating red blood cells from platelets.

FIG. 4 is a configuration diagram of a blood cell differentiation apparatus according to a preferred embodiment of the present invention, and FIG. 5 is a detailed block diagram of the blood cell differentiation apparatus shown in FIG.

4 and 5, the blood cell differentiation apparatus 10 according to the preferred embodiment of the present invention includes a division unit 20 for dividing a captured blood cell image into a plurality of regions, a division unit 20 for dividing the divided blood cell image into a plurality of regions, A first extracting unit 30 for extracting first classification feature data from a leukocyte image obtained by the residual network method, a second classification feature data extracting unit 30 for extracting second classification feature data from a white blood cell, And a determination unit (50) for extracting characteristics of the blood cell candidate based on the first and second classification characteristic data, determining the type of the blood cell, and outputting the result.

In addition, the blood cell differentiation apparatus 10 according to the preferred embodiment of the present invention may further include an image interface 60 for storing and managing white blood cells, nuclear and cytoplasmic images divided by the division unit 20.

The division unit 20 divides a blood cell image picked up at a predetermined magnification, for example, x100, into a plurality of, for example, eight regions in a blood cell image capturing apparatus (not shown), removes a hemocyte image edge portion from the divided regions, You can select images.

As described above, the blood cell image selected by the division unit 20 can be stored and managed in the image interface 60.

The first extractor 30 extracts the first classification feature data (ResNet Feature) from the leukocyte image transmitted from the image interface 60 by the residual network method.

The first extraction unit 30 is provided as a convolution layer of a residual type, and the convolution layer functions to extract correct classification feature data through a filter.

The second extracting unit 40 extracts second RF feature data from the white blood cells, nucleus, and cytoplasmic image transmitted from the image interface 60 in a random forest fashion.

The judging unit 50 includes a fully concealed layer module 51 for receiving both the first and second classification characteristic data and outputting a random forest and a residual network characteristic (RF + ResNet Feature) A classifier 52 for classifying the blood cell image into a random forest type using the feature data output from the pulley connected layer module 51 and outputting a random forest coefficient (RF Coefficient) A first determination unit 53 for extracting the characteristics of the blood cell candidate using the forest coefficient, and a second determination unit 54 for determining the type of the blood cell using the feature of the extracted blood cell candidate and outputting the result .

The pulley connected layer module 51 is a module applied to the residual network system and is composed of an input layer, a hidden layer and an output layer, and when a plurality of hidden layers are applied, It is called a deep neural network.

That is, the artificial neural network model is classified into a 'single layer neural network' and a 'multi-layer neural network' according to the number of layers, and the multi-layer neural network may include an input layer, a hidden layer and an output layer.

The hidden layer is located between the input layer and the output layer, receives the signal from the input layer, extracts the characteristics, and outputs the output layer. The input layer is a layer that receives external data. The number of neurons in the input layer is equal to the number of input variables. And the output layer receives a signal from the hidden layer and outputs it to the outside.

The input signal between the neurons is multiplied by the respective connection strengths between 0 and 1, and then summed. When the sum is larger than the threshold value of the neuron, the neuron is activated and implemented as an output value through the activation function.

The pulley connected layer module 51 configured as described above functions to generate a final neural network using the feature data of the multi-layer channel.

For example, the first classification feature data extracted by the random forest method and the second classification feature data extracted by the residual network method may be independently connected to the pulley connected layer module 51, respectively.

Therefore, the pulley connected layer module 51 normalizes the values of the independently connected first and second classification feature data within a range of 0.f to 1.f, and outputs a residual network and a random forest vector Merging, and finally generating a neural network using a merging vector.

The classifier 52 is applied to the random forest method and functions to classify the blood cell image according to the feature data output from the pulley connected layer module 51 using the soft-max function.

The classifier 52 may classify the blood cell image and output a random forest coefficient (RF Coefficient).

Accordingly, the first determination unit 53 can extract the feature of the blood cell candidate on the basis of the random forest coefficient value output from the classifier 52.

As described above, the present invention can improve the accuracy of the discrimination result by sorting the blood cells by classifying the captured blood cell images using the classification feature data extracted by the random forest method and the residual network method.

Next, referring to FIG. 6, a blood cell differentiation method of a blood cell differentiation apparatus according to a preferred embodiment of the present invention will be described in detail.

FIG. 6 is a process diagram for explaining a blood cell differentiation method of the blood cell differentiation apparatus according to a preferred embodiment of the present invention in stages.

In step S10 of FIG. 6, the blood cell differentiating device 10 receives a blood cell image captured from the blood cell image capturing device.

In step S12, the segmenting unit 20 divides the blood cell image into a plurality of regions, removes the edge of the blood cell image from the divided regions, and selects each of the blood cell images. The image interface 60 is connected to the segmenting unit 20 And stores and manages the white blood cells and nuclear and cytoplasmic images.

In step S14, the first extraction unit 30 extracts first classification feature data from the leukocyte image transmitted from the image interface 60 in a residual network manner, and the second extraction unit 40 extracts the first classification feature data from the image interface 60 And extracts second classification feature data from the white blood cells, nucleus, and cytoplasm images transmitted in a random forest system.

In step S16, the pulley connected layer module 51 generates a final neural network using the characteristic data of the multi-layer channel.

That is, the pulley connected layer module 51 receives the first classification feature data extracted by the random forest method and the second classification feature data extracted by the residual network method independently and receives them, 2 classification feature data is normalized to within a range of 0.f to 1.f, the residual network and the random forest vector are merged, and the merged vector is used to finalize Generate a neural network.

In step S18, the classifier 52 classifies the blood cell images into a random forest type using the random forest and the residual network characteristic (RF + ResNet Feature) data output from the pulley connected layer module 51 and outputs a random forest coefficient RF Coefficient.

At this time, the classifier 52 classifies the blood cell image according to the feature data output from the pulley connected layer module 51 using the soft-max function.

In step S20, the first determination unit 53 extracts the feature of the blood cell candidate based on the random forest coefficient value output from the classifier 52. [

In step S22, the second determination unit 54 determines the type of the blood cell using the feature of the extracted blood cell candidate, and outputs it.

Through the above process, the present invention extracts classification feature data from the captured blood cell image using the residual network method and the random forest method, classifies the blood cell image using the extracted classification feature data, It can be distinguished.

Although the invention made by the present inventors has been described concretely with reference to the above embodiments, the present invention is not limited to the above embodiments, and it goes without saying that various changes can be made without departing from the gist of the present invention.

The present invention relates to a blood cell differentiation apparatus and method for extracting classification feature data from a captured blood cell image in a residual network system and a random forest system and classifying blood cell images using the extracted classification feature data, .

10: hemocyte differentiation device
20: minute installment
30:
40:
50:
51: Pulley connected layer module
52: Classifier
53:
54:
60: Image interface

Claims (10)

A division unit for dividing the captured blood cell image into a plurality of regions and selecting each blood cell image,
A first extracting unit for extracting first classification feature data from the divided hemocyte images in the partitioning unit in a residual network manner,
A second extracting unit for extracting second classification feature data in a random forest manner from the divided blood cells, nuclear and cytoplasmic images in the divided unit,
And a determination unit for extracting characteristics of the blood cell candidate based on the first and second classification feature data, and determining and outputting the type of the blood cell,
A pulley connected layer module for receiving both the first and second classification feature data and outputting a random forest and residual network feature data of a blood cell image,
A classifier for classifying blood cell images into random forest patterns using feature data output from the pulley connected layer module and outputting a random forest coefficient,
A first determination unit for extracting characteristics of the blood cell candidate using the random forest coefficients output from the classifier;
And a second determination unit for determining the type of the blood cell using the extracted characteristics of the blood cell candidate and outputting the determined result. The method according to claim 1,
Further comprising an image interface for storing and managing the blood cells, nuclear and cytoplasmic images divided by the division unit. delete The method according to claim 1,
Wherein the first sorting feature data and the second sorting feature data are independently connected to the pulley connected layer module,
Wherein the pulley connected layer module normalizes the values of the first and second classification characteristic data independently connected within a predetermined setting range, merges a residual network and a random forest vector, Wherein the blood cell discrimination device generates the blood cell discrimination signal. 5. The method of claim 4,
Wherein the classifier classifies blood cell images according to characteristic data output from the pulley connected layer module using a soft max function in a random forest method. (a) dividing an image of a blood cell image picked up by using a divided part and selecting each blood cell image,
(b) extracting first classification feature data from the divided hemocyte images using a residual network method using the first extracting unit,
(c) extracting second classification feature data in a random forest manner in divided blood cells, nucleus and cytoplasmic images using the second extraction unit, and
(d) extracting features of the blood cell candidate based on the first and second classification feature data using the determination unit, and determining and outputting the type of the blood cell; and (d)
(d1) receiving the first and second classification feature data using the pulley connected layer module to output random forest and residual network feature data of the blood cell image,
(d2) classifying the blood cell image into a random forest type based on the feature data output from the pulley connected layer module using a classifier and outputting a random forest coefficient;
(d3) extracting characteristics of the blood cell candidate based on the random forest coefficients output from the classifier using the first determination unit; and
(d4) determining the type of the blood cell based on the characteristics of the extracted blood cell candidate using the second determination unit, and outputting the result. The method according to claim 6,
(e) storing and managing divided blood cells, nuclear and cytoplasmic images using an image interface, and providing images stored in the first and second extracting units. delete 8. The method of claim 7,
Wherein the first sorting feature data and the second sorting feature data are independently connected to the pulley connected layer module,
The pulley connected layer module normalizes the values of the first and second classification characteristic data independently connected within a predetermined set range, merges the residual network and the random forest vector, and uses the merging vector to calculate the final neural network Wherein the blood vessel discrimination method comprises the steps of: 10. The method of claim 9,
Wherein the classifier classifies the blood cell image according to the feature data output from the pulley connected layer module using the soft max function in the random forest method in the step (d2).

Images