KR20240022892A - Apparatus and method for evaluating red blood cells based on deep-learning - Google Patents

Abstract

A red blood cell evaluation method according to an embodiment of the present disclosure is a method in which at least part of each step is performed by a processor, and at least a part of each step is performed by a processor, comprising: receiving a phase image of red blood cells; A step of acquiring a marker image of red blood cells and a red blood cell classification image by inputting the image into an inference model based on a convolutional layer, and applying a marker-based watershed algorithm to the binary mask image and marker image of red blood cells generated based on the red blood cell classification image. It may include a step of generating segmentation information that distinguishes regions of red blood cells.

Description

Device and method for evaluating red blood cells based on deep learning {APPARATUS AND METHOD FOR EVALUATING RED BLOOD CELLS BASED ON DEEP-LEARNING}

The present disclosure relates to an apparatus and method for identifying red blood cells based on deep learning. More specifically, an apparatus and method for segmenting the exact area of red blood cells and determining the type of red blood cells based on deep learning. It's about.

Determining the type of blood cell is important in diagnosing various diseases. For example, prior art 1, which seeks to diagnose a disease based on the number of white blood cells, classifies blood cells into white blood cells, red blood cells, and platelets, and discloses a technology for discriminating white blood cells into subclasses.

However, prior art 1 does not disclose a technology for determining the subclass of red blood cells, but it is important to determine the status of red blood cells in order to determine the possibility of using blood for blood donation, etc.

In addition, the learning model structure of prior art 1 is for determining the type of white blood cell, and does not present results as to whether the structure is suitable for distinguishing areas of red blood cells and determining the type of red blood cells.

Conventionally, one of the methods for region division is a watershed-based region division method. Watershed is an algorithm that uses brightness information to expand homogeneous areas and uses boundary information to classify areas. It has better performance than algorithms that only use individual information, but it is not robust to noise or subtle changes in brightness, so it has different characteristics. A marker-based watershed method was introduced to group homogeneous areas into one homogeneous area.

However, the marker-based watershed method inevitably shows results dependent on the characteristics of the marker.

Prior Art 1: Korean Patent Publication No. 10-1995763 (registered on June 27, 2019)

The conventional marker-based watershed method has a problem in that it cannot distinguish red blood cells from different red blood cells, especially when multiple red blood cells overlap.

An embodiment of the present disclosure provides a method and device that can accurately distinguish (segment) the region of each red blood cell, including overlapping red blood cells, and determine subtypes of the divided red blood cells.

An embodiment of the present disclosure provides a method and device for accurately segmenting the region of each red blood cell, including overlapping red blood cells, based on a marker generated based on deep learning.

An embodiment of the present disclosure provides a method and device for estimating the storage period or usability of blood based on the determined ratio of each type of red blood cells.

The red blood cell evaluation method according to an embodiment of the present disclosure is a method in which at least part of each step is performed by a processor, including receiving a phase image of the red blood cell, inputting the phase image into an inference model based on a convolutional layer to determine the marker of the red blood cell. A step of acquiring an image and a red blood cell classification image, and applying a marker-based watershed algorithm to the binary mask image and marker image of the red blood cell generated based on the red blood cell classification image to generate segmentation information dividing the red blood cell area. May include steps.

The red blood cell evaluation device according to an embodiment of the present disclosure includes a processor and a memory electrically connected to the processor and storing at least one code to be executed by the processor, and the memory is configured to cause the red blood cell to be evaluated when the memory is executed through the processor. Input the phase image into an inference model based on a convolutional layer to obtain a marker image of red blood cells and a red blood cell classification image, and apply a marker-based watershed algorithm to the binary mask image and marker image of red blood cells generated based on the red blood cell classification image. By applying , you can store the code that causes segmentation information to be generated by distinguishing the areas of red blood cells.

The red blood cell evaluation device and method according to an embodiment of the present disclosure can accurately distinguish regions even in the case of overlapping red blood cells by applying a marker generated based on deep learning to the watershed algorithm.

The red blood cell evaluation device and method according to an embodiment of the present disclosure can prevent accidents caused by unhealthy blood by estimating the storage period or usability of blood based on the ratio of each type of red blood cell.

1 is a diagram showing a portion of the configuration of a digital hologram acquisition device for acquiring a hologram used in an embodiment of the present disclosure.
FIG. 2 is a diagram illustrating an environment for performing a red blood cell evaluation method or driving a red blood cell evaluation device according to an embodiment of the present disclosure.
Figure 3 is a block diagram showing the configuration of a red blood cell evaluation device according to an embodiment of the present disclosure.
Figure 4 is a block diagram showing the configuration of a learning model training device for red blood cell identification according to an embodiment of the present disclosure.
Figure 5 is a flowchart for explaining a red blood cell evaluation method according to an embodiment of the present disclosure.
Figure 6 is a diagram illustrating an inference model for dividing regions and determining types of red blood cells according to an embodiment of the present disclosure.
Figure 7 is a diagram illustrating the structure of an inference model for classifying regions and determining types of red blood cells according to an embodiment of the present disclosure.
Figure 8 is a diagram showing the structure of a discrimination model used when learning an inference model that classifies regions and determines types of red blood cells according to an embodiment of the present disclosure.
FIG. 9 is a diagram illustrating the structure of a residual block used in an inference model for distinguishing regions and determining types of red blood cells according to an embodiment of the present disclosure.
FIG. 10 is a diagram illustrating a method and results of accurately distinguishing areas of overlapping red blood cells based on markers generated by a deep learning learning model according to an embodiment of the present disclosure.
Figure 11 is a diagram showing the results of a red blood cell region classification and type discrimination experiment of a learning model according to an embodiment of the present disclosure.
Figure 12 is a diagram showing the accuracy of the results of the red blood cell region classification and type discrimination experiment of the learning model according to an embodiment of the present disclosure.
Figure 13 is a diagram showing the results of an experiment for categorizing red blood cells and determining their type using a learning model for blood by storage period according to an embodiment of the present disclosure.
Figure 14 is a diagram showing the results of an experiment on changes in red blood cell types of a learning model for blood by storage period according to an embodiment of the present disclosure.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numbers regardless of reference numerals, and duplicate descriptions thereof will be omitted. The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves. Additionally, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted. In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Referring to FIG. 1, the configuration of a digital holographic microscope (quantitative phase digital holographic microscopy: QP-DHM), which may be part of the digital hologram acquisition device 200 for acquiring a hologram used in an embodiment of the present disclosure, is explained. do.

1 is explained with reference to an off-axis digital holographic microscope based on a Mach-Zender interferometer, but embodiments of the present invention include holograms generated from an off-axis digital holographic microscope as well as holograms generated from an off-axis digital holographic microscope. It can also be implemented in a similar way based on a phase image reconstructed from a hologram generated from an in-line digital holographic microscope.

A digital holographic microscope can acquire phase information of an object such as a cell. Digital holographic microscopes can operate based on Mach-Zehnder interferometry. The coherent laser source of a digital holographic microscope can split the light source into an object wave and a reference wave using a beam splitter with a small tilt angle between the two beams. Object waves can illuminate a specimen and create an object wave front. A microscope objective (MO) magnifies the object wavefront, and the object and reference wavefronts can be combined by a beam collector at the exit of the interferometer to create a hologram. The generated hologram can be transmitted to a computing device that controls the digital holographic microscope, and the digital holographic microscope or the computing device that controls the digital holographic microscope can perform filtering processing in the frequency domain of the hologram to remove unwanted signals. You can.

A digital holographic microscope or a computing device that controls a digital holographic microscope can generate a phase image by illuminating the filtered hologram using a replica of the reference wave and reconstructing the hologram by the Fresnel approximation. .

When a plane wave propagates to an object such as a cell and the plane wave passes through the cell, output waves with different phases are output depending on the thickness or refractive index of the object such as a cell, and the output wave compares the phase of the input plane wave. An image that has the amount of phase change as a quantitative pixel value can be called a phase image.

In this specification, an image does not necessarily mean a form composed of pixel intensity values. In cases where quantitative values are composed in the form of a matrix, any form can be imaged and displayed, so it is referred to as an image including this form. .

Since digital holographic imaging and reconstruction techniques can be clearly understood by those skilled in the art, further detailed descriptions are omitted.

Referring to FIG. 2 , the operating environment of the computing device 100 for segmenting and identifying red blood cells in a phase image according to an embodiment of the present disclosure will be described. Segmentation refers to the separation of an individual red blood cell's area from the background area or other red blood cells.

An environment for driving the computing device 100a according to an embodiment of the present disclosure may include a digital hologram acquisition device 200 and a learning model training device 300.

The computing device 100 generates a phase image by reconstructing a hologram taken of blood containing red blood cells provided from the digital hologram acquisition device 200 or another device, or collects blood containing red blood cells from the digital hologram acquisition device 200. The phase image created by reconstructing the captured hologram can be provided through a network or storage medium.

The computing device 100 can input the phase image into a learning model based on deep learning (deel-learning) to distinguish regions of red blood cells and determine the type of red blood cells.

The digital hologram acquisition device 200 may be a digital holographic microscope or a device that controls a digital holographic microscope. In another embodiment, the digital hologram acquisition device 200 may be a single device combining a digital holographic microscope and a device for controlling the digital holographic microscope, or may be a system composed of a plurality of devices. The device that controls the digital holographic microscope is a computing device that includes a processor, memory, non-transitory storage media, etc., and may be a personal computer (PC), laptop, tablet computer, or server device, and is connected to the digital holographic microscope to create a digital holographic microscope. Microscope imaging and hologram creation can be controlled. Additionally, digital holograms can be reconstructed or phase images can be generated.

The learning model training device 300 can train a deep learning-based learning model that distinguishes areas of red blood cells and determines the type of red blood cells in the computing device 100, and the learning model is a generative adversarial network. : Can be trained based on GAN). The learning model includes an inference model (generator) with the structure of FIGS. 5 and 7, and the inference model is a discrimination model with the structure of FIG. 6 trained to determine whether the inferred result is a GT (ground truth) image or a generated image. It can be trained based on the discriminator.

The learning model training device 300 may distribute or transmit an inference model among the trained learning models to the computing device 100.

The learning model may be implemented in hardware, software, or a combination of hardware and software. If part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in memory.

Referring to FIG. 3 , the configuration of a computing device 100 for segmenting and identifying red blood cells in a phase image according to an embodiment of the present disclosure will be described.

The computing device 100 drives a communication unit 110 and a processor 140 that receive a learning model based on deep learning or a phase image, or transmit a result of dividing the region of red blood cells in the phase image or determining the type of red blood cells. A memory 120 that temporarily stores the code, topological images, and learning models for loading and temporarily storing them, a processor 130 that controls components and performs calculations, and a learning processor that accelerates and processes calculations based on deep learning. (140), a marker image to distinguish the region of red blood cells in the phase image, and a learning model (150) trained to generate a red blood cell classification image containing probability information for each type of red blood cell, receiving user commands and providing the results. It may include an interface 160 that provides and a display 170 that displays various results. The learning processor 140 may be GPU-based or a separate processor that supports dedicated AI acceleration. The learning model 150 may be an inference model learned based on a generative adversarial network.

In one embodiment, the computing device 100 may include a communication unit (transceiver) for communicating with the digital hologram acquisition device 200 or the learning model training device 300.

The communication unit may include a wireless communication unit or a wired communication unit.

The wireless communication unit may include at least one of a mobile communication module, a wireless Internet module, a short-range communication module, and a location information module.

The mobile communication module transmits and receives wireless signals with at least one of a base station, an external terminal, and a server on a mobile communication network built according to LTE (Long Term Evolution), a communication method for mobile communication.

The wireless Internet module is a module for wireless Internet access, which may be built into or external to the computing device 100, and supports wireless LAN (WLAN), Wireless-Fidelity (Wi-Fi), Wireless Fidelity (Wi-Fi) Direct, and DLNA. (Digital Living Network Alliance) etc. may be used.

The short-range communication module is a module for transmitting and receiving data through short-range communication, including Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, and NFC (Near Field). Communication), etc. can be used.

The location information module is a module for acquiring the location of the computing device 100, and is a GPS (Global Positioning System) module based on satellite navigation technology, or a location information module that acquires the location based on wireless communication with a wireless communication base station or wireless access point. It can be a module. The location information module may include a WiFi module.

In one embodiment, the computing device 100 may include an input unit or output unit for user input.

The input unit includes a user interface (UI: User Interface) including a microphone and a touch interface for receiving information from the user, and the user interface may include a mouse, a keyboard, as well as mechanical and electronic interfaces implemented in the device. As long as the user's command can be input, the method and form are not particularly limited. The electronic interface includes a display capable of touch input.

The output unit is used to deliver information to the user by displaying the output of the computing device 100 to the outside, and may include a display, LED, speaker, etc. for displaying visual output, auditory output, or tactile output.

The computing device 100 may include a peripheral device interface unit for data transmission with various types of connected external devices, such as a memory card port, an external device input/output (I/O) port, etc. may include.

A method of training a learning model of the learning model training apparatus 300 according to an embodiment of the present disclosure will be described with reference to FIG. 4 .

The processor 330 of the training device 300 receives a phase image and trains an inference model that generates a red blood cell classification image and a red blood cell marker image including a marker image of red blood cells and probability information about the type of red blood cells. You can. The inference model can be trained based on a generative adversarial network in conjunction with a discrimination model that determines whether the result generated by the inference model is a GT (Ground Truth) image or an image generated by the inference model.

In one embodiment, a learning model may be trained using a machine learning-based learning processor 340 implemented appropriately for training a deep learning learning model. The composition of the training data is described in detail below.

The training device 300 can receive learning data through the communication unit 310 or train a learning model by loading learning data from a database in the memory 320, and display a display for setting up an environment for training. It may include an interface 350 including an input device such as a display device, mouse, and keyboard.

The training device 300 may transmit a learning model trained based on a generative adversarial network, the learning model to an external device, or transmit a trained inference model as part of the learning model to an external device.

The learning model may be based on an artificial neural network, where artificial neurons (nodes) that form a network through the combination of synapses change the strength of the synapse connection through learning, creating an overall model with problem-solving capabilities. It can mean.

The structure and learning method of an inference model according to an embodiment of the present disclosure will be described with reference to FIG. 5.

The learning model may be based on a plurality of convolutional layers, and in one embodiment, the inference model may have a structure including a plurality of residual blocks based on the convolutional layer.

The inference model consists of an encoder and a decoder including residual blocks, and at least one feature map generated in the intermediate stage of the encoder may be concatenated with a skip connection in the intermediate stage of the decoder.

The residual block may include a plurality of convolutional layers as shown in the structure of FIG. 7, and may be a residual block with identity mapping in which the feature map input to the residual block is transmitted to the output of the residual block.

A convolutional layer may have a single size stride with a filter size of 3 x 3, but the first convolutional layer of the residual block may have a stride size of 2. Therefore, a neural network can be constructed with fewer parameters and the success of learning can be guaranteed even if the network becomes deeper.

The inference model may be trained to receive a phase image 101 as input and output a plurality of images 102 and 103.

The inference model can be trained with learning data in which the phase image 101 has a marker image 103, which is a plurality of GT images, and a red blood cell classification image 102, which is probability information about the type of red blood cell, as labels.

The marker image 103 is a segment image in which the inner region of the red blood cell is marked in pixel units (which may be elements of a matrix), and the inner region of the red blood cell in the marker image 103 is finally displayed in the computing device 100. It may be a smaller area than the final segmented region of the red blood cells being produced. The marker image generated by the learned inference model can be used as a marker for applying the marker-based watershed algorithm in the computing device 100.

According to an embodiment of the present invention, the marker applied to the marker-based watershed algorithm is generated from a deep learning-based inference model rather than an image processing method such as a conventional distance-based marker generation method, thereby creating a computing device. (100) can accurately distinguish areas of overlapping red blood cells.

The red blood cell classification image 102 contains probability information of multiple types of red blood cells, for example, discocytes, spherocytes, echinocytes, and stomatocytes, in pixel units (elements of the matrix). It may be an image displayed as (may be). Accordingly, the red blood cell classification image 102 may be the result of a type of semantic segmentation.

Therefore, the learned inference model can be trained to generate two different types of images, a marker image and a red blood cell classification image, and the final output layer of the inference model has a plurality of different output layers, each of the plurality of output layers It can be connected to different types of activation functions. For example, an output layer that generates a marker image may include a sigmoid activation function, and an output layer that generates a red blood cell classification image may include a softmax activation function.

The structure and learning method of a discrimination model according to an embodiment of the present disclosure will be described with reference to FIG. 6.

The discriminant model may include a plurality of convolutional layers, and the concatenated image of the marker image generated from the inference model and the red blood cell classification image is training data 105 labeled as fake, and a marker image that is a GT image. The image connected to the red blood cell classification image may be trained with training data 104 labeled as real.

The objective function for training the inference model may be a combination of different types of loss functions, and may be a loss that reflects the adversarial loss between the inference model and the discrimination model, as shown in Equation 1. function( ) and a loss function (Ls), which represents the cost of the red blood cell classification image, and a loss function (LM), which represents the cost of the marker image.

G and D refer to the inference model and discrimination model, and means empirical weight parameters.

Each loss function can be defined by <Equation 2> to <Equation 4>.

In <Equation 2>, x refers to the phase image of red blood cells, and y refers to the concatenation image of the red blood cell classification image, which is a GT image, and the marker image. G(x) refers to the combined image of the red blood cell classification image and marker image generated from the inference model.

In <Equation 3>, LDice refers to a loss function based on dice loss, and LCE refers to a loss function based on cross entropy loss.

y1, G1(x) refer to the red blood cell classification image, which is a GT image, and the classification image generated from the inference model, respectively, and y2, G2(x) refer to the marker image, which is a GT image, and the marker image generated from the inference model, respectively. .

Therefore, the learning device 300 uses a loss function (Ls), which means the cost of the red blood cell classification image, and a loss function (LM), which means the cost of the marker image, as a loss function ( ), it can be trained to accurately generate both different types of marker images and red blood cell classification images.

A red blood cell evaluation method according to an embodiment of the present disclosure will be described with reference to FIGS. 8 to 10.

The red blood cell evaluation device generates a phase image 901 by reconstructing a hologram taken of blood containing red blood cells provided from a digital holographic acquisition device such as a digital holographic microscope or another device, or collecting red blood cells containing red blood cells from a digital holographic acquisition device. A phase image 901 created by reconstructing a hologram taken of blood can be provided through a network or storage medium (S110).

In one embodiment, the red blood cell evaluation device may improve calculation speed by dividing the provided phase image 901 into patches and inputting them into the inference model 150.

The red blood cell evaluation device may generate a red blood cell marker image 903 and a red blood cell classification image 902 by inputting the red blood cell phase image 901 into the learned inference model 150 (S120). The red blood cell classification image 902 is an image that displays probability information about multiple types of red blood cells in pixel units, and may be a type of semantic segmentation result.

The red blood cell evaluation device may generate a binary mask image 902a used in the watershed algorithm based on the red blood cell classification image 902 containing probability information about the type of red blood cell, and the binary mask image 902a is a watershed algorithm. It can be used as a ridge (boundary) for an algorithm. The binary mask image 902a of FIG. 9 is created by generating a binary image of the red blood cell classification image 902, which includes probability information about the type of red blood cells of a plurality of red blood cells, using an image processing method such as a threshold. , It may be an image 902a from which noise has been removed through morphology processing.

The red blood cell evaluation device may acquire marker images 903 and 903a of red blood cells by inputting the phase image 901 into the deep learning-based inference model 150 (S130).

As previously explained in the learning device, the marker images (903, 903a) are segmented images in which the clear inner region of the red blood cell is marked in pixel units, and the inner region of the red blood cell in the marker images (903, 903a) is the final image generated by the red blood cell evaluation device. The area may be smaller than the final division area 906 of the red blood cell. The marker images 903 and 903a generated by the learned inference model can be used as markers for applying the marker-based watershed algorithm in the red blood cell evaluation device.

The red blood cell evaluation device applies the watershed algorithm to the marker images (903, 903a) generated from the inference model (150) and the binary mask image (902a) based on the red blood cell classification image (902) to identify red blood cells included in the phase image (101). Segmentation information 906 may be generated in which the regions of the cells are distinguished, and the segmentation information 906 may be information in which a plurality of overlapped red blood cells are distinguished from each other as separate red blood cells. That is, when red blood cells overlap in the binary mask image 902a, they are not distinguished as separate red blood cells, but by applying the marker images 903 and 903a generated by the inference model 150 together to the watershed algorithm, the overlapped red blood cells are also They can be accurately distinguished as individual red blood cells.

In one embodiment, the red blood cell evaluation device displays (906) the region of each individual red blood cell in the phase image (901) based on segmentation information, and types of red blood cells individually divided into regions based on the red blood cell classification image (902). After finally determining, it can be displayed (907) on the phase image (901).

Some of the steps in FIG. 8 may be executed before other steps, but this does not necessarily mean that they are executed in this order. For example, generating or acquiring a binary mask image (S120) may be performed before or after acquiring a marker image based on an inference model (S130).

Referring to FIG. 10, the conceptual steps of generating a final image 1007 in which the area and type of red blood cells including the final overlapping red blood cells are displayed in the phase image 1001 and the image without using a marker based on an embodiment of the present invention are shown. You can check the results by comparing them.

For example, segmentation information (1006) generated by applying the binary mask image (1002) and marker image (1003) generated from the inference model that received the phase image (1001) to the marker-based watershed algorithm and the type of red blood cell. Based on the red blood cell classification image that includes probability information about the red blood cells, the type of red blood cells 1006 divided into individual regions can be finally determined and displayed on the phase image 1001 to generate the final image 1007. .

Figure 10(b) is a phase image, Figure 10(c) is a result without using a marker image according to an embodiment of the present invention, and Figure 10(d) is a result using a marker image according to an embodiment of the present invention. am. In Figure 10(d), it can be seen that the overlapping red blood cells are accurately classified into different red blood cells compared to Figure 10(c).

The results of a red blood cell identification experiment according to an embodiment of the present disclosure will be described with reference to FIGS. 11 and 12.

Figure 11 is a diagram showing the results of an experiment for classifying red blood cells and determining their type using a learning model according to an embodiment of the present disclosure.

Referring to FIG. 11 , it can be seen that the overlapping red blood cells (arrows) in the phase image 111 are accurately classified into individual red blood cells in the experimental results 114 according to an embodiment of the present invention, as in the GT image 112. In addition, it can be confirmed that different types of red blood cells are accurately classified into individual red blood cells. In contrast, it can be confirmed that the conventional experimental results based on VGG16-FCN (113) cannot accurately distinguish between overlapping red blood cells.

Figure 12 is a diagram showing the accuracy of the results of the red blood cell region classification and type discrimination experiment of the learning model according to an embodiment of the present disclosure. As can be seen in Figure 12, it can be seen that the types of each red blood cell are accurately distinguished even among different types of red blood cells.

This is the result of a quantitative evaluation on a pixel basis based on the Dice coefficient and a quantitative evaluation on an object basis based on the Aggregated Jaccard Index (AJI), and a comparison between the conventional methods and the embodiment of the present invention is shown in <Table 1>. It can also be confirmed that the classification and discrimination of each red blood cell, including overlapping red blood cells, is more accurate than conventional methods.

The results of a red blood cell identification experiment according to another embodiment of the present disclosure will be described with reference to FIGS. 13 and 14.

The red blood cell evaluation device may check the ratio of discoid red blood cells in the phase image based on the red blood cell classification image and the segmentation information, and estimate the storage period or usability of blood from which the phase image was obtained based on the ratio of discoid red blood cells.

FIG. 13 is a diagram showing the results of an experiment for classifying red blood cells region and determining type of the learning model for blood by storage period according to an embodiment of the present disclosure, and FIG. 14 is a diagram showing the change in ratio of red blood cell types of the learning model for blood by storage period This is a drawing showing the results of the experiment.

Referring to Figures 13 and 14, red blood cell region classification and type for blood phase images containing red blood cells at the storage period of 8, 13, 16, 23, 27, 30, 34, 37, 40, 47, and 57 days. From the results of the discrimination experiment, it can be seen that the number of discocytes decreases and the number of spherocytes and echinocytes increases. In particular, it can be seen that the number of discoid red blood cells decreases rapidly at 30 and 40 days of storage.

Accordingly, the red blood cell evaluation device determines the ratio of discoid red blood cells and warns when the ratio of discoid red blood cells in blood falls below a preset ratio, thereby preventing damaged blood from being used in surgery, etc.

The present disclosure described above can be implemented as computer-readable code on a program-recorded medium. Computer-readable media includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is. Additionally, the computer may include a processor for each device.

Meanwhile, the program may be specially designed and configured for the present disclosure, or may be known and usable by those skilled in the art of computer software. Examples of programs may include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

In the specification (particularly in the claims) of the present disclosure, the use of the term “above” and similar referential terms may refer to both the singular and the plural. In addition, when a range is described in the present disclosure, the invention includes the application of individual values within the range (unless there is a statement to the contrary), and each individual value constituting the range is described in the detailed description of the invention. It's the same.

Unless there is an explicit order or description to the contrary regarding the steps constituting the method according to the present disclosure, the steps may be performed in any suitable order. The present disclosure is not necessarily limited by the order of description of the steps above. The use of any examples or illustrative terms (e.g., etc.) in the present disclosure is merely to describe the present disclosure in detail, and unless limited by the claims, the scope of the present disclosure is limited by the examples or illustrative terms. It doesn't work. In addition, those skilled in the art will recognize that various modifications, combinations and changes may be made according to design conditions and factors within the scope of the appended claims or their equivalents.

Therefore, the spirit of the present disclosure should not be limited to the above-described embodiments, and the scope of the patent claims described below as well as all scopes equivalent to or equivalently changed from the claims are within the scope of the spirit of the present disclosure. It will be said to belong to

100: computing device
200: Digital hologram acquisition device
300: Learning model training device

Claims (20)

A method in which at least a portion of each step is performed by a processor, comprising:
Receiving a phase image of red blood cells;
Inputting the phase image into an inference model based on a convolutional layer to obtain a marker image of the red blood cell and a red blood cell classification image; and
Comprising the step of applying a marker-based watershed algorithm to the binary mask image of red blood cells generated based on the red blood cell classification image and the marker image to generate segmentation information dividing the region of the red blood cells.
Methods for evaluating red blood cells.
According to claim 1,
The red blood cell classification image includes probability information about the type of red blood cells obtained by inputting the phase image into the inference model,
Methods for evaluating red blood cells.
According to claim 1,
displaying a region of the red blood cells in the phase image based on the segmentation information; and
Further comprising displaying the type of the red blood cell based on the red blood cell classification image in the phase image,
Methods for evaluating red blood cells.
According to claim 1,
The inference model is learned based on a generative adversarial network (GAN),
Methods for evaluating red blood cells.
According to clause 4,
The inference model has a plurality of output layers, and the plurality of output layers are set to output the marker image and the red blood cell classification image, respectively.
Methods for evaluating red blood cells.
According to clause 4,
The inference model has a plurality of output layers, and the plurality of output layers are each connected to different types of activation functions,
Methods for evaluating red blood cells.
According to clause 4,
The inference model is,
An inference model learned based on the results of inputting the red blood cell marker image and red blood cell classification image generated from the above inference model into a discriminative model of a generative adversarial network along with GT (ground truth) data,
Methods for evaluating red blood cells.
According to clause 7,
The inference model includes a plurality of residual blocks in which an input of a residual block based on a convolutional layer is added to the final layer of the residual block,
The inference model is composed of an encoder and a decoder including the residual block, and at least one feature map generated in the intermediate stage of the encoder is concatenated with a skip connection to the intermediate stage of the decoder,
Methods for evaluating red blood cells.
According to claim 1,
The step of generating the segmentation information is,
Including generating the segmentation information to distinguish a plurality of overlapped red blood cells into different red blood cells,
Methods for evaluating red blood cells.
According to clause 2,
Confirming the ratio of discoid red blood cells in the phase image based on the red blood cell classification image and the segmentation information; and
Comprising the step of estimating the storage period or usability of the blood from which the phase image was obtained based on the proportion of discoid red blood cells,
Methods for evaluating red blood cells.
processor; and
A memory electrically connected to the processor and storing at least one code executed by the processor,
When the memory is executed through the processor, the processor
Input the phase image of red blood cells into an inference model based on a convolutional layer to obtain a marker image of the red blood cells and a red blood cell classification image, and apply a marker-based watershed to the binary mask image of the red blood cells generated based on the red blood cell classification image and the marker image. storing a code that causes the application of a (watershed) algorithm to generate segmentation information that distinguishes regions of the red blood cells,
Red blood cell assessment device.
According to claim 11,
The red blood cell classification image includes probability information about the type of red blood cells obtained by inputting the phase image into the inference model.
According to claim 11,
The memory allows the processor to:
Displaying the area of the red blood cells in the phase image based on the segmentation information,
further storing a code that causes the type of red blood cell to be displayed based on the red blood cell classification image in the phase image,
Red blood cell assessment device.
According to claim 11,
The inference model is learned based on a generative adversarial network (GAN),
Red blood cell assessment device.
According to claim 14,
The inference model has a plurality of output layers, and the plurality of output layers are set to output the marker image and the red blood cell classification image, respectively.
Red blood cell assessment device.
According to claim 14,
The inference model has a plurality of output layers, and the plurality of output layers are each connected to different types of activation functions,
Red blood cell assessment device.
According to claim 14,
The inference model is,
An inference model learned based on the results of inputting the red blood cell marker image and red blood cell classification image generated from the above inference model into a discriminative model of a generative adversarial network along with GT (ground truth) data,
Red blood cell assessment device.
According to claim 17,
The inference model includes a plurality of residual blocks in which an input of a residual block based on a convolutional layer is added to the final layer of the residual block,
The inference model is composed of an encoder and a decoder including the residual block, and at least one feature map generated in the intermediate stage of the encoder is concatenated with a skip connection to the intermediate stage of the decoder,
Red blood cell assessment device.
According to claim 11,
The memory allows the processor to:
further storing code that causes the segmentation information to be generated to distinguish the overlapped plurality of red blood cells as different red blood cells,
Red blood cell assessment device.
According to claim 12,
The memory allows the processor to:
Code that causes to determine the proportion of discoid red blood cells in the phase image based on the red blood cell classification image and the segmentation information, and to estimate the storage period or usability of blood from which the phase image was obtained based on the proportion of discoid red blood cells. To save more,
Red blood cell assessment device.

Images