KR102639140B1 - Method for biopsy and apparatus for biopsy - Google Patents

Abstract

The present invention relates to a tissue examination method, comprising the steps of extracting tissue by fine needle aspiration; providing the tissue to a tissue analysis device; and performing a suitability test to determine whether the tissue is suitable for performing a biopsy through analysis of the tissue, using a first deep learning learning model, by one or more computers.

Description

Biopsy method and biopsy apparatus {METHOD FOR BIOPSY AND APPARATUS FOR BIOPSY}

The present invention relates to a tissue examination method and tissue examination device.

A biopsy is a method of examining a patient's condition by examining tissue. Methods for extracting tissue can be divided into FNA (Fine Needle Aspiration) methods and non-FNA methods. In the case of non-FNA methods, sufficient tissue can be obtained or the tissue extraction process can be easily performed again. However, the FNA method, which requires tissue extraction during an endoscopic procedure, has the inconvenience of having to perform endoscopy again if the amount of tissue is not sufficient.

In order to determine whether the tissue obtained through the FNA method is in sufficient quantity to perform a biopsy, a ROSE (Rapid On-Site Cytological Evaluation) process must be performed through a clinical pathologist to determine whether the tissue is a sufficient amount for examination, or an endoscopy laboratory The reality is that it is difficult to have a clinical pathologist on staff at all times. Accordingly, a few days after the endoscopic procedure, the clinical pathologist must determine whether the amount of tissue is sufficient for examination, and if the amount is not sufficient, there is the inconvenience of having to obtain the tissue again through the endoscopic procedure.

Accordingly, there is a need for devices and methods that can determine in real time whether the amount of tissue obtained through FNA is sufficient to perform a biopsy.

US Patent Publication No. 10025271, 2018.07.17.

The problem to be solved by the present invention is to provide a tissue examination method and a tissue examination device that can determine in real time whether the amount of tissue obtained by FNA is sufficient to perform a tissue examination.

The problem to be solved by the present invention is to provide a tissue examination method and a tissue examination device that can determine using deep learning technology whether the amount of tissue obtained by FNA is sufficient to perform a tissue examination.

The problem to be solved by the present invention is to provide a biopsy method and a biopsy device that can biopsy tissue obtained by FNA using deep learning technology.

The object of the present invention is to extract tissue by fine needle aspiration; providing the tissue to a tissue analysis device; and performing, by one or more computers, a suitability test to determine whether the tissue is suitable for performing a biopsy through analysis of the tissue, using a first deep learning learning model. there is.

The object of the present invention can be achieved by a tissue examination device coupled with a computer as hardware and stored in a medium for executing the tissue examination method.

According to the present invention, it is possible to provide a tissue examination method and a tissue examination device that can determine in real time whether the amount of tissue obtained by FNA is sufficient to perform a tissue examination.

According to the present invention, it is possible to provide a tissue examination method and a tissue examination device that can determine using deep learning technology whether the amount of tissue obtained by FNA is sufficient to perform a tissue examination.

According to the present invention, it is possible to provide a biopsy method and a biopsy device that can biopsy tissue obtained by FNA using deep learning technology.

Figure 1 is a block diagram schematically showing a tissue examination device according to an embodiment of the present invention.
Figure 2 is a flowchart schematically showing a tissue examination method according to an embodiment of the present invention.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms. The present embodiments are merely intended to make the disclosure of the present invention complete and to provide a general understanding of the technical field to which the present invention pertains. It is provided to fully inform those skilled in the art of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for describing embodiments and is not intended to limit the invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other elements in addition to the mentioned elements. Like reference numerals refer to like elements throughout the specification, and “and/or” includes each and every combination of one or more of the referenced elements. Although “first”, “second”, etc. are used to describe various components, these components are of course not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may also be a second component within the technical spirit of the present invention.

In this specification, a computer includes all various devices that can perform computational processing and visually present the results to the user. For example, computers include not only desktop PCs and laptops (Note Books), but also smart phones, tablet PCs, cellular phones, PCS phones (Personal Communication Service phones), and synchronous/asynchronous computers. This may also include IMT-2000 (International Mobile Telecommunication-2000) mobile terminals, Palm Personal Computers (Palm PCs), and personal digital assistants (PDAs). Additionally, computers may also refer to medical equipment that acquires or observes medical images. Additionally, the computer may be a server computer connected to various client computers.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless clearly specifically defined.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a block diagram schematically showing a tissue examination device according to an embodiment of the present invention.

Referring to FIG. 1, the tissue examination device 10 according to an embodiment of the present invention may include an analysis server 100, a tissue analysis device 200, and an output device 300.

The tissue examination device 10 according to an embodiment of the present invention determines whether the amount of tissue extracted by fine needle aspiration (FNA) is sufficient to perform a tissue examination, using a first deep learning learning model (Deep Neural Network; DNN) (110).

Tissue extracted by fine needle aspiration is provided to a tissue analysis device. The tissue may be provided without any additional processing, or may be provided after smearing and H&E (hematoxylin & eosin) staining.

A smear refers to observing cells in a tissue using a microscope. H&E staining means staining the nucleus with hematoxylin, a basic dye, and staining the rest with eosin, an acidic dye.

In one embodiment, tissue extracted through fine needle aspiration may be photographed with an imaging device or the like and provided to the tissue analysis device 200 in the form of image data.

The tissue examination device 10 according to an embodiment of the present invention can perform a suitability test to determine whether the tissue is suitable for performing a tissue examination using the first deep learning learning model 110 by one or more computers. . For example, the tissue examination device 10 according to an embodiment of the present invention determines whether the amount of tissue is suitable for performing a tissue examination by using the first deep learning model 110 by one or more computers. can be inspected.

Below, the first deep learning learning model 110 will be described.

The analysis server 100 is composed of one or more computers to form a first deep learning learning model 110, and serves to determine whether the amount of tissue extracted through fine needle aspiration is sufficient to perform a biopsy.

The first deep learning learning model 110 according to embodiments of the present invention refers to a system or network that builds one or more layers in one or more computers and makes decisions based on a plurality of data. For example, the first deep learning model 110 is a set of layers including a convolutional pooling layer, a locally-connected layer, and a fully-connected layer. It can be implemented as: A convolutional pooling layer or local connection layer can be configured to extract features within the image. The fully connected layer can determine correlations between features of the image. In some embodiments, the overall structure of the first deep learning model 110 may be a convolutional pooling layer followed by a local connection layer, and a local connection layer followed by a fully connected layer. The first deep learning model 110 may include various judgment standards (i.e., parameters), and new judgment standards (i.e., parameters) may be added through analysis of input images. Parameters may include, for example, cellularity, adequacy of specimen, amount of blood, and diagnosis.

Cytoplasm can be judged, for example, by the number of cells per slide. For example, if the number of cells per slide is less than 100, it can be judged as inadequate, if the number of cells per slide is more than 100 but less than 1,000, it can be judged as good, and if it has more than 1,000, it can be judged as suitable. Cytoplasm can be learned, for example, by adjusting the mixing ratio of normal pancreatic cell lines and pancreatic cancer cell lines. Additionally, learning can be done by controlling the number of cells smeared among the cells placed on the slide.

The suitability of the sample can be judged, for example, as insufficient or adequate. The suitability of the sample can be learned by, for example, adjusting the mixing ratio of normal pancreatic cell lines and pancreatic cancer cell lines. Additionally, learning can be done by controlling the number of cells smeared among the cells placed on the slide.

Blood volume can be judged as insufficient, good, etc., for example. Blood volume can be learned from, for example, red blood cell count.

Diagnosis can be determined as, for example, benign, atypical tumor, suspicious for malignancy, malignancy, or inadequate for reporting. Diagnosis can be learned, for example, by labeling lesions and normal parts of tissue.

The first deep learning learning model 110 according to embodiments of the present invention is a structure called a convolutional neural network suitable for image analysis, and has the characteristic of automatically learning the features with the greatest discriminative power from given image data. It can be composed of an integrated structure of an extraction layer (Feature Extraction Layer) and a prediction layer that learns a prediction model to produce the highest prediction performance based on the extracted features.

The feature extraction layer is a convolution layer that creates a feature map by applying multiple filters to each area of the image and spatially integrates the feature map to extract features that are invariant to changes in position or rotation. It can be formed in a structure in which the pooling layer that allows extraction is alternately repeated several times. Through this, various levels of features can be extracted, from low-level features such as points, lines, and surfaces to complex and meaningful high-level features.

The convolution layer obtains a feature map by taking a non-linear activation function as the inner product of the filter and the local receptive field for each patch of the input image, and compares it with other network structures. Therefore, CNN is characterized by using filters with sparse connectivity and shared weights. This connection structure reduces the number of parameters to learn, makes learning through the backpropagation algorithm efficient, and ultimately improves prediction performance.

The integration layer (Pooling Layer or Sub-sampling Layer) creates a new feature map using the local information of the feature map obtained from the previous convolution layer. In general, the feature map newly created by the integration layer is reduced to a smaller size than the original feature map. Representative integration methods include max pooling, which selects the maximum value of the corresponding area within the feature map, and There is average pooling, which calculates the average value of an area. The feature map of the integrated layer is generally less affected by the location of arbitrary structures or patterns present in the input image than the feature map of the previous layer. In other words, the integration layer can extract features that are more robust to local changes such as noise or distortion in the input image or previous feature map, and these features can play an important role in classification performance. Another role of the integration layer is to reflect the characteristics of a wider area as you go up to the higher learning layer in the deep structure. As the feature extraction layer accumulates, regional characteristics are reflected in the lower layers, and as you go up to the upper layer, you can reflect the characteristics of a wider area. Features that reflect the characteristics of the entire abstract image can be generated.

In this way, the features finally extracted through repetition of the convolutional layer and the integration layer are classified into fully connected layers such as a classification model such as a multi-layer perception (MLP) or a support vector machine (SVM). -connected layer) and can be used for classification model learning and prediction.

However, the structure of the first deep learning model 110 according to embodiments of the present invention is not limited to this, and may be formed of a neural network of various structures.

The client may include at least one of a tissue analysis device 200 and an output device 300 that receive tissue. The tissue analysis device 200 and the output device 300 may be implemented as one device or may be implemented as separate devices.

The tissue analysis device 200 may be a device that receives tissue extracted through fine needle aspiration and transmits tissue amount data to the analysis server 100.

The output device 300 may receive the determination result of the amount of tissue from the analysis server 100 and provide the determination result to the user in various ways. For example, the output device 300 may include a display unit to visually display the result of determining the amount of tissue and provide it to the user. Additionally, when receiving a determination result that the amount of tissue is insufficient, the output device 300 may generate vibration to inform the user that the amount of tissue is insufficient. However, the method by which the output device 300 provides the user with the result of determining the amount of tissue is not limited to this, and various output methods that can be provided to the user, such as audio output, can be utilized. Additionally, the output device 300 according to an embodiment of the present invention may include a mobile terminal.

The tissue examination device 10 according to an embodiment of the present invention can analyze tissue extracted by fine needle aspiration using the second deep learning model 120 by one or more computers.

Below, the second deep learning learning model 120 will be described. Below, we will focus on explaining the differences between the second deep learning model 120 and the first deep learning model 110, and the parts not explained are the same as those explained in the first deep learning model 110. can do.

The analysis server 100 is composed of one or more computers to form a second deep learning learning model 120, and performs the role of analyzing whether the tissue extracted through fine needle aspiration is normal. The analysis server of the second deep learning model 120 may be implemented as one server with the analysis server of the first deep learning model 110, or may be implemented as separate servers.

The second deep learning learning model 120 according to embodiments of the present invention refers to a system or network that builds one or more layers in one or more computers and makes decisions based on a plurality of data. For example, a first deep learning learning model may be implemented as a set of layers including a convolutional pooling layer, a locally-connected layer, and a fully-connected layer. You can.

The second deep learning model 120 may include various judgment standards (i.e., parameters), and new judgment standards (i.e., parameters) may be added through analysis of input images.

For example, the second deep learning model 120 has parameters that distinguish pancreatic normal cell lines (HPDE6c6) and pancreatic cancer cell lines (Capan-1, Panc-1, MIA-PaCa2, HPAC cell line) from normal tissues. This may include whether there are any abnormalities in the collected aspiration sample.

The client may include a tissue analysis device 200 or an output device 300 that receives tissue. The tissue analysis device 200 and the output device 300 may be implemented as one device or may be implemented as separate devices.

The tissue analysis device 200 may be a device that receives tissue extracted through fine needle aspiration and transmits tissue amount data to the analysis server 100. There may be one tissue analysis device 200 or there may be multiple tissue analysis devices 200.

The output device 300 may receive the determination result of the amount of tissue from the analysis server 100 and provide the determination result to the user in various ways. The output device of the second deep learning model 120 may be implemented as one device and the output device of the first deep learning model 110, or may be implemented as separate devices.

There may be one output device 300 or there may be multiple output devices 300.

Hereinafter, a tissue examination method according to an embodiment of the present invention will be described. Hereinafter, differences from the tissue examination device according to an embodiment of the present invention mentioned above will be described in detail, and parts not described may be the same as the tissue examination device according to an embodiment of the present invention.

Figure 2 is a flowchart schematically showing a tissue examination method according to an embodiment of the present invention.

Referring to Figures 1 and 2, the tissue examination method according to an embodiment of the present invention includes a tissue extraction step (S100), a tissue provision step (S200), and a compatibility test step (S300).

Tissue is extracted using fine needle aspiration (FNA) (S100). Fine needle aspiration can be performed, for example, with a syringe needle. Fine needle aspiration may be performed at the patient's lesion site.

The extracted tissue is provided to the tissue analysis device 200. The tissue may be provided to the tissue analysis device 200 without any additional processing, or may be provided after being smeared and H&E (hematoxylin & eosin) staining. The tissue may be provided to the tissue analysis device 200 and then subjected to smearing and H&E staining.

In one embodiment, tissue extracted through fine needle aspiration may be photographed with an imaging device or the like and provided to the tissue analysis device 200 in the form of image data.

A suitability test is performed by one or more computers to determine whether the tissue is suitable for performing a biopsy using the first deep learning learning model 110 (S300).

In one embodiment, the step of performing a suitability test (S300) includes the analysis server 100 receiving tissue amount data corresponding to the amount of tissue obtained by the tissue analysis device 200, and the analysis server 100 ) may include providing the status of the analyzed amount of tissue to the output device 300. The output device 300 and the tissue analysis device 200 may be implemented as one device or may be implemented as separate devices.

For example, when it is recognized that the amount of extracted tissue is insufficient to perform a biopsy, a notification may be sent to the output device 300. Notifications may refer to various forms that the user can perceive, such as visual, auditory, tactile, etc.

In one embodiment, the step of performing a suitability test (S300) may include accumulating tissue amount data of the tissue provided from one or more tissue analysis devices to generate learning tissue amount data. Through analysis of learning tissue volume data, the minimum amount of tissue required for biopsy can be calculated. It can be determined whether the amount of tissue provided to the tissue analysis device 200 is more than the minimum amount of tissue required for tissue examination. If the amount is less than the minimum amount, a notification can be sent to the output device 300.

If it is determined by the first deep learning learning model 110 that the amount of tissue provided to the tissue analysis device 200 is less than the minimum amount of tissue required for tissue examination, additional tissue may be extracted through fine needle aspiration.

The tissue examination method according to an embodiment of the present invention may further include the step of examining the tissue using the second deep learning model 120 by one or more computers. For example, when it is determined that the amount of tissue provided to the tissue analysis device 200 is more than the minimum amount of tissue required for tissue examination by the first deep learning learning model 110, using the second deep learning learning model , the tissue can be examined.

In the step of examining the tissue, abnormalities in the tissue may be recognized and a notification may be sent to the output device. Whether there is an abnormality in the organization can be determined based on the parameters of the first deep learning model 110 and the second deep learning model 120 mentioned above.

In one embodiment, the step of examining a tissue may include accumulating tissue data corresponding to the tissue provided from one or more tissue analysis devices 200 to generate learning tissue data. Normal organization data is calculated through analysis of learning organization data. It can be determined whether the tissue data exceeds a specific range formed based on normal tissue data.

In one embodiment, the step of examining a tissue may include accumulating tissue data corresponding to the tissue provided from one or more tissue analysis devices 200 to generate learning tissue data. Information on abnormal organizational characteristics can be extracted from learning organization data.

In one embodiment, the step of examining a tissue may include matching specific situation data to abnormal characteristic information and accumulating it. Abnormal characteristic information may mean, for example, having a value that falls below the normal range based on cellularity, adequacy of specimen, amount of blood, etc. Specific abnormal characteristic information can be recognized within organizational data. Situation data corresponding to the recognized abnormal feature information can be transmitted to the client.

The tissue examination method according to an embodiment of the present invention may be implemented as a program (or application) to be executed in conjunction with a hardware computer and stored in a medium included in the tissue examination device.

A program is a code coded in a computer language such as C, C++, JAVA, or machine language that the computer's processor (CPU) can read through the computer's device interface in order for the computer to read the program and execute the methods implemented in the program. (Code) may be included. These codes may include functional codes related to functions that define the necessary functions for executing methods, and may include control codes related to execution procedures necessary for the computer's processor to execute the functions according to predetermined procedures. . In addition, these codes may further include memory reference-related codes that indicate from which location (address address) in the computer's internal or external memory additional information or media required for the computer's processor to execute functions should be referenced. Additionally, if the computer's processor needs to communicate with any other remote computer or server to execute functions, how should the code communicate with any other remote computer or server using the computer's communication module? , It may further include communication-related codes regarding what information or media should be transmitted and received during communication.

A storage medium is not a medium that stores data for a short period of time, such as a register, cache, or memory, but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of storage media include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. That is, the program can be stored in various recording media on various servers that the computer can access or in various recording media on the user's computer. Additionally, the medium may be distributed across computer systems connected to a network, and computer-readable code may be stored in a distributed manner.

The tissue examination method and tissue examination device according to an embodiment of the present invention can determine in real time whether the amount of tissue obtained by FNA is sufficient to perform a tissue examination. The tissue examination method and tissue examination device according to an embodiment of the present invention can determine whether the amount of tissue obtained by FNA is sufficient to perform a tissue examination using deep learning technology.

The biopsy method and biopsy apparatus according to an embodiment of the present invention can biopsy tissue obtained by FNA using deep learning technology.

Above, embodiments of the present invention have been described with reference to the attached drawings, but those skilled in the art will understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. You will understand that it exists. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

10: Biopsy device
100: Analysis server
110: First deep learning learning model
120: Second deep learning learning model
200: tissue analysis device
300: output device

Claims (10)

A step of the analysis server receiving tissue amount data and tissue data corresponding to tissue extracted by fine needle aspiration (FNA) from the tissue analysis device;
Performing a suitability test on whether the tissue is a suitable amount for performing a biopsy through analysis of the tissue amount data, using a first deep learning learning model, by one or more computers; and
When it is determined that the tissue is in a suitable amount for performing the tissue examination, examining the tissue for abnormalities through analysis of the tissue data using a second deep learning learning model by the one or more computers; Contains,
The step of examining abnormalities in the tissue is
generating learning tissue data by accumulating tissue data received from one or more of the tissue analysis devices;
extracting normal tissue data and abnormal tissue characteristic information through analysis of the learning tissue data;
determining whether the tissue data exceeds a specific range formed based on the normal tissue data; and
Matching specific situation data to the abnormal characteristic information, and when specific abnormal characteristic information is recognized in the organization data based on the matching result, transmitting situation data corresponding to the recognized abnormal characteristic information to a client; Including,
The first deep learning model uses cytoplasm, specimen suitability, blood volume, and diagnosis as parameters,
Learning about the cytoplasm and the specimen suitability is performed by controlling the mixing ratio of normal cell lines and cancer cell lines, and learning is performed by controlling the number of cells smeared among the cells placed on the slide,
Learning about the blood volume is performed based on the red blood cell count,
A biopsy method in which learning about the diagnosis is performed by labeling lesions and normal parts of tissue. According to paragraph 1,
The steps for performing the conformity test are
A tissue examination method comprising: providing to an output device whether the amount of tissue analyzed by the analysis server is appropriate. According to paragraph 1,
The steps for performing the conformity test are
A tissue examination method comprising recognizing that the amount of tissue is insufficient to perform the tissue examination, and transmitting a notification to an output device. According to paragraph 1,
The steps for performing the conformity test are
generating learning tissue volume data by accumulating tissue volume data received from one or more of the tissue analysis devices;
calculating the minimum amount of tissue required for the tissue examination through analysis of the learned tissue amount data; and
A tissue examination method comprising: determining whether the amount of tissue included in the tissue amount data is greater than or equal to the minimum amount of tissue. delete According to paragraph 1,
The step of examining abnormalities in the tissue is
A tissue examination method comprising recognizing abnormalities in the tissue and transmitting a notification to an output device. delete delete delete A program combined with a hardware computer and stored in a computer-readable recording medium to execute the method of any one of claims 1 to 4 and 6.