WO2023182702A1

WO2023182702A1 - Artificial intelligence diagnosis data processing device and method for digital pathology images

Info

Publication number: WO2023182702A1
Application number: PCT/KR2023/003136
Authority: WO
Inventors: 윤길중; 김승종
Original assignee: 주식회사 몰팩바이오
Priority date: 2022-03-25
Filing date: 2023-03-08
Publication date: 2023-09-28
Also published as: KR102476888B1

Abstract

The present invention relates to an artificial intelligence diagnosis data processing device and method for digital pathology images. The artificial intelligence diagnosis data processing device comprises: an image pre-treatment unit for converting pathology slice images into matrix data; a smoothing processing unit for correcting the converted matrix data according to a preset threshold value; a polygon generation unit for converting a correction image obtained through the smoothing process unit into a polygon image; an area calculation unit for calculating a polygon's area in the converted polygon image; and a result output unit that converts an extraction polygon, which is extracted by filtering according to the calculated polygon's area, into a file format for artificial intelligence diagnosis and returns same. Thus, artificial intelligence diagnosis data of pathology images can be effectively processed.

Description

Artificial intelligence diagnostic data processing device and method for digital pathology images

The present invention converts a pathology slide image into matrix data, corrects the converted matrix data according to a preset threshold, converts the obtained corrected image into a polygon image, and calculates the polygon area from the converted polygon image. After that, the extracted polygons extracted by filtering according to the calculated polygon area are converted to a file format for artificial intelligence diagnosis and returned, thereby providing artificial intelligence diagnosis data of digital pathology images that can effectively process pathology images for artificial intelligence learning. It relates to a processing device and method.

As is well known, an artificial intelligence (AI) system is a computer system that implements human-level intelligence, and refers to a system in which machines learn and make decisions on their own and become smarter.

Because the recognition rate of these artificial intelligence systems improves as they are used and they can more accurately understand users' tastes, existing rule-based smart systems are gradually being replaced by deep learning-based artificial intelligence systems.

Artificial intelligence technology as described above consists of machine learning and element technologies utilizing machine learning. Machine learning refers to an algorithm that classifies and learns the characteristics of input data on its own, and element technologies include machine learning such as deep learning. It is a technology that utilizes algorithms and can consist of technical fields such as linguistic understanding, visual understanding, inference/prediction, knowledge expression, and motion control.

Here, fields where artificial intelligence technology is applied may include, for example, linguistic understanding, visual understanding, reasoning and prediction, knowledge expression, and motion control. In particular, in the field of visual understanding, for example, object recognition and object tracking , may include technologies such as image search, person recognition, scene understanding, spatial understanding, and image improvement.

Recently, among artificial intelligence algorithms, deep learning has been in the spotlight in various fields. Especially in the field of object recognition, a technology called convolutional neural network (CNN), a type of deep learning, is in the spotlight.

This CNN is a model that mimics the human brain function created based on the assumption that when a person recognizes an object, he or she extracts the basic features of the object and then performs complex calculations in the brain to recognize the object based on the results. Various filters to extract image features through convolution operations and pooling or non-linear activation functions to add non-linear characteristics (e.g., sigmod, ReLU (rectified linear unit) ), etc.) can be used.

Meanwhile, in modern medicine, medical imaging is used as a very important tool for effective disease diagnosis and patient treatment. Developments in imaging technology enable the acquisition of more sophisticated medical imaging data, but in response, the amount of data is becoming increasingly vast. Therefore, it is difficult to analyze medical image data relying on human vision.

In order to solve the above-mentioned problems, extensive pathological images are acquired through various medical devices (e.g., ultrasound, CT (computed tomography), MRI (magnetic resonance imaging), etc.), and artificial intelligence is used to obtain extensive pathological images. Various technologies for diagnosing diseases by analyzing and learning are being researched and developed, and processing technologies to improve artificial intelligence diagnostic performance are also being researched.

[Prior art literature]

[Patent Document]

1. Korean Patent No. 10-2237696 (registered on 2021.04.02)

The present invention converts a pathology slide image into matrix data, corrects the converted matrix data according to a preset threshold, converts the obtained correction image into a polygon image, and calculates the polygon area from the converted polygon image, Artificial intelligence diagnosis data processing of digital pathology images that can effectively process artificial intelligence diagnosis data of pathology images by converting and returning the extracted polygons extracted by filtering according to the calculated polygon area into a file format for artificial intelligence diagnosis. The purpose is to provide a device and method.

The purposes of the embodiments of the present invention are not limited to the purposes mentioned above, and other purposes not mentioned will be clearly understood by those skilled in the art from the description below. .

According to one aspect of the present invention, an image pre-processing unit for converting a pathology slide image into matrix data; a smoothing processor that corrects the converted matrix data according to a preset threshold; a polygon generator that converts the correction image obtained through the smoothing processing unit into a polygon image; an area calculation unit that calculates the polygon area from the converted polygon image; And a result output unit that converts the extracted polygon extracted by filtering according to the calculated polygon area into a file format for artificial intelligence diagnosis and returns it. An artificial intelligence diagnostic data processing device for a digital pathology image including a.

In addition, according to one aspect of the present invention, the image pre-processing unit divides the pathology slide image into patch images of a preset size, and then divides the pathology slide image into patch images of a preset size, and then divides the pathological slide image into patch images of a preset size, and then outputs binary data results according to the diagnosis results of each patch image. An artificial intelligence diagnostic data processing device for converting digital pathology images into matrix data may be provided.

In addition, according to one aspect of the present invention, the smoothing processor applies a Gaussian blur effect (Gaussian smoothing, Gaussian blur) to the matrix data and then corrects the artificial intelligence diagnostic data of the digital pathology image according to the preset threshold. A processing device may be provided.

In addition, according to one aspect of the present invention, the polygon generator converts the coordinates of the correction image into points and then converts them into the polygon image using a concave hull algorithm. An intelligent diagnostic data processing device may be provided.

In addition, according to one aspect of the present invention, the result output unit may be provided with an artificial intelligence diagnostic data processing device for a digital pathology image that calculates the polygon area using a shoelace formula in the polygon image. .

According to another aspect of the present invention, converting a pathology slide image into matrix data; Obtaining a corrected image by correcting the converted matrix data according to a preset threshold; Converting the obtained correction image into a polygon image; Calculating a polygon area from the converted polygon image; and converting and returning the extracted polygons extracted by filtering according to the calculated polygon area into a file format for artificial intelligence diagnosis. A method of processing artificial intelligence diagnostic data of a digital pathology image may be provided, including:

In addition, according to another aspect of the present invention, the step of converting to matrix data includes dividing the pathology slide image into patch images of a preset size, and then binary data results output according to the diagnosis results of each patch image. An artificial intelligence diagnostic data processing method of a digital pathology image that converts the value into the matrix data may be provided.

In addition, according to another aspect of the present invention, the step of acquiring the corrected image includes applying Gaussian smoothing (Gaussian blur) to the matrix data and then correcting the digital pathology image according to the preset threshold. An artificial intelligence diagnostic data processing method may be provided.

In addition, according to another aspect of the present invention, the step of converting into a polygon image includes converting the coordinates of the correction image into points and then converting the digital image into the polygon image using a concave hull algorithm. A method of processing artificial intelligence diagnostic data of pathology images may be provided.

In addition, according to another aspect of the present invention, the step of calculating the polygon area includes an artificial intelligence diagnostic data processing method of a digital pathology image that calculates the polygon area using a shoelace formula in the polygon image. can be provided.

The present invention converts a pathology slide image into matrix data, corrects the converted matrix data according to a preset threshold, converts the obtained correction image into a polygon image, and calculates the polygon area from the converted polygon image, By converting and returning the extracted polygons extracted by filtering according to the calculated polygon area into a file format for artificial intelligence diagnosis, the processing of artificial intelligence diagnosis data of pathology images can be effectively performed.

1 is a block diagram showing an artificial intelligence diagnostic data processing device for digital pathology images according to an embodiment of the present invention;

2 to 10 are diagrams for explaining the processing of pathology images in an artificial intelligence diagnostic data processing device for digital pathology images according to an embodiment of the present invention;

Figure 11 is a flow chart showing the process of processing digital pathology images into artificial intelligence diagnosis data according to another embodiment of the present invention;

12 to 22 are diagrams for explaining a process of processing digital pathology images into artificial intelligence diagnosis data according to another embodiment of the present invention.

Advantages and features of the embodiments of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure of the present invention is complete and to be understood by those skilled in the art. It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

In describing embodiments of the present invention, if a detailed description of a known function or configuration is judged to unnecessarily obscure the gist of the present invention, the detailed description will be omitted. The terms described below are terms defined in consideration of functions in the embodiments of the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, the definition should be made based on the contents throughout this specification.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a block diagram showing an artificial intelligence diagnosis data processing device of a digital pathology image according to an embodiment of the present invention, and Figures 2 to 10 are artificial intelligence diagnosis data of a digital pathology image according to an embodiment of the present invention. This is a diagram to explain processing a pathology image in a processing device.

1 to 10, the artificial intelligence diagnostic data processing device for digital pathology images according to an embodiment of the present invention includes an image pre-processing unit 110, a smoothing processing unit 120, a polygon generating unit 130, and area calculation. It may include a unit 140, a result output unit 150, etc.

The image preprocessing unit 110 converts the pathology slide image into matrix data. After dividing the pathology slide image into patch images of a preset size, the binary data result value is output according to the diagnosis result of each patch image. It can be converted to matrix data.

For example, as shown in Figure 2, when diagnosing a pathology slide using artificial intelligence, the slide image exceeds tens of thousands to hundreds of thousands of pixels, so it must be divided according to a preset size for diagnosis.

Accordingly, after dividing the entire slide image into patch images of a preset size (e.g., 256 pixels, 512 pixels, etc.), the divided patch images can be analyzed, learned, and diagnosed using artificial intelligence.

In this case, misdiagnosis may occur in a local area during the analysis, learning, and diagnosis process of artificial intelligence, and there was a problem in clearly recognizing the boundary of the diagnosis area when diagnosing each part on a patch image basis, which will be described later. The image processing process must be performed.

First, a classification method using CNN (convolution neural network) is applied among the artificial intelligence models, but binary data of 0 or 1 is output for each patch image as a result, and the entire pathology slide image is divided into patches of a preset size. After dividing into images, each patch image is diagnosed and output as binary data, and then the output binary data can be converted into matrix data.

For example, when a pathology slide image as shown in FIG. 3 is divided and converted into matrix data as a result of one pixel per divided patch image, a converted image as shown in FIG. 4 can be obtained.

The smoothing processing unit 120 corrects the converted matrix data according to a preset threshold. After applying Gaussian blurring (Gaussian blur) to the matrix data converted through the image preprocessing unit 110, the preset It can be corrected according to the threshold.

For example, the matrix data converted through the image pre-processing unit 110 has a size of (existing image resolution/patch size), and is spaced apart from the main area (area of a certain size or more) of the converted image in the form of a small dot. After applying a Gaussian blur effect to remove the appearing values (noise), correction can be made by readjusting the resulting value of the matrix data using a preset threshold to dichotomize it based on a certain value.

Here, the Gaussian blur effect emphasizes the main area of the converted image or the surrounding background by smoothing, blurring, blurring, crushing, or distorting a specific area or the entire conversion image of matrix data. By blurring, noise in the converted image can be primarily removed as shown in FIG. 5.

After this, the converted image to which the Gaussian blur effect is applied can be obtained as a corrected image as shown in FIG. 6 by applying a preset threshold.

The polygon generator 130 converts the corrected image obtained through the smoothing processing unit 120 into a polygon image. After converting the coordinates of the corrected image into points, a polygon is created using the concave hull algorithm. It can be converted to an image.

For example, after converting the coordinates of each midpoint of the correction image obtained through the smoothing processing unit 120 into points and converting them into a point conversion image as shown in FIG. 7, the positive part of the result is concatenated. A polygon can be drawn using an algorithm and converted into a polygon image with a plurality of polygons as shown in FIG. 8.

Here, the Concave Hull algorithm creates a convex polygon (polygon) using some of the points when there are multiple points on a two-dimensional plane, and creates a polygon to include all points inside the convex polygon (polygon). This can be done by setting one reference point with the smallest Y coordinate in the point conversion image, aligning other points in a counterclockwise direction based on the set reference point, and sequentially connecting the outer points according to the angle, as shown in FIG. You can create a polygon image like this.

The area calculation unit 140 calculates the polygon area from the converted polygon image, and can calculate the polygon area using a shoelace formula from the polygon image generated through the polygon generation unit 130.

For example, there may be a plurality of polygons in the polygon image generated through the polygon generator 130, and the area of each polygon can be calculated using the new root formula for each polygon.

Here, the shoelace formula is a formula for calculating the area of a polygon when the coordinates of the vertices on the coordinate plane are known. The area of the polygon can be calculated by crossing and multiplying the coordinate values of each vertex of the polygon.

When calculating the area of each polygon in this way, the area calculation unit 140 can use the area calculated from the corresponding pathology slide image to specify the main area of each polygon as shown in FIG. 9.

The result output unit 150 can convert the extracted polygons extracted by filtering according to the calculated polygon area into a file format for artificial intelligence diagnosis and return it.

For example, when applying a polygon image in which a plurality of polygons are generated to an input pathology slide image, it can be displayed as an original applied image in which the main area of each polygon is specified, as shown in FIG. 9, and the result output unit 150 ) can compare the polygon area calculated through the area calculation unit 140 with a preset size condition. Here, the preset size condition may be based on 1/2 of the size (i.e., area) of the largest polygon.

As a result of the comparison, polygons whose size is relatively smaller than the preset size condition are removed, polygons that are the same as the preset size condition or relatively large are left, and then only the extracted polygons remain, and the extracted polygon image can be saved as a polygon. It can be converted to a set file format (e.g. xml, json, etc.), and the converted extracted polygon image can be returned for artificial intelligence learning, diagnostic data visualization, etc.

Here, the extracted polygon image can be converted into an image in which a plurality of extracted polygons are applied to the input pathology slide image as shown in FIG. 10.

Therefore, one embodiment of the present invention converts the pathology slide image into matrix data, corrects the converted matrix data according to a preset threshold, converts the obtained correction image into a polygon image, and converts the polygon image into a polygon image. After calculating the area, the extracted polygons filtered according to the calculated polygon area are converted to a file format for artificial intelligence diagnosis and returned, thereby effectively processing artificial intelligence diagnosis data of pathology images.

Figure 11 is a flow chart showing the process of processing digital pathology images into artificial intelligence diagnosis data according to another embodiment of the present invention, and Figures 12 to 22 are flow charts showing the process of processing digital pathology images into artificial intelligence diagnosis data according to another embodiment of the present invention. This is a diagram to explain the process of processing data.

Referring to FIGS. 11 to 22, the image pre-processing unit 110 can convert the pathology slide image into matrix data (step 1110).

In the step 1110 of converting to matrix data, the image pre-processing unit 110 divides the pathology slide image into patch images of a preset size, and then converts the pathological slide image into a binary data result value output according to the diagnosis result of each patch image. This can be converted to matrix data.

For example, among the artificial intelligence models, a classification method using CNN (convolution neural network) is applied, but binary data of 0 or 1 for each patch image is set to be output as a result, and the overall result as shown in FIG. 12 A patch image is created by dividing the pathology slide image into a specific image size, and as shown in Figure 13, artificial intelligence converts the patch images for which the diagnosis has been completed into binary data, and then uses the location information and diagnosis information of the patch image. By converting into matrix data, a converted image as shown in FIG. 14 can be obtained.

Here, in the patch images as shown in FIG. 13, (label 0) indicates positive and (label 1) indicates negative, and after diagnosing the image in each patch in WSI, the diagnosis is made as 0 or 1, and the location information By backtracking, diagnostic information can be output as binary data of 0 or 1 at the corresponding location.

Then, the smoothing processing unit 120 can obtain a corrected image by correcting the matrix data converted through the image pre-processing unit 110 according to a preset threshold (step 1120).

In the step 1120 of acquiring the corrected image, the smoothing unit 120 may apply a Gaussian blur effect to the matrix data and then correct it according to a preset threshold.

For example, the matrix data converted through the image pre-processing unit 110 has a size of (existing image resolution/patch size), and is spaced apart from the main area (area of a certain size or more) of the converted image as shown in Figure 14. After applying the Gaussian blur effect as shown in FIG. 15 to remove the values (noise) that appear in the form of small dots, the resulting value of the matrix data is obtained by using a preset threshold to dichotomize based on a certain value. By correcting by readjusting , a corrected image as shown in FIG. 16 can be obtained.

Next, the polygon generator 130 can convert the corrected image obtained through the smoothing processor 120 into a polygon image (1130).

In the step 1130 of converting to a polygon image, the polygon generator 130 may convert the coordinates of the correction image into points and then convert them into a polygon image using the concave hull algorithm.

For example, the correction image shown in FIG. 16 can be displayed as shown in FIG. 17 when applied to a pathology slide image, where the coordinates of each midpoint of the correction image obtained through the smoothing processing unit 120 are points. After converting it into a point conversion image as shown in FIG. 18, polygons are drawn using the concave hull algorithm for the positive part of the result, and a plurality of polygons as shown in FIG. 19 are generated. A polygon image. It can be converted to .

Also, the area calculation unit 140 can calculate the polygon area from the polygon image converted through the polygon generation unit 130 (step 1140).

In the step of calculating the polygon area (1140), the area calculation unit 140 can calculate the polygon area using the shoelace formula in the polygon image. The polygon image generated through the polygon generation unit 130 includes a plurality of There may be polygons, and the area of each polygon can be calculated using the new root formula for each polygon.

When calculating the area of each polygon in this way, the area calculation unit 140 can use the area calculated from the corresponding pathology slide image to specify the main area of each polygon as shown in FIG. 20.

Next, the result output unit 150 can convert the extracted polygons extracted by filtering according to the polygon area calculated through the area calculation unit 140 into a file format for artificial intelligence diagnosis and return it (step 1150).

In the step 1150 of converting and returning the file format for artificial intelligence diagnosis, the result output unit 150 applies the polygon image in which a plurality of polygons are generated to the input pathology slide image, as shown in FIG. 20. Likewise, the main area of each polygon can be expressed as an original application image that is specified, and the polygon area calculated through the area calculation unit 140 can be compared with a preset size condition. Here, the preset size condition may be based on 1/2 of the size (i.e., area) of the largest polygon.

As a result of the comparison, polygons whose size is relatively smaller than the preset size condition are removed, polygons that are the same as the preset size condition or relatively large are left, and then only the extracted polygons remain, as shown in FIG. 21. As shown in Figure 22, convert to a preset file format (e.g., xml, json, etc.) that can be saved as a polygon, and return the converted extracted polygon image for artificial intelligence learning, diagnostic data visualization, etc. You can.

Therefore, another embodiment of the present invention converts the pathology slide image into matrix data, corrects the converted matrix data according to a preset threshold, converts the obtained correction image into a polygon image, and converts the polygon image into a polygon image. After calculating the area, the extracted polygons filtered according to the calculated polygon area are converted to a file format for artificial intelligence diagnosis and returned, thereby effectively processing artificial intelligence diagnosis data of pathology images.

In the above description, various embodiments of the present invention have been presented and explained, but the present invention is not necessarily limited thereto, and those skilled in the art will understand various embodiments without departing from the technical spirit of the present invention. It will be easy to see that branch substitutions, transformations, and changes are possible.

[Explanation of symbols]

110: Image pre-processing unit

120: Smoothing processing unit

130: polygon creation unit

140: Area calculation unit

150: Result output unit

Claims

An image pre-processing unit that converts pathology slide images into matrix data;

a smoothing processor that corrects the converted matrix data according to a preset threshold;

a polygon generator that converts the correction image obtained through the smoothing processing unit into a polygon image;

an area calculation unit that calculates the polygon area from the converted polygon image; and

A result output unit that converts and returns the extracted polygons filtered and extracted according to the calculated polygon area into a file format for artificial intelligence diagnosis;

An artificial intelligence diagnostic data processing device for digital pathology images including.
In claim 1,

The image preprocessing unit,

After dividing the pathology slide image into patch images of a preset size, converting the binary data result value output according to the diagnosis result of each patch image into the matrix data.

Artificial intelligence diagnostic data processing device for digital pathology images.
In claim 2,

The smoothing processing unit,

After applying Gaussian smoothing (Gaussian blur) to the matrix data, correction is performed according to the preset threshold.

Artificial intelligence diagnostic data processing device for digital pathology images.
In claim 3,

The polygon generator,

After converting the coordinates of the correction image into points, converting them into the polygon image using the concave hull algorithm.

Artificial intelligence diagnostic data processing device for digital pathology images.
In claim 4,

The area calculation unit,

Calculate the polygon area using the shoelace formula in the polygon image.

Artificial intelligence diagnostic data processing device for digital pathology images.
Converting pathology slide images into matrix data;

Obtaining a corrected image by correcting the converted matrix data according to a preset threshold;

Converting the obtained correction image into a polygon image;

Calculating a polygon area from the converted polygon image; and

Step of converting and returning the extracted polygons extracted by filtering according to the calculated polygon area into a file format for artificial intelligence diagnosis:

Artificial intelligence diagnostic data processing method of digital pathology images including.
In claim 6,

The step of converting to matrix data is,

After dividing the pathology slide image into patch images of a preset size, the resulting binary data output according to the diagnosis result of each patch image is converted to the matrix data.

Artificial intelligence diagnostic data processing method of digital pathology images.
In claim 7,

The step of acquiring the corrected image is,

After applying Gaussian smoothing (Gaussian blur) to the matrix data, correction is performed according to the preset threshold.

Artificial intelligence diagnostic data processing method of digital pathology images.
In claim 8,

The step of converting to a polygon image is,

After converting the coordinates of the correction image into points, converting them into the polygon image using the concave hull algorithm.

Artificial intelligence diagnostic data processing method of digital pathology images.
In claim 9,

The step of calculating the polygon area is,

Calculate the polygon area using the shoelace formula in the polygon image.

Artificial intelligence diagnostic data processing method of digital pathology images.