KR102383495B1 - Medical image data extraction method - Google Patents

Abstract

The present invention relates to a medical image data extraction method for extracting medical image data which can be learned by a deep learning algorithm. According to the present invention, the method comprises: a first step of loading, by a pre-processing unit, a tissue image stored in a database and capturing a labeling area after enlarging the tissue image to generate a captured image; a second step of loading, by a data processing unit, the captured image from the pre-processing unit and adjusting the size of the captured image to the magnification of the plurality of objective lenses through a plurality of objective lenses; a third step of dividing, by the data processing unit, the captured image whose size has been adjusted to a plurality of magnifications into one or more regions without overlapping regions; a fourth step of determining, by the data processing unit, a lesion present in a region image of each region to assign labeling data to the lesion and remove the region image to which the labeling data is not assigned; and a fifth step of generating, by the data processor, an output image on the basis of a convolution operation between a filter and the region image to which the labeling data is assigned.

Description

Medical image data extraction method

The present invention relates to a method for extracting medical image data, and more particularly, to a method for extracting medical image data that can be learned by a deep learning algorithm.

Medical imaging is a very important tool for effective disease diagnosis and patient treatment in modern medicine. In addition, the development of imaging technology makes it possible to acquire more sophisticated medical image data. In exchange for such sophistication, the amount of data is gradually increasing, so it is difficult to analyze medical image data depending on the human perspective. Accordingly, clinical decision support systems and computer-aided reading systems have played an essential role in automatic medical image analysis for the past decade or so.

A conventional clinical decision support system or computer-aided reading system detects and displays a lesion area or presents reading information to a medical staff or a medical worker (hereinafter referred to as a user).

For example, in the 'method and apparatus for calculating disease diagnosis information based on medical images' disclosed in Korean Patent Application Laid-Open No. 10-2017-0017614, a region of interest in which an analysis target object is photographed is detected, a coefficient of variation is calculated, and variation It includes the step of creating a counting image and comparing it with a reference sample, and refers to the effect of diagnosing the degree of disease of a patient using medical images acquired through CT, MRI, MRa, and ultrasound imaging devices.

In particular, in recent years, artificial intelligence (AI) technology based on machine learning such as deep learning is the basis for bringing a leap forward in reading a patient's disease using medical image data. there is.

Deep learning refers to an artificial neural network-based machine learning method that simulates human biological neurons and allows machines to learn. Recently, deep learning technology has developed rapidly in the field of image recognition and is widely used in the field of reading using medical image data.

In the deep learning technology in medical images, machine learning is performed using a number of medical image data including diseases and the diseases as learning data to generate a machine learning model (hereinafter referred to as a 'reading model'), and a target to be read When the medical image data is input to the reading model, the presence of a lesion is diagnosed.

As described above, the deep learning-based machine learning method collects learning data used to generate a reading model, for example, a plurality of lesion image data and a plurality of normal image data, and learns the collected lesion image data and normal image data. Since the generated reading model reads whether the newly input medical image data to be read is a lesion or not, it is possible to increase the accuracy of the reading result when a variety of learning data is used for learning. In addition, when the reading target medical image data and the training data are similar, the accuracy of the reading result may be increased.

Republic of Korea Patent Publication No. 10-2017-0017614

Therefore, the present invention has been devised to solve the above problems, and medical image data extraction capable of extracting a plurality of medical image data by adjusting the magnification of the medical image data to be learned by the deep learning algorithm with a plurality of objective lenses The purpose is to provide a method.

Another object of the present invention is to provide a method for extracting medical image data in which correction and cropping of medical image data are performed so that a deep learning algorithm can learn.

However, the technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned are clearly to those of ordinary skill in the art to which the present invention belongs from the description below. can be understood

In the medical image data extraction method, which is a technical method for achieving the above object, a first step of generating a captured image by a preprocessor loading a tissue image stored in a database, enlarging the tissue image, and then capturing a labeling area ; a second step of a data processing unit loading the captured image from the preprocessing unit, and adjusting the size of the captured image through a plurality of objective lenses to the magnifications of the plurality of objective lenses; a third step of dividing, by the data processing unit, the captured image whose size is adjusted by a plurality of magnifications into one or more regions without overlapping regions; a fourth step of, by the data processing unit, determining a lesion present in the regional image of each region, assigning labeling data to the lesion, and removing the region image to which the labeling data is not assigned; and a fifth step of generating, by the data processing unit, an output image based on a convolution operation between a filter and a region image to which the labeling data is applied.

In addition, the second step may include: outputting a captured image before the size of the first display provided in the data processing unit is adjusted by the magnification; irradiating light to the first display by one or more LED lamps provided in the data processing unit; condensing, by the plurality of objective lenses, the reflected light from the first display on a second display provided in the data processing unit; and outputting the captured image whose size is adjusted to the plurality of magnifications on the second display.

The third step may include: calculating, by the data processing unit, a magnification and a volume value of the captured image output to the second display through a calculation algorithm; dividing, by the data processing unit, a region of the captured image by a preset number of regions per magnification calculated through the calculation algorithm using an image division algorithm; and calculating, by the data processing unit, the volume value of each zone through the calculation algorithm, and outputting the magnification and volume value of the captured image, and the volume value of each zone to the second display.

In addition, in the fifth step, in order to generate the output image, the data processing unit may use a Bilateral Filter to which a weight is given according to a distance.

In the fifth step, the data processing unit may generate the output image using the bidirectional filter and a boundary filter for detecting a boundary of an object in the area image.

In addition, the medical image data extraction method includes a sixth step of correcting the output image by the data processing unit using at least one of rotation, inversion, reduction or enlargement, and color adjustment so that the output image can be applied to a deep learning algorithm. ; may be included.

In the sixth step, the data processing unit may adjust the color of the output image by adjusting the brightness and/or saturation of the output image.

The method for extracting medical image data may further include a seventh step of cropping the corrected output image by the data processing unit to a size applicable to the deep learning algorithm.

And, in the first step, the pre-processing unit may include a capture board to capture the labeling area with at least one resolution of HD, FHD, QHD, and UHD.

The present invention has an effect of variously extracting medical image data to be learned by a deep learning algorithm through magnification adjustment using a plurality of objective lenses.

In addition, the present invention has the effect of improving the accuracy of the reading result of the deep learning algorithm by providing medical image data similar to the learning data of the deep learning algorithm to the deep learning algorithm through correction and cropping of the medical image data.

However, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be able

1 is a schematic block diagram of an apparatus for extracting medical image data according to an embodiment of the present invention.
2 is an example tissue image loaded by the preprocessor.
3 is an example of a captured image and a region image loaded by the data processing unit.
4 is an example output image generated by the data processing unit.
5 is an operation flowchart of a method for extracting medical image data according to an embodiment of the present invention.
6 is a detailed operation flowchart of the capture image loading and magnification adjustment steps.
7 is a detailed operation flowchart of the capture image segmentation step.
8 is a detailed operation flowchart of the step of generating an output image.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, since the description of the present invention is merely an embodiment for structural or functional description, the scope of the present invention should not be construed as being limited by the embodiment described in the text. That is, since the embodiment may have various changes and may have various forms, it should be understood that the scope of the present invention includes equivalents capable of realizing the technical idea. In addition, since the object or effect presented in the present invention does not mean that a specific embodiment should include all of them or only such effects, it should not be understood that the scope of the present invention is limited thereby.

The meaning of the terms described in the present invention should be understood as follows.

Terms such as “first” and “second” are for distinguishing one component from another, and the scope of rights should not be limited by these terms. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component. When a component is referred to as being “connected to” another component, it may be directly connected to the other component, but it should be understood that other components may exist in between. On the other hand, when it is mentioned that a certain element is "directly connected" to another element, it should be understood that the other element does not exist in the middle. Meanwhile, other expressions describing the relationship between elements, that is, "between" and "between" or "neighboring to" and "directly adjacent to", etc., should be interpreted similarly.

The singular expression is to be understood to include the plural expression unless the context clearly dictates otherwise, and terms such as "comprise" or "have" are not intended to refer to the specified feature, number, step, action, component, part or any of them. It is intended to indicate that a combination exists, and it should be understood that it does not preclude the possibility of the existence or addition of one or more other features or numbers, steps, operations, components, parts, or combinations thereof.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs, unless otherwise defined. Terms defined in the dictionary should be interpreted as being consistent with the meaning of the context of the related art, and cannot be interpreted as having an ideal or excessively formal meaning unless explicitly defined in the present invention.

1 is a schematic block diagram of an apparatus for extracting medical image data according to an embodiment of the present invention, FIG. 2 is an example tissue image loaded by the preprocessor, and FIG. 3 is an example captured image and region loaded by the data processing unit image, and FIG. 4 is an example output image generated by the data processing unit.

1 to 4 , the apparatus for extracting medical image data according to an embodiment of the present invention includes a pre-processing unit 10 and a data processing unit 20, and the pre-processing unit 10 and the data processing unit 20 can transmit and receive data (image, video, text) and various communication signals through the network. Also, the apparatus for extracting medical image data may include all types of servers, terminals, or devices having functions of data storage and processing.

The apparatus for extracting medical image data allows the data generated by the pre-processing unit 10 and the data processing unit 20 to be shared through a network, and these networks are specifically 3GPP (3rd Generation Partnership Project) network, LTE (Long) network. Term Evolution) network, 5G network, WIMAX (World Interoperability for Microwave Access) network, wired and wireless Internet (Internet), LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal) Area Network), Bluetooth (Bluetooth) network, Wifi network, NFC (Near Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc. may be included, but is not limited thereto.

The apparatus for extracting medical image data includes a database in which a tissue image 100, a capture image 200, a region image 300, and an output image 400, which are medical image data including lesions, are stored, respectively, and a deep learning algorithm The output image 400 to be used for learning of is transmitted to the deep learning algorithm. Accordingly, the deep learning algorithm of the present invention becomes possible to extract medical image data by learning through the output image 400 .

The preprocessor 10 loads the tissue image 100 from the database, enlarges the tissue image 100 at a certain magnification, and then captures the labeling area to be learned by the deep learning algorithm to generate the captured image 200, The captured image 200 is stored in the database. In this case, the predetermined magnification may be the same magnification as at least one of a plurality of magnifications of the objective lens 23 to be described later.

The pre-processing unit 10 is provided with a capture board 11 to capture the labeling area from the tissue image (100).

The capture board 11 captures the labeling area of the tissue image 100 in at least one resolution of HD (High Definition), FHD (Full High Definition), QHD (Quad High Definition), UHD (Ultra High Definition), and the A captured image 200 of resolution is generated. However, the resolution at which the capture board 11 captures the labeling area is not limited thereto.

The data processing unit 20 loads the captured image 200 generated by the pre-processing unit 10 from the database, and the first display 21 and the LED lamp 22 to adjust the magnification of the loaded captured image 200 . , an objective lens 23 and a second display 24 are provided.

The first display 21 loads the captured image 200 in a state before the size is adjusted by the magnification of the objective lens 23 from the database, and outputs the loaded captured image 200 .

The LED lamp 22 is provided with an LED that surrounds the optical axis of the objective lens 23 and irradiates light toward the captured image 200 output to the first display 21 , and the capture output to the first display 21 . A plurality of light irradiated toward the image 200 may be provided so as to be reflected by each of the plurality of objective lenses 23 .

The objective lens 23 allows the reflected light from the first display 21 generated by the plurality of LED lamps 22 to be focused on the second display 24, respectively, and the captured image 200 is sized at a plurality of magnifications. It may be provided in plurality in order to be adjusted.

In this case, the objective lens 23 may adjust the magnification of the captured image 200 to 200% (40x), 100% (20x), 50% (10x), 25% (5x), or 20% (4x). However, the magnification of the objective lens 23 is not limited thereto.

The objective lens 23 has an effect of variously extracting medical image data to be learned by the deep learning algorithm by adjusting the magnification of the captured image 200 .

The second display 24 outputs the captured image 200 condensed by the objective lens 23 in 1:1 correspondence with each objective lens 23, and 1:1 with each objective lens 23 It is preferable to be provided in plurality in order to respond to .

The plurality of second displays 24 may be displays that can be checked by a user, and at least one second display 24 not only outputs the captured image 200 , but also a region image 300 and an output image 400 . It may be a display that outputs the region image 300 and the output image 400 to the user.

The data processing unit 20 is further provided with a calculation algorithm 25 and an image segmentation algorithm 26 to segment the region 250 of the captured image 200 .

The calculation algorithm 25 calculates the magnification and volume values of the captured image 200 respectively output to the plurality of second displays 24, respectively, and the region 250 of the captured image 200 through the image segmentation algorithm 26 as shown in FIG. 3 , can be divided into 8 horizontal (column) x vertical (line) 4 sections. In addition, when the division of the region 250 is completed, the volume value of the region 250 is calculated, and the magnification and volume value of the captured image 200 and the volume value of the region 250 are plural together with the captured image 200 . to be respectively output to the second display 24 of

At this time, the division of the region 250 of the captured image 200 by the calculation algorithm 25 is not limited to 8 x 4 in width, and the number of divisions in the region 250 depends on the magnification of the captured image 200 . may vary.

The image segmentation algorithm 26 divides the regions 250 of the captured image 200 by the number of preset regions per magnification calculated through the calculation algorithm 25, so that one or more region images 300 per each region 250 to extract

The data processing unit 20 determines whether there is a lesion in the region image 300 , assigns the labeling data 350 to the lesion, and removes the noise data to which the labeling data 350 is not attached. Specifically, the data processing unit 20 includes a lesion determination algorithm (not shown), and by determining whether a lesion exists through the lesion determination algorithm, labeling data ( 350), and a noise removal algorithm (not shown) is provided to remove the second region image 300b to which the labeling data 350 is not assigned as the lesion does not exist from the region image 300 .

The data processing unit 20 constructs a convolutional neural network to generate an output image 400 based on a convolution operation between the region image 300 to which the labeling data 350 is given and the filter. Specifically, the data processing unit 20 obtains a weighted average from the surrounding elements of each pixel, and either uses the Bilateral Filter alone, which uses a weight determined by the difference in brightness from the central pixel value, or the bidirectional filter. The area image 300 is corrected (or image processed) by a convolution operation using a filter that combines a filter and a boundary filter that detects the boundary of an object in the area image 300, so that the area image 300 has the same number of output images ( 400) is created. However, the filter is not limited thereto, and an average blurring filter, a median blurring filter, a guided image filter, and the like may be further included to perform a convolution operation with the regional image 300 .

The data processing unit 20 rotates, inverts, reduces or enlarges the output image 400, and adjusts the color of the output image 400 so that the generated output image 400 can be applied to the deep learning algorithm. (400) is corrected. Specifically, the data processing unit 20 rotates 0 to 360° of the output image 400, up and down and/or left and right inversion, up and down and/or left and right reduction, up and down and/or left and right enlargement, brightness and/or Calibrate the output image 400 to adjust the saturation.

The correction process of the output image 400 may be omitted if the output image 400 is already applicable to a deep learning algorithm.

The data processing unit 20 crops the output image 400 so that the output image 400 is transformed to a size applicable to the deep learning algorithm. At this time, if the output image 400 is already a size applicable to the deep learning algorithm, the cropping process may be omitted.

The data processing unit 20 provides the deep learning algorithm with an output image 400 similar to the learning data of the deep learning algorithm through correction and cropping of the output image 400 to improve the accuracy of the reading result of the deep learning algorithm It works.

5 is an operation flowchart of a method for extracting medical image data according to an embodiment of the present invention, FIG. 6 is a detailed operation flowchart of a capture image loading and magnification adjustment step, and FIG. 7 is a detailed operation flowchart of a captured image region division step and FIG. 8 is a detailed operation flowchart of the step of generating an output image.

The method of extracting medical image data shown in FIGS. 5 to 8 may be performed by the apparatus for extracting medical image data according to an embodiment of the present invention. Accordingly, even if omitted below, the description of the apparatus for extracting medical image data may be equally applied to the description of the method of extracting medical image data.

First, the preprocessor 10 may load the tissue image 100 from the database, and capture the labeling area of a specific resolution through the capture board 11 to generate the captured image 200 (S10).

After the captured image 200 is stored in the database, the data processing unit 20 may load the captured image 200 from the database and adjust the magnification of the captured image 200 ( S20 ).

In this case, the operation flow in the loading and magnification adjustment step S20 of the captured image 200 may be subdivided as shown in FIG. 6 . Specifically, the first display 21 may load the captured image 200 from the database (S21) and output it, and the LED lamp 22 is directed toward the captured image 200 output to the first display 21 Light can be irradiated (S22), the objective lens 23 can collect the reflected light of the first display 21 (S23), and the second display 24 is an objective lens that the objective lens 23 focuses The captured image 200 whose size is adjusted at a magnification of (23) may be output (S24).

After adjusting the magnification of the captured image 200 , the data processing unit 20 may divide ( S30 ) the region 250 of the captured image 200 whose magnification has been adjusted to extract the region image 300 .

In this case, the operation flow in the segmentation step S30 of the region 250 of the captured image 200 may be subdivided as shown in FIG. 7 . Specifically, the data processing unit 20 may calculate the magnification and volume value of the captured image 200 output to the second display 24 through the calculation algorithm 25 (S31), and the image segmentation algorithm 26 The region 250 of the captured image 200 output to the second display 24 can be divided by the preset number of regions per magnification through (S32), and through the calculation algorithm 25, each region 250 is By calculating the volume value (S33), the magnification and volume value of the captured image 200 and the volume value of each region 250 may be output on the second display 24 (S34).

Meanwhile, in an embodiment of the present invention, the captured image 200 is sized at a magnification of 200% (40x), 100% (20x), 50% (10x), 25% (5x), and 20% (4x). The number of preset regions per magnification for the data processing unit 20 to divide the regions 250 of the captured image 200 through the image segmentation algorithm 26 is as shown in Table 1 below.

magnification size of captured image area X Y horizontal length Area Total 40x 7680 128 16 8 128 20x 3840 32 8 4 32 10x 1920 8 4 2 8 5x 960 4 2 One 2 4x 768 One One One One total number of districts 171

Referring to [Table 1], the data processing unit 20 can extract the region image 300 by dividing the region of the captured image 200 output to the plurality of second displays 24 into a total of 171 regions. there is. However, the present invention is not limited thereto, and a maximum of eight region images 300 may be extracted for each region 250 while moving within the region 250 . Accordingly, the region image 300 may be extracted from a minimum of 171 to a maximum of 1,368.

After the region image 300 is extracted, the data processing unit 20 gives the labeling data 350 to the lesion of the first region image 300a of the region image 300, and the lesion does not exist so that the labeling data 350 is not present. It is determined that the second area image 300b to which is not given is noise data and may be removed ( S40 ).

After applying the labeling data 350 and removing noise, the data processing unit 20 may generate an output image 400 based on a convolution operation between the region image 300 to which the labeling data 350 is applied and the filter ( S50).

In this case, the operation flow in the step S50 of generating the output image 400 may be subdivided as shown in FIG. 8 . Specifically, the data processing unit 20 applies the bidirectional filter to the region image 300 to which the labeling data 350 is given (S51) to generate the output image 400 (S53), or applies the bidirectional filter and the boundary filter (S52) to generate the output image 400 (S53).

After generating the output image 400, the data processing unit 20 may correct the output image 400 to be applicable to the deep learning algorithm (S60), and the corrected output image 400 may be applicable to the deep learning algorithm It can be cropped to the size (S70).

The detailed description of the preferred embodiments of the present invention disclosed as described above is provided to enable any person skilled in the art to make and practice the present invention. Although the above has been described with reference to preferred embodiments of the present invention, it will be understood by those skilled in the art that various modifications and changes can be made to the present invention without departing from the scope of the present invention. For example, those skilled in the art can use each configuration described in the above-described embodiments in a way in combination with each other. Accordingly, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The present invention may be embodied in other specific forms without departing from the spirit and essential characteristics of the present invention. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention. The present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In addition, claims that are not explicitly cited in the claims can be combined to form an embodiment or included as a new claim by amendment after filing.

10: preprocessor, 11: capture board,
20: data processing unit, 21: first display;
22: LED lamp, 23: objective lens,
24: second display, 25: calculation algorithm;
26: image segmentation algorithm, 100: tissue image,
200: captured image, 250: area,
300: zone image, 350: labeling data,
400: Output image.

Claims (9)

A first step of generating a captured image by the preprocessor loading the tissue image stored in the database, and capturing the labeling area after magnifying the tissue image;
The data processing unit loads the captured image from the preprocessor, and outputs the captured image output to the first display for outputting the loaded captured image as a captured image whose size is adjusted at a plurality of magnifications through a plurality of objective lenses. a second step;
a third step of dividing, by the data processing unit, the captured image whose size is adjusted by a plurality of magnifications into one or more regions without overlapping regions;
a fourth step of determining, by the data processing unit, a lesion existing in a regional image of each region, assigning labeling data to the lesion, and removing an image of a region to which the labeling data is not assigned; and
and a fifth step of the data processing unit generating an output image based on a convolution operation between the region image to which the labeling data is applied and a filter. The method of claim 1,
The second step is
outputting the captured image before the size of the first display provided in the data processing unit is adjusted by the magnification;
irradiating light to the first display by one or more LED lamps provided in the data processing unit;
condensing, by the plurality of objective lenses, the reflected light from the first display on a second display provided in the data processing unit; and
and outputting the captured image whose size is adjusted to the plurality of magnifications on the second display. 3. The method of claim 2,
The third step is
calculating, by the data processing unit, a magnification and a volume value of the captured image output to the second display through a calculation algorithm;
dividing, by the data processing unit, a region of the captured image by a preset number of regions per magnification calculated through the calculation algorithm using an image division algorithm; and
and calculating, by the data processing unit, the volume value of each zone through the calculation algorithm, and outputting the magnification and volume value of the captured image and the volume value of each zone to the second display. Medical image data extraction method. The method of claim 1,
The fifth step is
The method for extracting medical image data, characterized in that the data processing unit uses a Bilateral Filter to which a weight according to a distance is assigned to generate the output image. 5. The method of claim 4,
The fifth step is
and the data processing unit generates the output image by using the bidirectional filter and a boundary filter for detecting a boundary of an object in the area image. The method of claim 1,
A sixth step of correcting the output image by the data processing unit in at least one of rotation, inversion, reduction or enlargement, and color adjustment so that the output image can be applied to a deep learning algorithm; How to extract data. 7. The method of claim 6,
The sixth step is
The method for extracting medical image data, characterized in that the data processing unit adjusts the color of the output image by adjusting the brightness and/or saturation of the output image. 7. The method of claim 6,
A seventh step of cropping the corrected output image by the data processing unit to a size applicable to the deep learning algorithm; The method of claim 1,
The first step is
The method of extracting medical image data, characterized in that the pre-processing unit includes a capture board to capture the labeling area in at least one of HD, FHD, QHD, and UHD resolution.

Images