KR102564740B1 - Method for correcting X-ray image with improved contrast and device for the same - Google Patents

Abstract

generating a lung region image divided into a lung region and an outer lung region in the second X-ray image by inputting a second X-ray image including the lung region into a network; 2 An X-ray image correction method is disclosed, including generating a corrected X-ray image by stretching a histogram of the brightness of a plurality of pixels constituting the lung area of the X-ray image.

Description

Method for correcting X-ray image with improved contrast and device for the same

The present invention relates to a technique for correcting an X-ray image to have improved contrast, and relates to a technique for stretching a histogram by simulating and generating a lung region image.

1 shows a process of obtaining an X-ray image using an X-ray imaging device.

When X-rays are emitted from the X-ray source 30 and pass through the subject (patient) 40 to reach the X-ray detection sensor 50 on the opposite side, the X-ray detection sensor 50 outputs An X-ray image can be obtained using the signal.

2 illustrates the X-ray image and an image obtained by stretching a histogram of the X-ray image according to an exemplary embodiment.

In the case of the X-ray image 71, the contrast may not be large due to low contrast. In this case, a method of stretching the histogram of the image may be used to increase the contrast of the X-ray image 71 .

Specifically, an X-ray image may be composed of a plurality of pixels. At this time, the contrast of the X-ray image can be enhanced by stretching a histogram indicating which values each of the plurality of pixels corresponds to with respect to values representing the brightness of the pixels.

However, at this time, as in the image 72 to which the histogram stretching method is applied (specifically, as shown in reference numeral A), a cloud effect may appear. This may be a phenomenon caused by brightening the background of the image due to a contrast deterioration phenomenon. When the conversion is performed to brighten the black parts outside the body using the histogram stretching method, the original bright parts of the bone and soft-tissue areas inside the body become brighter. As a result, there is a problem that originally bright bone and soft-tissue portions become too bright and look like clouds (contrast deterioration phenomenon), and regions within the bone and soft-tissue are difficult to distinguish from each other and be recognized.

Therefore, there is a need for a method of enhancing the contrast of the soft-tissue area where the image contrast deterioration occurs.

In order to solve the above-described problem, the present invention intends to provide an X-ray image correction method for enhancing the contrast of a soft-tissue region where contrast deterioration of an image occurs.

In the contrast enhancement method of the lung region of the X-ray image provided according to one aspect of the present invention, the second X-ray image 704 including the lung region is input to the network 500, and the second X-ray image 704 is input to the network 500. generating a lung region image 705 divided into a lung region 705a and an outer lung region 705b from the image; and generating a corrected X-ray image (706 or 708) by stretching a histogram of brightness of a plurality of pixels constituting the lung region of the second X-ray image based on the lung region image. can

At this time, in the step of generating the corrected X-ray image, a first histogram showing the frequency of a value representing the brightness of a pixel for each of a plurality of pixels constituting the second X-ray image 704 ( 204); When each pixel of the second X-ray image corresponds to each pixel of the lung region image 705, the pixels of the part 704a corresponding to the lung region of the second X-ray image 2 generating a histogram 204a; determining a fifth value (v5) of the darkest pixel among the pixels of the portion; The determined fifth value among the values of the first histogram is mapped to a minimum value ('0', black) among values representing the brightness of a pixel, and corresponds to a fifth range (r5) between the minimum value and the fifth value. mapping the value of each pixel to the minimum value; linearly mapping values between the sixth value (v6) of the brightest pixel among the values of the first histogram and the fifth value to values between the sixth value (v6) and the minimum value; and generating the corrected X-ray image 706 having values mapped to each of a plurality of pixels of the second X-ray image.

Alternatively, in the method, between generating the lung region image and generating the corrected X-ray image, each pixel of the second X-ray image 704 is converted to the lung region image 705. The method may further include generating a second lung region image 707 cropped so that only the portion 704a corresponding to the lung region is included in the second X-ray image when corresponding to each pixel of . In this case, the generating of the corrected X-ray image 708 includes generating the corrected X-ray image by stretching a third histogram of brightness of a plurality of pixels constituting the second lung region image. may be a step. In this case, the generating of the corrected X-ray image may include generating the third histogram 207 for pixels of the second lung region image; determining a fifth value (v5) of a darkest pixel among pixels of the second lung region image; mapping the determined fifth value among values of the third histogram to a minimum value ('0', black) among values representing brightness of a pixel; linearly mapping values between a seventh value (v7) of the brightest pixel among values of the third histogram and the fifth value to values between the seventh value and the minimum value; and generating the corrected X-ray image having a value mapped to each of the pixels of the second lung region image.

At this time, the network projects the first CT scan information obtained by CT scan of the first scan object in one direction to generate a simulated X-ray image; extracting an air area from the first CT scan information; generating a simulated ground truth image by projecting only the air area among the first CT scan information in the one direction; and when the simulated X-ray image is input to the network including a neural network, pixels corresponding to the air area among the simulated ground truth images have a first value, and The simulated X-ray image is used as input information for learning and the simulated ground truth image is labeled so that pixels corresponding to the remaining area except for the air area output the lung area image having a second value It may be learned by a supervised learning method comprising the step of learning the network as .

At this time, the second X-ray image is generated by stretching the histogram of the first X-ray image output by the X-ray imaging device, or the second X-ray image is generated by the X-ray imaging device. It may be a printed image.

In this case, the generating of the second X-ray image by stretching the histogram of the first X-ray image may include a value representing the brightness of a pixel for each of a plurality of pixels constituting the first X-ray image. Generating a histogram showing the frequency for ; The second value, which is the boundary value of the first range r1 predetermined based on the first value of the brightest pixel among the plurality of pixels of the first X-ray image, is mapped to the maximum value among values representing the brightness of the pixel. and mapping a value of each pixel corresponding to the first range to the maximum value; The fourth value, which is the boundary value of the second range r2 predetermined based on the third value of the darkest pixel among the plurality of pixels constituting the first X-ray image, is set to the minimum value among the values representing the brightness of the pixel. mapping, and mapping a value of each pixel corresponding to the second range to the minimum value; And among the plurality of pixels constituting the first X-ray image, values representing the brightness of pixels in a third range (r3), which is a middle range excluding the first range and the second range, are between the maximum value and the minimum value. It may include linearly mapping to values of .

An apparatus for improving the contrast of a lung area of an X-ray image provided according to one aspect of the present invention includes a device interface unit capable of reading a computer-readable non-volatile recording medium; And a processing unit; may include. At this time, the non-volatile recording medium inputs the second X-ray image 704 including the lung area to the network 500, and in the second X-ray image, the lung area 705a and the outer lung area 705b ), twelfth instruction code for executing the step of generating a lung region image 705, and brightness of a plurality of pixels constituting the lung region of the second X-ray image based on the lung region image. A program including a 13th command code executing a step of generating a corrected X-ray image by stretching the histogram may be stored. At this time, the processing unit may be configured to generate the corrected X-ray image by reading and executing the twelfth command code and the thirteenth command code through the device interface unit.

At this time, in the step of generating the corrected X-ray image, a first histogram showing the frequency of a value representing the brightness of a pixel for each of a plurality of pixels constituting the second X-ray image 704 ( 204); When each pixel of the second X-ray image corresponds to each pixel of the lung region image 705, the pixels of the part 704a corresponding to the lung region of the second X-ray image 2 generating a histogram 204a; determining a fifth value (v5) of the darkest pixel among the pixels of the portion; The determined fifth value among the values of the first histogram is mapped to a minimum value ('0', black) among values representing the brightness of a pixel, and each pixel corresponding to a fifth range between the minimum value and the fifth value mapping the value of to the minimum value; linearly mapping values between the sixth value (v6) of the brightest pixel among the values of the first histogram and the fifth value to values between the sixth value (v6) and the minimum value; and generating the corrected X-ray image 706 having values mapped to each of a plurality of pixels of the second X-ray image.

Alternatively, when each pixel of the second X-ray image corresponds to each pixel of the lung region image 705 between the generating of the lung region image and the generating of the corrected X-ray image , generating a second lung region image 707 cropped so that only the portion 704a corresponding to the lung region is included in the second X-ray image. In this case, the generating of the corrected X-ray image includes generating the corrected X-ray image 708 by stretching a third histogram of brightness of a plurality of pixels constituting the second lung region image. may be a step. In this case, the generating of the corrected X-ray image may include generating a third histogram 207 of pixels of the second lung region image; determining a fifth value (v5) of a darkest pixel among pixels of the second lung region image; mapping the determined fifth value among values of the third histogram to a minimum value ('0', black) among values representing brightness of a pixel; linearly mapping values between a seventh value (v7) of the brightest pixel among values of the third histogram and the fifth value to values between the seventh value and the minimum value; and generating the corrected X-ray image having a value mapped to each of the pixels of the second lung region image.

At this time, in the non-volatile recording medium, projecting the first CT scan information obtained by CT scanning the first scan object in one direction to generate a simulated X-ray image; extracting an air area from the first CT scan information; generating a simulated ground truth image by projecting only the air area among the first CT scan information in the one direction; and when the simulated X-ray image is input to the network including a neural network, pixels corresponding to the air area among the simulated ground truth images have a first value, and The simulated X-ray image is used as input information for learning and the simulated ground truth image is labeled so that pixels corresponding to the remaining area except for the air area output the lung area image having a second value It may include a closed region learning command code for learning the network by a supervised learning method comprising the step of learning the network by doing. At this time, the processing unit may be configured to learn the network by reading and executing the closed region learning command code through the device interface unit.

At this time, the second X-ray image is generated by stretching the histogram of the first X-ray image output by the X-ray imaging device, or the second X-ray image is generated by the X-ray imaging device. It may be a printed image.

At this time, the non-volatile recording medium, generating a histogram representing the frequency of the value representing the brightness of the pixel for each of the plurality of pixels constituting the first X-ray image; The second value, which is the boundary value of the first range r1 predetermined based on the first value of the brightest pixel among the plurality of pixels of the first X-ray image, is mapped to the maximum value among values representing the brightness of the pixel. and mapping a value of each pixel corresponding to the first range to the maximum value; The fourth value, which is the boundary value of the second range r2 predetermined based on the third value of the darkest pixel among the plurality of pixels constituting the first X-ray image, is set to the minimum value among the values representing the brightness of the pixel. mapping, and mapping a value of each pixel corresponding to the second range to the minimum value; And among the plurality of pixels constituting the first X-ray image, values representing the brightness of pixels in a third range (r3), which is a middle range excluding the first range and the second range, are between the maximum value and the minimum value. A program including an 11th instruction code that executes the step of linearly mapping to the values of may be stored. At this time, the processing unit may be configured to generate the second X-ray image by reading and executing the eleventh command code through the device interface unit.

In the contrast enhancement method of the lung region of the X-ray image provided according to another aspect of the present invention, the second X-ray image 704 including the lung region is input to the network 500, and the second X-ray image 704 is input to the network 500. generating a lung region image 705 divided into a lung region 705a and an outer lung region 705b from the image; and enhancing contrast of the lung region of the second X-ray image based on the lung region image. At this time, the network projects the first CT scan information obtained by CT scan of the first scan object in one direction to generate a simulated X-ray image; extracting an air area from the first CT scan information; generating a simulated ground truth image by projecting only the air area among the first CT scan information in the one direction; and when the simulated X-ray image is input to the network including a neural network, pixels corresponding to the air area among the simulated ground truth images have a first value, and The simulated X-ray image as input information for learning and the simulated ground truth image as a label so that pixels corresponding to the remaining area except for the air area output a lung area image having a second value It may be learned by a supervised learning method comprising the step of learning the network by doing so.

At this time, the step of enhancing the contrast of the lung region of the second X-ray image based on the lung region image 705 may include setting each pixel of the second X-ray image to each of the lung region images 705. generating a second lung region image cropped so that only a portion 704a corresponding to the lung region is included in the second X-ray image when corresponding to pixels; and generating the corrected X-ray image by stretching a histogram of brightness of a plurality of pixels constituting the second lung region image.

In this case, the generating of the corrected X-ray image may include generating a third histogram 207 of pixels of the second lung region image; determining a fifth value (v5) of a darkest pixel among pixels of the second lung region image; mapping the determined fifth value among values of the third histogram to a minimum value ('0', black) among values representing brightness of a pixel; linearly mapping values between a seventh value (v7) of the brightest pixel among values of the third histogram and the fifth value to values between the seventh value and the minimum value; and generating the corrected X-ray image having a value mapped to each of the pixels of the second lung region image.

An apparatus for improving the contrast of a lung area of an X-ray image provided according to another aspect of the present invention includes a device interface unit capable of reading a computer-readable non-volatile recording medium; And a processing unit; may include. At this time, the non-volatile recording medium inputs the second X-ray image 704 including the lung area to the network 500, and in the second X-ray image, the lung area 705a and the outer lung area 705b ), and a 13th command code for executing a step of generating a lung region image 705 divided by ), and a 13th command code executing a step of enhancing the contrast of the lung region of the second X-ray image based on the lung region image. A program including a command code and a closed area learning command code for learning the network may be stored. At this time, the processing unit generates the corrected X-ray image by reading and executing the twelfth command code and the thirteenth command code through the device interface unit, and the closed region learning command code generating a simulated X-ray image by projecting first CT scan information obtained by CT scanning the scan object in one direction; extracting an air area from the first CT scan information; generating a simulated ground truth image by projecting only the air area among the first CT scan information in the one direction; and when the simulated X-ray image is input to the network including a neural network, pixels corresponding to the air area among the simulated ground truth images have a first value, and The simulated X-ray image as input information for learning and the simulated ground truth image as a label so that pixels corresponding to the remaining area except for the air area output a lung area image having a second value It may be a command code for learning the network by a supervised learning method comprising the step of learning the network by doing so.

At this time, the step of enhancing the contrast of the lung region of the second X-ray image based on the lung region image 705 may include setting each pixel of the second X-ray image to each of the lung region images 705. generating a second lung region image cropped so that only a portion 704a corresponding to the lung region is included in the second X-ray image when corresponding to pixels; and generating the corrected X-ray image by stretching a third histogram of brightness of a plurality of pixels constituting the second lung region image.

In this case, the generating of the corrected X-ray image may include generating a third histogram 207 of pixels of the second lung region image; determining a fifth value (v5) of a darkest pixel among pixels of the second lung region image; mapping the determined fifth value among values of the third histogram to a minimum value ('0', black) among values representing brightness of a pixel; linearly mapping values between a seventh value (v7) of the brightest pixel among values of the third histogram and the fifth value to values between the seventh value and the minimum value; and generating the corrected X-ray image having a value mapped to each of the pixels of the second lung region image.

A device interface unit capable of reading a computer-readable non-volatile recording medium according to another aspect of the present invention; And a processing unit; including, the contrast enhancement device of the lung region of the X-ray image may be provided. At this time, the non-volatile recording medium causes the processing unit to input the second X-ray image 704 including the lung area to the network 500 and extract the lung area 705a from the second X-ray image. twelfth command code for executing the step of generating specific lung region information, and correcting the brightness of a plurality of pixels constituting the lung region of the second X-ray image based on the lung region information. A program including a 13th command code to be executed is stored, and the processing unit reads and executes the 12th command code and the 13th command code through the device interface unit to generate the corrected X-ray image. has been

In this case, the step of correcting the brightness may include determining a fifth brightness value (v5) of a first pixel that is the darkest among pixels of the lung area in the second X-ray image; and stretching the histogram of the image of the lung region so that the brightness value of the first pixel has a first minimum brightness value that is darker than the fifth brightness value.

In this case, the first minimum brightness value may be the darkest value allowed in the second X-ray image.

In this case, the step of correcting the brightness may further include determining a sixth brightness value (v6) of a second pixel that is the brightest among pixels of the lung area in the second X-ray image. The stretching of the histogram may include pixels constituting the image of the lung region such that the fifth brightness value is mapped to the first minimum brightness value and the sixth brightness value is mapped to a predetermined first maximum brightness value. It may include a step of linearly correcting the brightness of them.

In this case, the first minimum brightness value may be the darkest value allowed in the second X-ray image. The first maximum brightness value may be equal to or brighter than the sixth brightness value.

In this case, the first maximum brightness value may be the brightest value allowed in the second X-ray image.

In this case, the step of correcting the brightness may include correcting the brightness of a plurality of pixels constituting an area other than the lung area in the second X-ray image based on the lung area information.

At this time, between the step of generating the lung region information and the step of correcting the brightness, a second lung region image 707 composed of only the part 704a corresponding to the lung region of the second X-ray image is generated. A generating step may be further included. The correcting of the brightness may include correcting the second lung region image by stretching a third histogram of brightness of a plurality of pixels constituting the second lung region image.

In this case, the step of correcting the brightness may include generating a third histogram 207 representing frequency counts of values representing the brightness of pixels of the second lung region image; determining a fifth brightness value (v5) of a first darkest pixel among pixels of the second lung region image; determining a sixth brightness value (v6) of a second brightest pixel among pixels of the second lung region image; stretching a histogram of the second closed region image so that the brightness value of the first pixel has a first minimum brightness value that is darker than the fifth brightness value; and a first value in which the fifth brightness value is mapped to a first minimum brightness value that is darker than the fifth brightness value, and the sixth brightness value is equal to or brighter than the sixth brightness value. The method may include linearly correcting brightness of pixels constituting the second lung region image so as to be mapped to a maximum brightness value. The first minimum brightness value may be the darkest value allowed in the second X-ray image, and the first maximum brightness value may be the brightest value allowed in the second X-ray image.

In this case, the network may be learned by the second computing device. generating a simulated X-ray image by projecting, by the second computing device, first CT scan information obtained by CT scanning a first scan target in one direction; extracting an air area from the first CT scan information; generating a simulated ground truth image by projecting only the air area among the first CT scan information in the one direction; and when the simulated X-ray image is input to the network including a neural network, pixels corresponding to the air area among the simulated ground truth images have a first value, and The simulated X-ray image is used as input information for learning and the simulated ground truth image is labeled so that pixels corresponding to the remaining area except for the air area output the lung area image having a second value It may be configured to execute the step of learning the network.

According to the present invention, it is possible to provide an X-ray image correction method for enhancing the contrast of a soft-tissue (soft tissue) region where contrast deterioration of an image occurs.

According to the present invention, in the step of stretching the histogram of the soft-tissue area, the contrast inside the lung can be enhanced without losing information inside the lung by stretching the histogram by setting the minimum value of the area inside the lung to the minimum value of all pixel values in the image. .

1 shows a process of obtaining an X-ray image using an X-ray imaging device.
2 illustrates the X-ray image and an image obtained by stretching a histogram of the X-ray image according to an exemplary embodiment.
3 is a conceptual diagram illustrating a method of generating a lung region image by inputting an X-ray image to a network provided according to an embodiment of the present invention.
4 illustrates a method for learning a network provided according to an embodiment of the present invention.
5 shows a set of learning data according to an embodiment of the present invention.
6 illustrates a learning method of a network provided according to an embodiment of the present invention.
7 is a flowchart of an X-ray image correction method according to an embodiment of the present invention.
8 is a diagram for explaining an X-ray image correction method according to an embodiment of the present invention.
9 shows histograms for explaining step S1200 according to an embodiment of the present invention.
10 is a flowchart of detailed steps of step S1200 according to an embodiment of the present invention.
11 is a flowchart of an X-ray image correction method according to another embodiment of the present invention.
12 is a diagram for explaining an X-ray image correction method according to another embodiment of the present invention.
13 is a histogram of a second lung region image according to another embodiment of the present invention.
14 is a flowchart of detailed steps of step S1202 according to another embodiment of the present invention.
15 is a diagram for explaining a method of generating a second X-ray image according to an embodiment of the present invention.
16 shows a histogram of a first X-ray image according to an embodiment of the present invention.
17 illustrates the structure and function of a computing device provided according to an embodiment of the present invention.
18 is a diagram explaining the concept of an X-ray image including noise.
19 illustrates a method of removing noise from an X-ray image according to a conventional supervised training technique.
20 illustrates a method of removing noise from an X-ray image according to another supervised learning technique.
21 illustrates a process of generating a label image for supervising a network that removes noise from an X-ray image according to an embodiment of the present invention.
22 is a conceptual diagram illustrating a process of removing noise from an X-ray image using a denoising network that has been trained according to an embodiment of the present invention.
23 is a conceptual diagram illustrating a method for learning the denoising network shown in FIG. 22.
24 is a conceptual diagram illustrating a method for generating data for network learning provided according to an embodiment of the present invention.
25 is a conceptual diagram illustrating a process of removing noise from an X-ray image using a denoising network that has been trained according to a comparative embodiment.
26 is a conceptual diagram illustrating a method of generating learning data for learning a denoising network according to a comparative embodiment.
27 is a conceptual diagram illustrating a method of learning a denoising network using training data prepared in FIG. 26 .
28 is a flowchart of an X-ray image pre-processing method provided according to an embodiment of the present invention.
29 is a flowchart illustrating a method for learning a network for removing noise provided according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described herein and may be implemented in various other forms. Terms used in this specification are intended to aid understanding of the embodiments, and are not intended to limit the scope of the present invention. Also, the singular forms used herein include the plural forms unless the phrases clearly dictate the contrary.

3 is a conceptual diagram illustrating a method of generating a lung region image by inputting an X-ray image to a network provided according to an embodiment of the present invention.

The network 500, which is an image conversion network provided according to an embodiment of the present invention, may be an information conversion unit including a neural network. The network 500 may have completed learning through supervised learning.

When an X-ray image 701 is input to the network 500, the network 500 may output a simulated lung region image 702 based on the X-ray image 701.

The X-ray image 701 may be an actual X-ray image obtained by taking an X-ray in a state in which one side (eg, the front) of the patient faces one direction.

Alternatively, the X-ray image 701 is a two-dimensional image obtained by projecting CT scan information obtained by CT (Computer Tomography) scanning of the patient in one direction, and may not actually be an image taken by an X-ray imaging device. .

In this case, the size of the X-ray image 701 and the lung region image 702 may be the same. For example, both the X-ray image 701 and the lung region image 702 may be composed of B1 * B1 pixels, respectively.

4 illustrates a method for learning a network provided according to an embodiment of the present invention.

The CT scan information 110 may be information obtained by CT scanning the body of a specific person. For example, the CT scan information 110 may be obtained by CT scanning the lungs and their surroundings of a healthy person without lung disease. At this time, the CT scan information 110 is 3D information.

A two-dimensional image projected on a plane orthogonal to the one direction can be obtained by projecting the CT scan information 110 in one direction. The first image 701 is a two-dimensional image obtained by projecting the CT scan information 110 in one direction. The first image 701 may be referred to as a 'simulated X-ray image'.

In the second image 703, only the air area among the CT scan information 110 is projected in the one direction, so that the first image 701 is obtained from the air area (ie, the lung area) and the remaining area excluding the air area (ie, the air area). It may be an image divided into an area outside of). In this case, the second image 703 may be referred to as a 'simulated ground truth image'.

The first image 701 is provided as input data for training of the network 500 . The network 500 may generate a first output image 702 by transforming the first image 701 . At this time, the network 500 may be trained to reduce an error between the first output image 702 and the second image 703 .

In this case, a pair of {first simulated X-ray image, first simulated ground truth image} may be referred to as a first set of training data.

At this time, in a preferred embodiment of the present invention, the network 500 can be learned using a total of M sets of learning data. That is, the network 500 can be completed by repeatedly executing a learning episode for supervising the network using one set of learning data for M sets of learning data.

At this time, the kth set of training data prepared to train the network 500 may be generated from the kth CT scan information obtained by CT scanning the body of the kth scan target (human). That is, the kth CT scan information (110[k]) can be obtained by CT scanning the kth scan object (k=1, ..., M, a natural number).

The kth set of training data may be {kth simulated X-ray image, kth simulated ground truth image}.

5 shows a set of learning data according to an embodiment of the present invention.

One set of training data may be {simulated X-ray image 701, simulated ground truth image 703}.

At this time, the histogram 201 is for the simulated X-ray image 701, and the histogram 203 is for the simulated ground truth image 703. At this time, the horizontal axis of the histogram represents the brightness value of a pixel (hereinafter, pixel value), and the vertical axis represents the number of pixels having the same read brightness value (hereinafter, frequency count) among pixels.

The closer to zero (0) in the histograms 201 and 203 means that the pixels of the corresponding images 701 and 703 are darker, and the larger the value of the positive integer, the brighter the pixels of the images 701 and 703 that can mean

Looking at the histogram 201, the simulated X-ray image 701 has more bright parts (ie, pixels) than the simulated ground truth image 703, so the value of a positive integer (eg, 4) increases as It can be seen that the frequency increases. On the other hand, looking at the histogram 203, the simulated ground truth image 703 has more dark parts (ie, pixels) than the simulated X-ray image 701, so the frequency increases as it approaches zero (0). can know that

6 illustrates a learning method of a network provided according to an embodiment of the present invention.

Hereinafter, it will be described with reference to FIGS. 4 to 6 together.

In step S1110, a simulated X-ray image (eg 701) may be generated by projecting the first CT scan information (eg, 110) obtained by CT scanning the first scan object in one direction.

In step S1120, an air area may be extracted from the first CT scan information 110. For example, a CT number (HU value) may be used to extract an air area. For example, when the tissue is bone, the bone has the highest X-ray absorptivity, so the CT number is +1000, and when the tissue is air, the air has the lowest X-ray absorptivity, and the CT number is -1000. . That is, in step S1120, an air area may be extracted by selecting an area having a CT number of -1000 among the first CT scan information 110.

In step S1130, a simulated ground truth image 703 may be generated by projecting only the air area among the first CT scan information 110 in the one direction.

In step S1140, when the simulated X-ray image 701 is input to the network 500 including the neural network, the pixels corresponding to the air area of the ground truth image 703 simulated by the network 500 are A closed area image having a first value (eg, a positive integer) and pixels corresponding to areas other than the air area of the simulated ground truth image 703 having a second value (eg, zero (0)) The network 500 may be trained to output 702 by using the simulated X-ray image 701 as input information for learning and the simulated ground truth image 703 as a label.

7 is a flowchart of an X-ray image correction method according to an embodiment of the present invention.

8 is a diagram for explaining an X-ray image correction method according to an embodiment of the present invention.

Hereinafter, it will be described with reference to FIGS. 7 and 8 together.

In step S1100, the second X-ray image 704 including the lung area is input to the network, and the second X-ray image 704 is divided into a lung area 705a and an outer lung area 705b. A lung region image 705 may be generated.

The network of step S1100 may be one learned through steps S1110 to S1140 described above. Steps S1110 to S1140 are steps of a learning episode for supervising a network using a set of learning data, and learning of the network can be completed by repeatedly executing these learning episodes a plurality of times.

In step S1200, the histogram of the brightness of a plurality of pixels constituting the lung region of the second X-ray image 704 is stretched based on the generated lung region image 705, and the corrected X-ray Image 706 may be generated.

9 shows histograms for explaining step S1200 according to an embodiment of the present invention.

9(a) shows a second histogram 204a of a portion 704a corresponding to the lung region among the second X-ray images 704, and FIG. 9(b) shows a second histogram 204a. A first histogram 204 of the image 704 is shown.

The horizontal axis of the histograms 204 and 204a represents pixel values, and the vertical axis represents the frequency.

10 is a flowchart of detailed steps of step S1200 according to an embodiment of the present invention.

Hereinafter, it will be described with reference to FIGS. 8 to 10 together.

Step S1200 may include steps S1210 to S1260.

In step S1210, a first histogram 204 representing a frequency of a value representing brightness of a pixel for each of a plurality of pixels constituting the second X-ray image 704 may be generated.

In step S1220, when each pixel of the second X-ray image 704 corresponds to each pixel of the lung region image 705, it corresponds to the lung region of the second X-ray image 704. A second histogram 204a for the pixels of the portion 704a may be generated.

In step S1230, a fifth value v5 of the darkest pixel among the pixels of the portion 704a may be determined.

In step S1240, the fifth value (v5) among the values of the first histogram 204 is mapped to the minimum value ('0', black) among the values representing the brightness of the pixel, and from the minimum value ('0') A value of each pixel corresponding to the fifth range r5 between the fifth value v5 may be mapped to the minimum value '0'.

In step S1250, values between the sixth value v6 and the fifth value v5 of the brightest pixel among the values of the first histogram 204 are converted to values between the sixth value v6 and the minimum value. It can be mapped linearly. In this case, the sixth value may be, for example, the maximum value ('255', white) among values representing pixels, or may be a value smaller than the maximum value. In a preferred embodiment, the sixth value may be the maximum value.

In step S1260, the corrected X-ray image 706 having values mapped to each of a plurality of pixels of the second X-ray image 704 may be generated.

At this time, for example, the second X-ray image 704 is generated by stretching the histogram of the first X-ray image output by the X-ray imaging device, or the second X-ray image 704 is -ray It may be the image itself output by the recording device.

11 is a flowchart of an X-ray image correction method according to another embodiment of the present invention.

12 is a diagram for explaining an X-ray image correction method according to another embodiment of the present invention.

Hereinafter, it will be described with reference to FIGS. 11 and 12 together.

In step S1100, the second X-ray image 704 including the lung area is input to the network, and the second X-ray image 704 is divided into a lung area 705a and an outer lung area 705b. A lung region image 705 may be generated.

The network of step S1100 may be one learned through steps S1110 to S1140 described above. Steps S1110 to S1140 are steps of a learning episode for supervising a network using a set of learning data, and learning of the network can be completed by repeatedly executing these learning episodes a plurality of times.

In step S1200, the histogram of the brightness of a plurality of pixels constituting the lung region of the second X-ray image 704 is stretched based on the generated lung region image 705, and the corrected X-ray Image 708 may be created.

At this time, step S1200 may include step S1201 and step S1202.

In step S1201, when each pixel of the second X-ray image 704 corresponds to each pixel of the lung region image 705, it corresponds to the lung region of the second X-ray image 704. A cropped second lung region image 707 may be generated to include only the part 704a to be removed.

In step S1202, the corrected X-ray image 708 may be generated by stretching the histogram 207 of brightness of a plurality of pixels constituting the second lung region image 707.

13 is a histogram of a second lung region image according to another embodiment of the present invention.

The horizontal axis of the histogram 207 represents the pixel value, and the vertical axis represents the frequency.

14 is a flowchart of detailed steps of step S1202 according to another embodiment of the present invention.

Hereinafter, it will be described with reference to FIGS. 12 to 14 together.

Step S1202 may include steps S1310 to S1350.

In step S1310, a third histogram 207 for pixels of the second lung region image 707 may be generated.

In operation S1320, a fifth value v5 of the darkest pixel among pixels of the second lung region image 707 may be determined.

In step S1330, the determined fifth value v5 among values of the third histogram 207 may be mapped to a minimum value ('0', black) among values representing brightness of a pixel.

In step S1340, the values between the seventh value v7 of the brightest pixel among the values of the third histogram 207 and the fifth value are linearly converted into values between the seventh value v7 and the minimum value. can be mapped. In this case, the seventh value v7 may be, for example, the maximum value ('255', white) among values representing pixels or may be a value smaller than the maximum value.

In step S1350, the corrected X-ray image 708 having values mapped to each of the pixels of the second lung region image 707 may be generated.

Hereinafter, referring to FIGS. 2, 15, and 16 together, when the second X-ray image 704 is generated by stretching the histogram of the first X-ray image output by the X-ray imaging device, the first 2 A method of generating an X-ray image 704 will be described.

15 is a diagram for explaining a method of generating a second X-ray image according to an embodiment of the present invention.

16 shows a histogram of a first X-ray image according to an embodiment of the present invention.

The horizontal axis of the histogram 24 represents the pixel value, and the vertical axis represents the frequency.

Step S1000 may be performed before step S1100 described above.

In step S1000, second X-ray images 72 and 704 may be generated by stretching the histogram of the first X-ray image 71.

Step S1000 may include steps S1010 to S1040.

In step S1010, a histogram 24 representing a frequency of a value representing brightness of a pixel for each of a plurality of pixels constituting the first X-ray image 71 may be generated.

In step S1020, a first range r1 (that is, a predetermined first range r1) based on the first value v1 of the brightest pixel among the plurality of pixels of the first X-ray image 71 The second value v2, which is the boundary value of 1 white outlier), is mapped to the maximum value ('255'), and the value of each pixel corresponding to the first range r1 (first white outlier) is mapped to the maximum value ('255'). It can be mapped to the maximum value ('255'). In this case, the white outlier may mean an outlier of a predetermined % based on the maximum brightness. For example, the predetermined first white outlier is a 0.5% outlier and may mean bright pixels belonging to an upper 0 to 0.5% range.

In step S1030, a second range r2 (ie, a predetermined second range r2) based on the third value v3 of the darkest pixel among the plurality of pixels constituting the first X-ray image 71 The fourth value (v4), which is the boundary value of the first black outlier), is mapped to the minimum value ('0'), and the value of each pixel corresponding to the second range (r2) (first black outlier) is It can be mapped to the minimum value ('0'). In this case, the black outlier may mean an outlier of a predetermined % based on the minimum brightness. For example, the predetermined first black outlier may refer to dark pixels belonging to a lower 0 to 0.5% range.

In step S1040, the first range r1 (first white outlier) and the second range r2 (first black out of the plurality of pixels constituting the first X-ray image 71) Values representing the brightness of pixels in the third range (r3), which is the middle range excluding outliers), may be linearly mapped to values between the maximum value ('255') and the minimum value ('0').

17 illustrates the structure and function of a computing device provided according to an embodiment of the present invention.

In the present invention, the computing device may be referred to as a device for enhancing the contrast of the lung region of an X-ray image.

The computing device 1 may include a device interface unit 3 and a processing unit 4 capable of reading a computer-readable non-volatile recording medium 2 .

The non-volatile recording medium 2 includes the 11th command code for executing the step (S1000) including the above steps (S1010) to (S1040), the 12th command code for executing the step (S1100), and the step ( The 13th instruction code for executing step S1200 including steps S1210 to S1260, the 14th instruction code for executing steps S1201 and S1202, and steps S1110 to S1140. A program including a closed area learning command code to be executed may be stored.

The processing unit 4 may be configured to generate a second X-ray image by reading and executing the 11th command code through the device interface unit 3 . Also, the processing unit 4 may be configured to generate a closed region image by reading and executing the twelfth command code through the device interface unit 3 . In addition, the processing unit 4 may be configured to generate a corrected X-ray image by reading and executing the thirteenth command code through the device interface unit 3. In addition, the processing unit 4 reads and executes the 14th command code through the device interface unit 3 to generate a corrected X-ray image that is a corrected second lung region image according to another embodiment. may have been In addition, the processing unit 4 may be configured to learn the network by reading and executing the closed region learning command code through the device interface unit 3 .

The noise present in the image (ex: 704) input to the network 500 is preliminarily removed by using a denoising network (f _θ ) 111 learned by a noise removal method of an X-ray image, which is an application example to be described later. can be removed That is, an image (ex: 704) is input to the denoising network (f _θ ) 111, and the noise-removed image output from the denoising network (f _θ ) 111 is input to the network 500. can do.

[Application Example - X-ray image noise removal method that can be applied to the present invention described above]

An X-ray photographed image may be stored and provided as a digital image composed of pixels in a matrix form. For example, an X-ray photographed image may be composed of R0 * C0 pixels consisting of R0 rows and C0 columns. When the X-ray image is digitally captured, the R0 * C0 pixels may correspond to R0 * C0 different image sensor elements that detect X-rays. Alternatively, when the X-ray image is captured by a film, it may be reconstructed into a digital image consisting of the R0 * C0 number of pixels by scanning the developed film.

18 is a diagram explaining the concept of an X-ray image including noise.

An X-ray image (x+n) 600 captured by an X-ray imaging device or synthesized from CT information may be composed of a real image (x) 601 and noise (n) 602. .

19 illustrates a method of removing noise from an X-ray image according to a conventional supervised training technique.

The X-ray image (x+n) 600 may be input to the denoising network (f _θ ) 111 (hereinafter referred to simply as the network (f _θ )). The operation goal of the network (f _θ ) 111 is to output a post-processed image (f _θ (x + n)) 603, which is an image from which noise is removed from the X-ray image (x + n) 600. will be. To this end, the parameter θ constituting the network (f _θ ) 111 needs to be optimized. To this end, to minimize the loss function L between the real image (x) 601 and the post-processed image (f _θ (x + n)) 603 for the X-ray image (x + n) 600 Optimize the parameter θ.

However, in the technique shown in FIG. 19, the real image (x) 601, which is a ground truth image for the X-ray image (x+n) 600, must be prepared in advance. ) 601 itself may be very difficult or impossible.

20 illustrates a method of removing noise from an X-ray image according to another supervised learning technique.

The X-ray image (x+n) 600 shown in FIG. 19 is rewritten as an X-ray image (x+n ₁ ) in FIG. 20 .

In the technique of FIG. 20 , instead of using the real image (x) 601 as a label for supervised learning for an X-ray image (x+n ₁ ) 605 containing noise, another label image (x +n ₂ ) 604 is used as a label for the X-ray image (x+n ₁ ) 605 . In the network learning method described with reference to FIG. 20 , the loss function L between the another label image (x+n ₂ ) 604 and the post-processed image (f _θ (x+n ₁ )) 603 is minimized. Optimize the parameter θ.

The loss function L can be defined as [Equation 1].

[Equation 1]

Here, a real image (x) from which noise n ₂ is removed from the another label image (x+n ₂ ) 604 and a real image from which noise n ₁ is removed from the X-ray image (x+n ₁ ) 605 (x) must be equal to each other.

At this time, if ① n ₁ and n ₂ are independent iid, ② Ε[n ₁ ] = Ε[n ₂ ] = 0, ③ n ₁ and n ₂ are symmetric, and ④ satisfy the L2 norm loss, The technique exhibits substantially the same effect as the technique of FIG. 19 . That is, if the above condition is satisfied, instead of the _real image (x) 601 as shown in FIG. 19 , the another label image (x+n ₂ ) 604 can be used. Even if the network (f _θ ) 111 is trained using the label image (x+n ₂ ) 604, the same learning effect can be obtained. However, despite this theory, for the technique presented in FIG. 20, it is necessary to prepare another label image (x+n ₂ ) 604 for the X-ray image (x+n ₁ ) 605. Preparing another label image (x+n ₂ ) 604 can also be very difficult.

Therefore, it is necessary to provide an effective technique for generating an image that can replace the real image (x) 601 for learning the network (f _θ ) 111 and learning the network (f _θ ) 111. there is

<How to generate labels for supervised learning>

21 illustrates a process of generating a label image for supervising a network that removes noise from an X-ray image according to an embodiment of the present invention.

An X-ray image (x+n) 600 including noise may be composed of a plurality of pixels arranged in a matrix form. 21 shows four pixels adjacent to each other in the row and column directions.

At this time, based on the information of the pixels 61 of the first group among the four adjacent pixels of the X-ray image (x + n) 600, the X-ray image (x + n) 600 ) to generate the X-ray image (x+n ₁ ) 605 of FIG. 21 .

And based on the information of the pixels 62 of the second group among the four adjacent pixels of the X-ray image (x + n) 600, the X-ray image (x + n) 600 The label image (x+n ₂ ) 604 of FIG. 21 may be generated by downsampling .

In one embodiment, the pixels 61 of the first group and the pixels 62 of the second group may not share the same pixel.

The generated X-ray image (x+n ₁ ) 605 may be used as training data for training the network 111, and the generated label image (x+n ₂ ) 604 may be used as data for training the network 111. It can be used as a label for supervised learning for the -ray image (x+n ₁ ) 605. At this time, it can be seen that ① n ₁ and n ₂ are independent iids, ② Ε[n ₁ ] = Ε[n ₂ ] = 0, ③ n ₁ and n ₂ are symmetric, and ④ L2 norm loss is satisfied.

The resolution of the generated X-ray image (x+n ₁ ) 605 and the generated label image (x+n ₂ ) 604 is the original image, the X-ray image (x+n) 600 ) can be understood as being smaller than the resolution of However, instead of losing the resolution of the X-ray image (x+n) 600, which is the original image, the learning X-ray image (x+n ₁ ) 605 for supervised learning and the label image (x+ n ₂ ) (604).

Hereinafter, in the present specification, the X-ray image (x+n ₁ ) 605 and the label image (x+n ₂ ) 604 are referred to as a first sub-image 605 and a second sub-image 604, respectively. Alternatively, it may be referred to as a first induction image 605 and a second induction image 605.

<X-ray denoising method and denoising network training method according to a preferred embodiment of the present invention>

22 is a conceptual diagram illustrating a process of removing noise from an X-ray image using a denoising network that has been trained according to an embodiment of the present invention.

The prepared X-ray image 700 is converted into a first image 710 through a downsampling process.

The converted first image 710 may be input to the first network 111, which is the denoising network for which learning has been completed.

The first network 111 may have a structure in which data of size N1 * N1 is input and data of size N1 * N1 is output.

The first network 111 may generate and output a second image 720 from the first image 710 .

At this time, the X-ray image 700 is a digital image, and its size may be R0*C0. That is, the X-ray image 700 may be composed of R0 * C0 pixels.

The sizes of the first image 710 and the second image 720 are the same, and both may be N1*N1. That is, both the first image 710 and the second image 720 may be composed of N1*N1 pixels, respectively. In this case, R0 >= N1 and C0 >= N1.

The first network 111 may convert an input image composed of N1*N1 pixels to generate an output image composed of N1*N1 pixels. Also, the first network 111 may be trained to generate an output image from which noise is removed from an input image. The first network 111 may include a neural network capable of learning, and the first network 111 may have already been sufficiently learned.

23 is a conceptual diagram illustrating a method for learning the denoising network shown in FIG. 22.

The first network 111 of FIG. 23 may be supervised. The input data for learning is a learning X-ray image (x+n ₁ ) 805 containing noise, and the supervised learning label corresponding to the X-ray image (x+n 1 ) 805 is a label image (x+n ₁ ). n ₂ ) (804). The first network 111 may generate a post-processing image (f _θ (x+n ₁ )) 803 from an X-ray image (x+n ₁ ) 805 for training. Here, the label for supervised learning may mean data used for calculating the loss function defined as in [Equation 1].

The parameter θ constituting the first network 111 may be optimized to minimize the loss function L (= loss ) defined as in [Equation 1].

By repeating the supervised learning using a plurality of different learning X-ray images (x+n ₁ ) 805 and a plurality of different corresponding label images (x+n ₂ ) 804, the first The learning state of the network 111 can be improved.

Here, a real image (x) from which noise n ₂ is removed from the label image (x+n ₂ ) 804 and a real image from which noise n ₁ is removed from the training X-ray image (x+n ₁ ) 805 ( x) must be equal to each other.

In this case, ① n ₁ and n ₂ are independent iids, ② Ε[n ₁ ] = Ε[n ₂ ] = 0, ③ n ₁ and n ₂ are symmetric, and ④ L2 norm loss must be satisfied. do.

Helps prepare the plurality of different learning X-ray images (x+n ₁ ) 805 and the corresponding plurality of different label images (x+n ₂ ) 804 to meet these conditions. 24 explains.

24 is a conceptual diagram illustrating a method for generating data for network learning provided according to an embodiment of the present invention.

In order to prepare a pair of the X-ray images (x+n ₁ ) 805 and a corresponding label image (x+n ₂ ) 804, an arbitrary X-ray image 80 may be prepared. there is.

Next, a first training image 800 may be generated by converting (ex: downsampling) the X-ray image 80 .

23 by downsampling the first training image (x+n) 800 based on the information of the pixels of the first group among the four adjacent pixels of the first training image 800. An X-ray image (x+n ₁ ) 805 of may be generated. The X-ray image (x+n ₁ ) 805 of FIG. 23 may include, for example, representative values (ex: average values) of the pixels of the first group.

And by downsampling the first training image (x+n) 800 based on the information of the pixels of the second group among the four adjacent pixels of the first training image (x+n) 800. The label image (x+n ₂ ) 804 of FIG. 23 may be generated. The label image (x+n ₂ ) 804 of FIG. 23 may include, for example, a representative value (ex: average value) of the pixels of the second group.

In one embodiment, the pixels of the first group and the pixels of the second group may not share the same pixel. At this time, it can be considered that ① n ₁ and n ₂ are independent iid, ② Ε[n ₁ ] = Ε[n ₂ ] = 0, ③ n ₁ and n ₂ are symmetric, and ④ L2 norm loss is satisfied. .

In one embodiment, the four adjacent pixels may be adjacent to each other in the form of a 2*2 matrix.

In another embodiment, the four adjacent pixels may be pixels in a 1*4 matrix form adjacent to each other or pixels in a 4*1 matrix form adjacent to each other.

In this case, the pixels of the first group may be composed of selected two of the four pixels, and the pixels of the second group may be composed of the remaining two of the four pixels.

Alternatively, the pixels of the first group may be composed of one selected from among the four pixels, and the pixels of the second group may be composed of the remaining three of the four pixels.

According to a preferred embodiment of the present invention, as shown in the example of FIG. 24, N2 should be twice as large as N1.

And in the example of FIG. 24 , R2 may be greater than or equal to N1 and C1 may be greater than or equal to N1.

A plurality of pairs of X-ray images (x+n ₁ ) 805 and corresponding label images (x+n ₂ ) 804 may be generated based on the plurality of X-ray images 80, The first network 111 can be trained using these.

The X-ray image 700 of FIG. 22 and the X-ray image 80 of FIG. 24 may be X-ray dicom images. For example, R0 and C0 may each have a value of 2,000. In FIG. 22, the size of data input to the first network 111 is N1 * N1, where N1 may be 256. And in FIG. 24, N2 may be 512 in the size N2 * N2 of the data of the first training image 800.

<X-ray denoising method and denoising network training method according to a comparative embodiment>

The comparative embodiment to be described from now on is to compare the advantages of the above-described preferred embodiment. It is not admitted that this comparative example has already been published by others at the time of filing this patent.

25 is a conceptual diagram illustrating a process of removing noise from an X-ray image using a denoising network that has been trained according to a comparative embodiment.

The diagram presented in FIG. 25 is different from FIG. 22 only in that the first network 111 is replaced with the second network 112 as a denoising network.

A method of generating learning data to be used for learning the second network 112 according to the comparative example may be presented as shown in FIG. 26 .

26 is a conceptual diagram illustrating a method of generating learning data for learning a denoising network according to a comparative embodiment.

First, an arbitrary X-ray image 80 may be prepared.

Then, a second training image 900 may be generated by converting (ex: downsampling) the X-ray image 80 .

Next, X-ray downsamples the second training image (x+n) 900 based on the information of the pixels of the first group among the four adjacent pixels of the second training image 900. An image (x+n ₁ ) 905 may be created. The X-ray image (x+n ₁ ) 905 may include, for example, a representative value (ex: average value) of the pixels of the first group.

And by downsampling the second training image (x+n) 900 based on the information of the pixels of the second group among the four adjacent pixels of the second training image (x+n) 900. A label image (x+n ₂ ) 904 may be generated. The label image (x+n ₂ ) 904 may be formed of, for example, a representative value (ex: average value) of the pixels of the second group.

At this time, the size of the second learning image (x+n) 900 is N1 * N1, and the X-ray image (x+n ₁ ) 905 and the label image (x+n ₂ ) 904 The size of each may be {N1}/2 * {N1}/2.

27 is a conceptual diagram illustrating a method of learning a denoising network using training data prepared in FIG. 26 .

The second network 112 of FIG. 27 may be supervised. As shown in FIG. 22, the second network 112 may generate N1*N1-sized output data from N1*N1-sized input data.

The input data for learning is an X-ray image (x+n ₁ ) 905 for learning containing noise, and a label for supervised learning corresponding to the X-ray image (x+n 1 ) 905 is a label image (x+n ₁ ). n ₂ ) (904). The second network 112 may generate a post-processing image (f _θ (x+n ₁ )) 903 from an X-ray image (x+n ₁ ) 905 for training.

The parameter θ constituting the second network 112 may be optimized to minimize the loss function L (= loss ) defined as in [Equation 1].

By repeating the supervised learning using a plurality of different learning X-ray images (x+n ₁ ) 905 and a plurality of different corresponding label images (x+n ₂ ) 904, the second The learning state of the network 112 may be improved.

Here, since the sizes of the X-ray image (x+n ₁ ) 905 and the label image (x+n ₂ ) 904 are {N1}/2 * {N1}/2, respectively, the second network ( 112) is learned based on images of size {N1}/2 * {N1}/2.

To elaborate, the second network 112 learned by the method of FIG. 27 is used to remove noise by the method shown in FIG. 25, and the first network 111 learned by the method of FIG. 23 is The method presented in Fig. 22 is used to remove noise. At this time, both the noise removal method according to FIG. 22, which is an embodiment of the present invention, and the noise removal method according to FIG. 25, which is a comparative embodiment, input an X-ray image of size N1 * N1 to the denoising networks 111 and 112. At this time, the first network 111 learned according to an embodiment of the present invention is learned by using images of size N1 * N1, whereas the second network 112 learned according to the comparative embodiment is { Since it is learned from images of size N1}/2 * {N1}/2, the method of FIG. 22 shows better noise removal performance than the method of FIG. 25 .

The network 111 shown in FIG. 22 and the network 112 shown in FIG. 25 may be operable regardless of the size of the image used for input and output, respectively, and the size of the output image is the same as the input image. It may be a network designed to be equal to the size of .

28 is a flowchart of an X-ray image pre-processing method provided according to an embodiment of the present invention.

In step S110, the X-ray image 700 may be converted into a first image 710.

In step S120 , the first image 710 may be input to the first network 111 and the second image 720 may be obtained from the first network 111 .

At this time, the size of the X-ray image 700 is R0*C0, and both the sizes of the first image 710 and the second image 720 are N1*N1 (however, R0 >= N1, C0 > = N1).

29 is a flowchart illustrating a method for learning a network for removing noise provided according to an embodiment of the present invention.

In step S210, a learning image 80 selected from a set of X-ray images for learning may be converted into a first learning image 800.

In step S220, a first sub-image 805 may be generated by downsampling the first training image 800.

In step S230, a second sub-image 804 may be generated by down-sampling the first training image 800.

In step S240, the first network 111 reduces the error between the output image 803 obtained by converting the first sub-image 805 and the second sub-image 804 ( 111) can be supervised.

At this time, the size of the first training image 800 is N2*N2, and the sizes of the first sub-image 805, the output image 803, and the second sub-image 804 are all N1*N1. (However, N2 = 2 * N1).

28 and 29, and the X-ray image preprocessing method described herein in relation thereto may also be referred to as an X2X noise removal method.

Referring again to FIG. 17, the non-volatile recording medium 2 includes a program including a first command code for executing step S110 and a second command code for executing step S120. may be storing

The processing unit 4 may be configured to obtain the second image by reading and executing the first command code and the second command code through the device interface unit 3 .

The program may further include learning instruction codes for executing the steps S210, S220, S230, and S240.

Further, the processing unit 4 may be configured to learn the first network by reading and executing the learning command code through the device interface unit 3 .

<Test case of the present invention>

In X2X, each CXR for training was resized (e.g. 512 x 512) and 4 CXRs (c ₁ , c ₂ , c ₃ , c ₄ ) of the same size (e.g. 256 x 256) with independent noise. ) was downsampled. Among them, two pairs of CXRs were randomly selected, averaged, and used as input data (ex: 605 in FIG. 21 ) and labels (ex: 604 in FIG. 21 ) for learning (eg, c ₁ / 2 + c ₄ /2, and c ₂ /2 + c ₃ /2 for labels). Using these input and label CXRs, we trained a denoising network 111 using a noise-to-noise algorithm (see Fig. 20). Therefore, there is no need to prepare a clean CXR (i.e. noise-free CXR) for training. In the test, the four downsampled CXRs were averaged and fed into a denoising network.

In the experiments, we first used X2X and other self-supervised methods (Noise2Void (N2V), Noise2Same (N2S) and Neighbor2Neighbor (Ne2Ne)) to test publicly available CXR datasets (10,000 CXRs from VinBigData, 9,000 CXRs for training). and performed denoising on 1,000 CXRs for evaluation). In addition, synthetic Gaussian noise was added to the data set, and the denoising performance (i.e., PSNR) was measured using X2X and other denoising methods (supervised: Noise2Clean (N2CL) and self-supervised method). Second, before and after applying X2X to CXR training and testing, we trained two deep neural networks for CXR abnormality classification and measured their classification performance (i.e., AUC, sensitivity, and specificity). Here, the abnormality is an abnormality including consolidation, ground-glass opacity, nodules or mass. In addition, the CXRs are 5,000 different CXRs obtained from the noise control data set of VigBigData, of which 4,500 are training data and 500 are evaluation data.

After denoising the CXR data set, X2X successfully improved the image, displaying little or no structure-dependent signal. In synthetic noise experiments, the average PSNR of X2X (55.05 ± 3.75) was significantly higher than that of the other self-supervised methods (N2V: 46.64 ± 4.65, p <0.001; N2S: 47.06 ± 5.01, p < 0.001, and Ne2Ne: 51.63 ± 51.63 ± 0.001). 4.19, p < 0.001) and showed similar results to those of the supervised method (N2CL: 55.95 ㅁ 3.88). In addition, the anomaly classification performance improved after noise removal using X2X (AUC: from 0.9637 (95%CI: 0.9627 - 0.9647) to 0.9741 (0.9632 - 0.9650), p <0.001; sensitivity: from 0.9637 (95%CI: specificity: from 0.9637 (95% CI: 0.9627 - 0.9647) to 0.9741 (0.9632 - 0.9650), p < 0.001).

In this test case, it can be confirmed that the new self-supervised denoising method for CXR anomaly classification not only improves the PSNR of CXR, but also improves the classification performance.

Using the above-described embodiments of the present invention, those belonging to the technical field of the present invention will be able to easily implement various changes and modifications without departing from the essential characteristics of the present invention. The content of each claim of the claims may be combined with other claims without reference relationship within the scope understandable through this specification.

Claims (14)

A second X-ray image 704 including the lung region is input to the network 500, and a lung region image 705 divided into a lung region 705a and an outer lung region 705b in the second X-ray image ) generating; and
generating a corrected X-ray image (706 or 708) by stretching a histogram of brightness of a plurality of pixels constituting the lung region of the second X-ray image based on the lung region image;
Including,
The step of generating the corrected X-ray image,
generating a first histogram 204 representing a frequency of a value indicating brightness of a pixel for each of a plurality of pixels constituting the second X-ray image 704;
When each pixel of the second X-ray image corresponds to each pixel of the lung region image 705, the pixels of the part 704a corresponding to the lung region of the second X-ray image 2 generating a histogram 204a;
determining a fifth value (v5) of the darkest pixel among the pixels of the portion;
The determined fifth value among the values of the first histogram is mapped to a minimum value ('0', black) among values representing the brightness of a pixel, and corresponds to a fifth range (r5) between the minimum value and the fifth value. mapping the value of each pixel to the minimum value;
linearly mapping values between the sixth value (v6) of the brightest pixel among the values of the first histogram and the fifth value to values between the sixth value (v6) and the minimum value; and
generating the corrected X-ray image 706 having values mapped to each of a plurality of pixels of the second X-ray image;
including,
Contrast enhancement method of lung area in X-ray image. According to claim 1,
the network,
generating a simulated X-ray image by projecting first CT scan information obtained by CT scanning the first scan target in one direction;
extracting an air area from the first CT scan information;
generating a simulated ground truth image by projecting only the air area among the first CT scan information in the one direction; and
When the simulated X-ray image is input to the network including a neural network, pixels corresponding to the air area in the simulated ground truth image have a first value, and the simulated ground truth image Pixels corresponding to the remaining areas except for the air area use the simulated X-ray image as input information for learning and the simulated ground truth image as a label so as to output the lung area image having a second value learning the network by
Which is learned by a supervised learning method comprising
Contrast enhancement method of lung area in X-ray image. delete A second X-ray image 704 including the lung region is input to the network 500, and a lung region image 705 divided into a lung region 705a and an outer lung region 705b in the second X-ray image ) generating;
When each pixel of the second X-ray image 704 corresponds to each pixel of the lung region image 705, only a portion 704a corresponding to the lung region is included in the second X-ray image. generating a cropped second lung region image 707; and
generating a corrected X-ray image (706 or 708) by stretching a third histogram of brightness of a plurality of pixels constituting the second lung region image;
Including,
The step of generating the corrected X-ray image,
generating the third histogram 207 for pixels of the second lung region image;
determining a fifth value (v5) of a darkest pixel among pixels of the second lung region image;
mapping the determined fifth value among values of the third histogram to a minimum value ('0', black) among values representing brightness of a pixel;
linearly mapping values between a seventh value (v7) of the brightest pixel among values of the third histogram and the fifth value to values between the seventh value and the minimum value; and
generating the corrected X-ray image having a value mapped to each of the pixels of the second lung region image;
including,
Contrast enhancement method of lung area in X-ray image. delete A second X-ray image 704 including the lung region is input to the network 500, and a lung region image 705 divided into a lung region 705a and an outer lung region 705b in the second X-ray image ) generating; and
generating a corrected X-ray image (706 or 708) by stretching a histogram of brightness of a plurality of pixels constituting the lung region of the second X-ray image based on the lung region image;
Including,
The second X-ray image is generated by stretching the histogram of the first X-ray image output by the X-ray imaging device,
Generating the second X-ray image by stretching the histogram of the first X-ray image,
generating a histogram representing a frequency of a value representing brightness of a pixel for each of a plurality of pixels constituting the first X-ray image;
The second value, which is the boundary value of the first range r1 predetermined based on the first value of the brightest pixel among the plurality of pixels of the first X-ray image, is mapped to the maximum value among values representing the brightness of the pixel. and mapping a value of each pixel corresponding to the first range to the maximum value;
The fourth value, which is the boundary value of the second range r2 predetermined based on the third value of the darkest pixel among the plurality of pixels constituting the first X-ray image, is set to the minimum value among the values representing the brightness of the pixel. mapping, and mapping a value of each pixel corresponding to the second range to the minimum value; and
Among the plurality of pixels constituting the first X-ray image, values representing the brightness of pixels in the third range (r3), which is the middle range excluding the first range and the second range, are between the maximum value and the minimum value. mapping linearly to values;
including,
Contrast enhancement method of lung area in X-ray image. a device interface unit capable of reading a computer-readable non-volatile recording medium; And a processing unit; including,
The non-volatile recording medium causes the processing unit to:
A second X-ray image 704 including the lung region is input to the network 500 to execute a step of generating lung region information specifying the lung region 705a in the second X-ray image. 12 command code, and
Thirteenth instruction code for executing a step of correcting the brightness of a plurality of pixels constituting the lung area of the second X-ray image based on the lung area information.
It stores programs that include
The processing unit generates the corrected X-ray image by reading and executing the twelfth command code and the thirteenth command code through the device interface unit,
In the step of correcting the brightness,
determining a fifth brightness value (v5) of a first darkest pixel among pixels of the lung area in the second X-ray image; and
stretching the histogram of the image of the lung region so that the brightness value of the first pixel has a first minimum brightness value that is darker than the fifth brightness value;
including,
Contrast enhancement device for lung area in X-ray images. delete a device interface unit capable of reading a computer-readable non-volatile recording medium; And a processing unit; including,
The non-volatile recording medium causes the processing unit to:
A second X-ray image 704 including the lung region is input to the network 500 to execute a step of generating lung region information specifying the lung region 705a in the second X-ray image. 12 command code, and
Thirteenth instruction code for executing a step of correcting the brightness of a plurality of pixels constituting the lung area of the second X-ray image based on the lung area information.
It stores programs that include
The processing unit generates the corrected X-ray image by reading and executing the twelfth command code and the thirteenth command code through the device interface unit,
The correcting of the brightness may include correcting the brightness of a plurality of pixels constituting an area other than the lung area in the second X-ray image based on the lung area information. A contrast enhancing device in the lung area. According to claim 7,
The lung region information is a lung region image 705 divided into the lung region 705a and the lung outer region 705b in the second X-ray image,
The non-volatile recording medium,
generating a simulated X-ray image by projecting first CT scan information obtained by CT scanning the first scan target in one direction;
extracting an air area from the first CT scan information;
generating a simulated ground truth image by projecting only the air area among the first CT scan information in the one direction; and
When the simulated X-ray image is input to the network including a neural network, pixels corresponding to the air area in the simulated ground truth image have a first value, and the simulated ground truth image Pixels corresponding to the remaining areas except for the air area use the simulated X-ray image as input information for learning and the simulated ground truth image as a label so as to output the lung area image having a second value learning the network by
Includes a closed region learning command code for learning the network by a supervised learning method that includes,
The processing unit is adapted to learn the network by reading and executing the closed area learning command code through the device interface unit.
Contrast enhancement device for lung area in X-ray images. According to claim 7,
The lung region information is a lung region image 705 divided into the lung region 705a and the lung outer region 705b in the second X-ray image,
The network is learned by the second computing device,
The second computing device,
generating a simulated X-ray image by projecting first CT scan information obtained by CT scanning the first scan target in one direction;
extracting an air area from the first CT scan information;
generating a simulated ground truth image by projecting only the air area among the first CT scan information in the one direction; and
When the simulated X-ray image is input to the network including a neural network, pixels corresponding to the air area in the simulated ground truth image have a first value, and the simulated ground truth image Pixels corresponding to the remaining areas except for the air area use the simulated X-ray image as input information for learning and the simulated ground truth image as a label so as to output the lung area image having a second value learning the network by
which is supposed to run
Contrast enhancement device for lung area in X-ray images. According to claim 7,
The second X-ray image is generated by stretching the histogram of the first X-ray image output by the X-ray imaging device, or
The second X-ray image is an image output by an X-ray imaging device,
Contrast enhancement device for lung area in X-ray images. A second X-ray image 704 including the lung region is input to the network 500, and from the second X-ray image, a lung region image ( 705); and
enhancing the contrast of the lung region of the second X-ray image based on the lung region image;
Including,
the network,
generating a simulated X-ray image by projecting first CT scan information obtained by CT scanning the first scan target in one direction;
extracting an air area from the first CT scan information;
generating a simulated ground truth image by projecting only the air area among the first CT scan information in the one direction; and
When the simulated X-ray image is input to the network including a neural network, pixels corresponding to the air area in the simulated ground truth image have a first value, and the simulated ground truth image Pixels corresponding to the remaining areas except for the air area use the simulated X-ray image as input information for learning and the simulated ground truth image as a label so as to output a lung area image having a second value learning the network
Which is learned by a supervised learning method comprising
Contrast enhancement method of lung area in X-ray image. a device interface unit capable of reading a computer-readable non-volatile recording medium; And a processing unit; including,
The non-volatile recording medium,
A second X-ray image 704 including the lung region is input to the network 500, and a lung region image 705 divided into a lung region 705a and an outer lung region 705b in the second X-ray image ) 12th instruction code for executing the step of generating,
A thirteenth instruction code for executing a step of enhancing the contrast of the lung region of the second X-ray image based on the lung region image; and
Closed region learning command code for learning the network
It stores programs that include
The processing unit generates an X-ray image with improved contrast of the lung region by reading and executing the twelfth command code and the thirteenth command code through the device interface unit,
The closed area learning command code,
generating a simulated X-ray image by projecting first CT scan information obtained by CT scanning the first scan target in one direction;
extracting an air area from the first CT scan information;
generating a simulated ground truth image by projecting only the air area among the first CT scan information in the one direction; and
When the simulated X-ray image is input to the network including a neural network, pixels corresponding to the air area in the simulated ground truth image have a first value, and the simulated ground truth image Pixels corresponding to the remaining areas except for the air area use the simulated X-ray image as input information for learning and the simulated ground truth image as a label so as to output a lung area image having a second value learning the network
A command code for learning the network by a supervised learning method comprising
Contrast enhancement device for lung area in X-ray images.

Images