JP7283878B2 - VIDEO PROCESSING DEVICE, VIDEO PROCESSING METHOD, AND VIDEO PROCESSING PROGRAM - Google Patents

Description

TECHNICAL FIELD The present invention relates to an image processing apparatus, an image processing method, and an image processing program, and more particularly, an image processing apparatus and an image processing method for calculating the ratio of constituent tissues within an evaluation range of an evaluation target tissue to be evaluated in an input medical image. , and video processing programs.

2. Description of the Related Art Conventionally, image diagnosis is known in which diagnosis is performed by displaying an image of the shape of an organ, the presence or absence of a lesion, etc., which cannot be determined only by examining the body from the outside. Imaging diagnosis is made by a doctor based on images created using, for example, X-rays, ultrasound, or magnetic resonance, or examination images taken under visible light such as endoscopy or a fundus camera. ing.

For example, in mammography examination, which is one of breast cancer examinations, an X-ray imaging apparatus exclusively for the breast called mammography is used. By performing image diagnosis by mammography screening, it is possible to safely detect lesions such as breast cancer at an early stage with a small amount of radiation (see, for example, Patent Document 1).

This mammography screening should be able to capture the whole picture of calcified mammary glands, it has been statistically proven that the mortality rate is reduced by continuing screening, It has advantages such as ease of comparison.

On the other hand, the disadvantages of screening mammography include pain and exposure to a small amount of radiation. Furthermore, at ages such as the 20s and early 30s, which are said to be suitable for pregnancy, the mammary glands are developed in the breast, and detailed diagnosis may not be possible due to the amount of mammary glands that have developed.

The reason why a detailed diagnosis cannot be made by mammography screening due to the amount of developed mammary glands is that both the mammary glands and cancer are drawn white in the mammography image taken by mammography screening, so if the mammary glands have developed in the breast, This is because it becomes difficult to accurately detect n.

Specifically, the breast examined in breast cancer screening consists of mammary gland tissue, the tissue that secretes milk during childbirth, and fat tissue, blood vessels, nerves, etc. that support the mammary gland tissue. The more fat to be transmitted, the more black and transparent the whole image is drawn.

FIG. 20 is an example of a mammography image showing the classification of breast structure.
As shown in FIG. 20, the mammography image of the breast was compared with the amount of mammary gland tissue drawn in white and the amount of adipose tissue drawn in black. are classified into four types.

Fatty is a condition in which the breast is almost completely replaced with fat. Scattered mammary glands are a condition in which mammary glands are scattered in the breast replaced with fat, and fat in mammary glands is considered to be approximately 70 to 90%.

A non-uniform high concentration is a state in which fat is mixed in the parenchyma of the mammary gland and has a non-uniform concentration. A very high concentration means that there is almost no fat mixed in the parenchyma of the mammary gland, and fat in the mammary gland is considered to be approximately 10 to 20%.

Regarding the classification of these breast structures, mammography guidelines define dense breast as the condition classified into non-uniform high density and extremely high density, and the breast is drawn white in mammography images. Therefore, even if there is a lesion such as a tumor in the breast, it may not be visible because it is hidden by the white-drawn mammary gland.

Specifically, in mammography screening, tumors that are important for detecting cancer are drawn white in mammography images because they absorb X-rays. However, in the case of the dense breast, which is drawn whitish as a whole due to the development of mammary gland tissue, the tumor is drawn white while overlapping the entire drawn whitish. It was very difficult to detect the mass in

Such dense breasts are associated with low sensitivity of screening mammography and high risk of breast cancer. Especially in the case of Asians, close to 70% of the total is said to be dense breast.

In this way, it is said that even those who are evaluated as having dense breasts can detect tumors that are difficult to find in mammography screening by conducting ultrasound examinations that can detect tumors even in those who are evaluated as having dense breasts. .

JP 2018-42891 A

However, as mentioned above, the definition of dense breast is an extremely high concentration of fat in the mammary glands of about 10-20%, or a non-uniform high concentration of fat in the mammary glands of about 40-50%. It is difficult to draw a line because the definition is not clear, and the evaluation of dense breast may differ each time depending on the doctor who examines mammography images, which is a problem of lack of credibility. In addition to the evaluation of dense breasts, it was also difficult to distinguish between fatty, scattered mammary glands, heterogeneous high density, and extremely high density.

Specifically, Dense Breast is determined by a physician examining mammography images. However, it is difficult to calculate the ratio of fat and mammary gland tissue just by looking at the image, and it is determined based on the experience and sense of the doctor. For this reason, doctors sometimes judge dense breast differently. In other words, even when looking at the same mammography image, doctors may judge dense breast differently.

For this reason, if it is evaluated as dense breast without ambiguity, it will not only make the examinee feel uneasy, but also misunderstand that mammography screening is meaningless, and the screening rate of mammography screening will decrease. It leads to the problem that the early detection rate of cancer decreases.

If it is evaluated as a dense breast, it is common to perform an ultrasound examination, but since an ultrasound examination cannot leave a picture of the whole, the information is partially limited when reviewing it later. There are disadvantages such as the difficulty of evaluation due to the large number of people, and the lack of objectivity because it is greatly affected by the skill of the person who performs the examination.

In addition, in the current medical field, ultrasound examination equipment for breast cancer screening has not yet spread to the whole. Since there are not enough receptacles for ultrasound examination equipment, it is conceivable that the examination work will not function normally.
In this way, the ambiguous determination of the dense breast not only increases the burden on the examiner, but also increases the burden on the medical practice.

The following is also considered as a cause of this ambiguous judgment of Dense Breast.
In recent years, diagnostic imaging using X-rays, including mammography, has shifted from an analog method, in which the X-ray information transmitted through the human body is exposed to film to create an analog X-ray photograph, to a digital method in which the X-ray information is digitized. .

Since digitized X-ray information can be digitally processed, variations in image quality can be suppressed, which greatly contributes to doctors' diagnosis. However, digitized X-ray information can be freely subjected to digital processing such as contrast adjustment, so it is possible to adjust the entire image to be white depending on the adjustment of digital processing.

In this way, there is a risk that digital images will be evaluated as having dense breasts if the doctor judges that the entire mammography image has been adjusted to white by digital processing such as contrast adjustment, There is a risk that an image that should be there may be judged as not densely breasted by adjusting the overall mammography image to be black in digital processing adjustments.

The present invention has been made in view of these points, and provides an image processing apparatus, an image determination method, and an image determination program capable of accurately evaluating and determining the proportion of tissue in a certain range in a medical image. intended to

In the present invention, in order to solve the above problem, in an image processing device for calculating the composition ratio in an evaluation range of a tissue whose condition is to be evaluated in an input medical image, a reference tissue is used as a reference for calculating the composition ratio. a reference tissue luminance calculating means for calculating a reference tissue luminance which is the luminance of the evaluation range luminance; an evaluation range luminance calculating means for calculating an evaluation range luminance which is the luminance of the evaluation range; and a reference luminance ratio calculating means for calculating the ratio.

Thereby, the reference tissue luminance calculation means calculates the reference tissue luminance, which is the luminance of the reference tissue used as a reference for calculating the composition ratio, and the evaluation range luminance calculation means calculates the evaluation range luminance, which is the luminance of the evaluation range. The reference luminance ratio calculating means calculates the ratio of the evaluation tissue luminance to the reference range luminance.

Further, in the present invention, in the image processing method for calculating the composition ratio in the evaluation range of the tissue whose condition is to be evaluated in the input medical image, the reference tissue luminance calculation means sets the reference as a reference for calculating the composition ratio. a step of calculating a reference tissue luminance that is the luminance of a tissue; a step of calculating an evaluation range luminance that is the luminance of the evaluation range by means of evaluation range luminance calculation means; and calculating the ratio of the evaluated tissue luminance.

Thereby, the reference tissue luminance calculation means calculates the reference tissue luminance, which is the luminance of the reference tissue used as a reference for calculating the composition ratio, and the evaluation range luminance calculation means calculates the evaluation range luminance, which is the luminance of the evaluation range. The reference luminance ratio calculating means calculates the ratio of the evaluation tissue luminance to the reference range luminance.

Further, in the present invention, in the image processing program for calculating the composition ratio of the tissue whose condition is to be evaluated in the evaluation range in the input medical image, the computer is operated with the luminance of the reference tissue used as the reference for calculating the composition ratio. reference tissue luminance calculation means for calculating a certain reference tissue luminance; evaluation range luminance calculation means for calculating evaluation range luminance that is the luminance of the evaluation range; a standard for calculating the ratio of the evaluation tissue luminance to the reference range luminance A video processing program characterized by functioning as luminance ratio calculation means is provided.

Thereby, the reference tissue luminance calculation means calculates the reference tissue luminance, which is the luminance of the reference tissue used as a reference for calculating the composition ratio, and the evaluation range luminance calculation means calculates the evaluation range luminance, which is the luminance of the evaluation range. The reference luminance ratio calculating means calculates the ratio of the evaluation tissue luminance to the reference range luminance.

According to the video processing device, the video processing method, and the video processing program of the present invention, the reference tissue brightness calculating means calculates the reference tissue brightness, which is the brightness of the reference tissue used as a reference for calculating the composition ratio, and evaluates it. The range luminance calculation means calculates the evaluation range luminance, which is the luminance of the evaluation range, and the reference luminance ratio calculation means calculates the ratio of the evaluation tissue luminance to the reference range luminance. It is possible to accurately evaluate and judge the ratio of the organization.

1 is a block diagram showing the concept of a video processing device according to a first embodiment; FIG. 4 is a block diagram showing details of a mask processing unit; FIG. FIG. 4 is a block diagram showing details of a reference tissue luminance calculator; 3 is a block diagram showing the details of an evaluation range luminance calculator; FIG. 3 is a block diagram showing details of a reference luminance ratio calculator; FIG. 3 is a block diagram showing details of an information storage unit; FIG. It is a figure which shows the image for learning which a learning part is made to learn by a learning information input means. 1 is a diagram showing a U-net structured neural network used in deep learning; FIG. FIG. 4 is a formula and a diagram showing a method of measuring a learning result in a process of learning by a learning unit; It is the graph which represented the result which the learning part learned using the DICE coefficient. It is a figure which shows the mask process which a mask process part processes. FIG. 4 is a diagram showing a state in which a reference tissue brightness calculator detects the brightness of a reference tissue; It is the figure which expanded and displayed a part of mammography image. FIG. 10 is an image diagram showing conversion processing performed by a relative luminance conversion processing unit; FIG. 10 is an image diagram showing a state processed by a relative luminance ratio calculator; FIG. 9 is a block diagram showing details of a reference tissue luminance calculator in the video processing device according to the second embodiment; It is a mammography image which shows the position which a scanning information acquisition part scans. It is a figure which shows the moving average line which visualized the moving average information which the graph which visualized the luminance which the luminance measurement part measured, and the moving average calculation part calculated. It is the figure which graphed the inclination information of the moving average line which the moving average inclination calculation part calculated. FIG. 1 shows an example of a mammography image showing classification of breast structure;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[First embodiment]
FIG. 1 is a block diagram showing the concept of a video processing device according to the first embodiment.

In the present invention, conventionally, a doctor makes a judgment by interpreting a medical image. By processing medical images, it establishes an objective evaluation method and increases the accuracy of diagnosis by doctors.

For the sake of convenience, a case in which the image processing apparatus 100 evaluates dense breast will be described by using the image processing apparatus 100 as a mammography image captured by a mammography examination, which is a medical image input from the medical imaging apparatus 200 .

The medical imaging device 200 is a device for taking medical images taken by, for example, an X-ray device, a computer tomography device, a nuclear magnetic resonance imaging device, an ultrasonic examination device, or the like.

The video processing device 100 of the present invention processes still images and moving images captured by the medical imaging device 200, and outputs evaluation results of dense breast from the input medical video such as mammography images to the display means 300. can.

The display unit 300 includes, for example, a display that displays information given by being connected to a personal computer, a printer that prints and displays input information, and a developing machine that develops photographs and the like.

Next, details of the video processing device 100 will be described.
As shown in FIG. 1, the image processing apparatus 100 includes a mask processing unit 110, a reference tissue luminance calculation unit 120, an evaluation range luminance calculation unit 130, a reference luminance ratio calculation unit 140, a state determination unit 150, and an information storage unit 160. I have. The video processing device 100 is also connected to the medical imaging device 200 and the display means 300 .

The mask processing unit 110 is connected to the medical imaging apparatus 200 , the reference tissue luminance calculation unit 120 and the information storage unit 160 , and performs mask processing on mammography images input from the medical imaging apparatus 200 .

The mask processing referred to here is image processing for extracting only a specific portion from the mammography image input from the medical imaging apparatus 200 and not displaying anything other than the extracted specific portion.

Specifically, muscles, skin, and nipples are excluded from mammography images taken by mammography screening, and only the range where the breast tissue targeted for dense breast evaluation exists is extracted as the evaluation range. The mask processing unit 110 performs image processing for not displaying a portion. A specific image processing method will be described later.

The mask processing unit 110 stores the image masked by the mask processing unit 110 and the mammography image before mask processing in the connected information storage unit 160 . Further, when the processing of the mask processing unit 110 is completed, the mask processing unit 110 notifies the connected reference tissue luminance calculation unit 120 of the completion of the processing.

The reference tissue brightness calculation unit 120 is connected to the mask processing unit 110, the evaluation range brightness calculation unit 130, and the information storage unit 160, and serves as a reference for evaluating dense breast from the mammography image input by the medical imaging apparatus 200. It is for calculating the brightness of adipose tissue.

As described above, dense breast is evaluated by the proportion of fat in the mammary gland, and here, the reference tissue brightness calculator 120 calculates the brightness of adipose tissue as a reference tissue. The term "luminance" as used herein indicates the degree of whiteness, which is the brightness of the mammography image input by the medical imaging apparatus 200. The reference tissue luminance calculation unit 120 calculates the luminance of adipose tissue in the mammography image. It is for

Upon receiving the notification that the processing of the mask processing unit 110 has been completed, the reference tissue brightness calculation unit 120 calls up the pre-masking mammography image stored in the information storage unit 160, and uses the called mammography image to determine the dense breast evaluation criteria. The brightness of the adipose tissue is calculated, and the calculated brightness of the adipose tissue is stored in the information storage unit 160 . A specific method for calculating the brightness of adipose tissue, which is a reference tissue, will be described later. Further, when the processing of the reference tissue brightness calculation unit 120 is completed, the reference tissue brightness calculation unit 120 notifies the connected evaluation range brightness calculation unit 130 of the completion of the processing.

The evaluation range brightness calculation unit 130 is connected to the reference tissue brightness calculation unit 120, the reference brightness ratio calculation unit 140, and the information storage unit 160, and calculates the brightness of the evaluation range in which the mammary tissue processed by the mask processing unit 110 exists. It is for calculation.

Upon receiving the notification that the processing of the reference tissue brightness calculation unit 120 has been completed, the evaluation range brightness calculation unit 130 calls up the masked mammography image stored in the information storage unit 160, and calculates each image in the evaluation range in which mammary gland tissue exists. The brightness of each part is calculated, and the brightness of each part in the calculated evaluation range is stored in the information storage unit 160 . A specific calculation method will be described later. Further, when the evaluation range luminance calculation unit 130 completes the processing, the evaluation range luminance calculation unit 130 notifies the connected reference luminance ratio calculation unit 140 that the processing has been completed.

The reference luminance ratio calculation unit 140 is connected to the evaluation range luminance calculation unit 130, the state determination unit 150, and the information storage unit 160. It is for calculating the ratio of

Upon receiving the notification that the processing of the evaluation range luminance calculation unit 130 has been completed, the reference luminance ratio calculation unit 140 calculates the luminance of the adipose tissue, which is the reference tissue stored in the information storage unit 160, and the evaluation range in which the mammary gland tissue exists. Then, the ratio of the luminance of the mammary gland tissue is calculated with respect to the luminance of the adipose tissue as a reference value, and the calculated ratio of the luminance is stored in the information storage unit 160 . A specific calculation method will be described later. Further, when the processing of the reference luminance ratio calculation unit 140 is completed, the reference luminance ratio calculation unit 140 notifies the connected state determination unit 150 that the processing is completed.

The state determination unit 150 is connected to the reference luminance ratio calculation unit 140, the information storage unit 160, and the display means 300, and the mammary gland when the luminance of the adipose tissue calculated by the reference luminance ratio calculation unit 140 is used as a reference value. It is for judging whether or not the mammography image input from the medical imaging apparatus 200 is dense breast based on the ratio of the luminance of the fatty tissue to the luminance of the evaluation range where the tissue exists.

When the state determination unit 150 receives the notification that the processing of the reference luminance ratio calculation unit 140 has been completed, the state determination unit 150 determines the luminance of the adipose tissue stored in the information storage unit 160 as the reference value in the evaluation range in which the mammary gland tissue exists. The ratio of luminance and the judgment criteria for judging whether or not the mammography image is dense breast stored in advance in the information storage unit 160 are called to determine whether the mammography image input from the medical imaging apparatus 200 is dense breast. determine whether or not A specific determination method will be described later. Further, when the processing of the state determination unit 150 is completed, the state determination unit 150 displays the determination result via the connected display means 300 or the like.

The information storage unit 160 is connected to the mask processing unit 110, the reference tissue brightness calculation unit 120, the evaluation range brightness calculation unit 130, the reference brightness ratio calculation unit 140, and the state determination unit 150, and stores information necessary for processing. , or a storage medium from which stored information can be called as needed.

As described above, from the mammography image input by the medical imaging apparatus 200, the reference tissue luminance calculation unit 120 calculates the luminance of adipose tissue as a reference value for evaluation, and the evaluation range luminance calculation unit 130 calculates mammary glands to be evaluated. The brightness of each site in the evaluation range where tissue exists is calculated, and the reference brightness ratio calculator 140 calculates the brightness rate of each site in the evaluation range where mammary tissue exists using the brightness of adipose tissue as a reference value. Dense breast criteria, which were previously ambiguous, can be expressed clearly as numerical values.

In addition, since the ratio of luminance in the evaluation range in which mammary gland tissue exists is calculated using the luminance of adipose tissue as a reference value, even if digital processing such as contrast adjustment is performed on the mammography image input by the medical imaging apparatus 200, A relative evaluation using adipose tissue as a reference value can be calculated. This makes it possible to objectively evaluate dense breast.

FIG. 2 is a block diagram showing details of the mask processing unit.
As shown in FIG. 2, the mask processing unit 110 includes a boundary determination unit 111, an image processing unit 112, and a learning unit 113.

The boundary determination unit 111 is connected to the medical imaging apparatus 200, the image processing unit 112, and the information storage unit 160, and removes muscles, skin, papillae, etc. from the mammography image input from the medical imaging apparatus 200 to determine mammary glands. This is for determining the boundary between the evaluation range in which the mammary gland tissue exists and other tissues in order to extract the evaluation range in which the tissue exists.

For example, while the doctor confirms the mammography image input from the medical imaging apparatus 200, the boundary determining unit 111 uses input means (not shown) to enclose only the range in which the mammary tissue to be evaluated exists. Able to determine organizational boundaries. Mammography images before mask processing can be stored in the information storage unit 160 by the boundary determination unit 111 .

Alternatively, as will be described later, for example, the learning unit 113 such as artificial intelligence is made to learn to identify the boundary between the range in which the mammary gland tissue to be evaluated exists and other tissues, and the mammary gland tissue obtained by learning is The learning unit 113 causes the information storage unit 160 to store boundary identification information between the existing range and other tissues, and the boundary determination unit 111 uses the boundary identification information stored in the information storage unit 160 to determine whether mammary tissue exists. It is also possible to identify boundaries between the evaluation range to be evaluated and other tissues to determine boundaries between mammary tissue and other tissues.

When a mammography image is input from the medical imaging apparatus 200, the boundary determination unit 111 determines the boundary between the evaluation range in which the mammary gland tissue exists and other tissues. sent to.

The image processing unit 112 is connected to the boundary determination unit 111, the information storage unit 160, and the reference tissue brightness calculation unit 120, extracts an evaluation range in which mammary tissue to be evaluated for dense breast exists, and determines whether the mammary tissue is present. This is for masking so that parts other than the existing evaluation range are not displayed.

When the image processing unit 112 receives a mammography image including boundary information from the boundary determination unit 111, the image processing unit 112 extracts muscles, skin, papilla, etc. from the input mammography image based on the boundary information determined by the boundary determination unit 111. Mask processing for exclusion is performed, and the mammography image after mask processing is stored in the information storage unit 160 . Further, when the processing of the image processing unit 112 is completed, the image processing unit 112 notifies the connected reference tissue luminance calculation unit 120 of the completion of the processing.

The learning unit 113 is connected to the learning information input unit 114 and the information storage unit 160. Based on the learning information input by the learning information input unit 114, the evaluation range in which the mammary gland tissue is determined by the boundary determination unit 111 and others. It is for learning the boundary identification information necessary to identify the boundary with the organization.

As a specific example of learning, medical professionals create a masking image that surrounds only the range where the mammary tissue to be evaluated exists, excluding muscles, skin, and nipples from the mammography images to be learned. By inputting a large amount of the masking images to the learning unit 113, learning for distinguishing between the evaluation range in which the mammary gland tissue exists and the other tissue is performed.

The input masking image is processed by deep learning using a convolutional neural network, specifically deep learning using a U-net structure model called U-net that specializes in image segmentation processing. Learn to identify the assessment scope in which the organization exists and the boundaries with other organizations. The learning unit 113 accumulates the boundary identification information for identifying the boundary between the evaluation range in which the mammary gland tissue exists and the other tissue obtained by learning in the information storage unit 160 .

FIG. 3 is a block diagram showing the details of the reference tissue luminance calculator.
As shown in FIG. 3 , the reference tissue brightness calculator 120 includes a scanning information acquisition unit 121 , a boundary detection unit 122 , a reference tissue brightness measurement unit 123 and an average value calculation unit 124 .

The scanning information acquisition unit 121 is connected to the mask processing unit 110, the boundary detection unit 122, and the information storage unit 160, scans the mammography image input from the medical imaging apparatus 200, and acquires information on the image. It is for

Upon receiving the notification that the processing of the mask processing unit 110 has been completed, the scanning information acquisition unit 121 calls up the pre-mask mammography image stored in the information storage unit 160, scans the pre-mask mammography image, and scans the mask. Information on mammography images before processing is acquired.

The scanning information acquisition unit 121 can obtain information on the mammography image before mask processing at an arbitrary position and in an arbitrary range based on the position information on the mammography image before mask processing that is input by an input means (not shown).

In addition, the scanning unit 121 scans in an arbitrary unit. For example, a mammography image before mask processing is scanned in units of pixels, which are the minimum units having color tone and gradation information when an image is handled by a computer. Scanning can also be performed, and information on mammography images before mask processing can be obtained while scanning a frame of arbitrary size such as 1 mm×1 mm. Note that the present embodiment will be described assuming that the scanning information acquisition unit 121 scans in units of pixels. In addition, the information on the mammography image before mask processing obtained by the scanning information acquisition unit 121 while scanning is transmitted to the boundary detection unit 122 .

The boundary detection unit 122 is connected to the scanning information acquisition unit 121 and the reference tissue luminance measurement unit 123, and detects the boundary between the background and the non-background from the information on the pre-masking mammography image acquired by the scanning information acquisition unit 121. is for detecting

Specifically, for example, in a mammography image before mask processing, the boundary detection unit 122 starts scanning at a position where a lot of adipose tissue is seen, specifically the upper part of the breast, and detects the boundary between the background and the breast, specifically A boundary detection unit 122 detects the boundary between the background and the skin seen on the surface of the breast.

Here, the luminance of the background is 0 because it is rendered black, and the luminance exceeds 0 because the skin is rendered at least whiter than the background. When the scanning information acquiring unit 121 starts scanning the mammography images before mask processing in order from the background side at a certain height, the skin is reached before long.

At this time, the brightness is switched from 0 to a value exceeding 0. The boundary detection unit 122 can detect the portion where the value changes from 0 to a numerical value exceeding 0 as the boundary between the background and the non-background, that is, the boundary between the background and the breast.

Conversely, when the scanning information acquisition unit 121 starts scanning mammography images before mask processing in order from the breast side, the background is reached before long. At this time, the brightness is switched from a value exceeding 0 to 0. The boundary detection unit 122 can detect the portion where the numerical value exceeding 0 is switched to 0 as the boundary between the background and the non-background, that is, the boundary between the background and the breast.

In addition to changing the scanning direction of the scanning information acquiring unit 121, before scanning by the scanning information acquiring unit 121, the mammography image before mask processing is horizontally reversed in advance, and then the scanning information acquiring unit 121 scans in a certain direction. 121 may scan.

The reference tissue luminance calculation unit 120 is for calculating the luminance of the reference adipose tissue. Start scanning at a certain height, such as the top of the breast. When the boundary detection unit 122 detects the boundary between the background and the non-background, it transmits the boundary position information to the reference tissue luminance measurement unit 123 .

The reference tissue luminance measurement unit 123 is connected to the boundary detection unit 122, the average value calculation unit 124, and the information storage unit 160, and is used to calculate the luminance at an arbitrary position from the boundary detected by the boundary detection unit 122. is.

Specifically, scanning is sequentially started at a certain height above the breast in mammography images before mask processing, and when the boundary detection unit 122 detects the boundary between the background and the skin, The reference tissue brightness measurement unit 123 measures the brightness of the fatty tissue at the position of the fatty tissue that is often seen on the skin side, specifically, at the position moved from the boundary between the background and the skin to the breast side by 10 to 100 pixels, and further measures the brightness of the fatty tissue. The reference tissue brightness measurement unit 123 measures the brightness of the fat tissue for a certain distance, such as 100 pixels, from the position where the reference tissue brightness is measured.

Accordingly, the reference tissue luminance measurement unit 123 can measure the luminance of the fatty tissue at a position where a large amount of fatty tissue exists in the mammography image. The luminance information measured by the reference tissue luminance measurement unit 123 is stored in the information storage unit 160 by the reference tissue luminance measurement unit 123 . Further, when the processing of the reference tissue brightness measurement unit 123 is completed, the reference tissue brightness measurement unit 123 notifies the connected average value calculation unit 124 of the completion of the processing.

The average value calculation unit 124 is connected to the reference tissue luminance measurement unit 123 , the information storage unit 160 , and the evaluation range luminance calculation unit 130 , and calculates fat by the processing of the reference tissue luminance measurement unit 123 stored in the information storage unit 160 . It is for calculating the average value of the brightness of adipose tissue based on the brightness of the tissue.

Upon receiving the notification that the processing of the reference tissue luminance measurement unit 123 has been completed, the average value calculation unit 124 calculates the luminance of adipose tissue in the specific range stored in the information storage unit 160, specifically, the boundary between the background and the skin. From the position moved 10 to 100 pixels to the skin side, the brightness of a certain distance such as 100 pixels is called, the average value is calculated from the brightness of these called constant distances, and the calculated average value of the brightness of the adipose tissue is stored in the information storage unit 160 . Further, when the processing of the average value calculation unit 124 is completed, the average value calculation unit 124 notifies the connected evaluation range luminance calculation unit 130 that the processing has been completed.

In this way, the reference tissue brightness calculator 120 measures the brightness of adipose tissue at an arbitrary position where a large amount of reference adipose tissue exists, and uses the average value as a reference. can also be based on the average value.

FIG. 4 is a block diagram showing the details of the evaluation range luminance calculator.
As shown in FIG. 4 , the evaluation range luminance calculator 130 includes an evaluation range scanning information acquisition section 131 and an evaluation range luminance measurement section 132 .

When the evaluation range scanning information acquisition unit 131 receives the notification that the processing of the reference tissue brightness calculation unit 120 is completed, it calls up the masked mammography image stored in the information storage unit 160, and extracts the masked mammography image excluding the background. Image information is obtained while scanning the image of the evaluation range where the tissue exists.

In the evaluation range scanning information acquisition unit 131, the scanning unit can be set arbitrarily. For example, the mammography image before mask processing can be scanned pixel by pixel. It is also possible to obtain information on the image of the evaluation range in which the mammary gland tissue exists after masking, excluding the background, while scanning the frame. In this embodiment, the evaluation range scanning information acquisition unit 131 scans in units of pixels. The evaluation range scanning information acquisition unit 131 obtains information on the image of the evaluation range in which mammary gland tissue exists after the mask processing, excluding the background, while scanning, and is transmitted to the evaluation range luminance measurement unit 132 .

The evaluation range luminance calculation unit 130 is connected to the evaluation range scanning information acquisition unit 131, the evaluation range luminance measurement unit 132, the information storage unit 160, and the reference luminance ratio calculation unit 140, and the evaluation range scanning information acquisition unit 131 acquires Based on the obtained information, the brightness of each part of the evaluation range in which the mammary gland tissue to be evaluated exists is measured.

Specifically, the evaluation range luminance measurement unit 132 measures the luminance of the evaluation range in which the mammary tissue exists from the image information of the evaluation range in which the mammary tissue exists, which is transmitted from the evaluation range scanning information acquisition unit 131. The measurement result is stored in the information storage unit 160 . Further, when the evaluation range luminance measurement unit 132 completes the processing, the evaluation range luminance measurement unit 132 notifies the connected reference luminance ratio calculation unit 140 of the completion of the processing.

FIG. 5 is a block diagram showing the details of the reference luminance ratio calculator.
As shown in FIG. 5 , the reference luminance ratio calculator 140 includes a relative luminance conversion processor 141 and a relative luminance ratio calculator 142 .

The relative luminance conversion processing unit 141 is connected to the evaluation range luminance calculation unit 130, the relative luminance ratio calculation unit 142, and the information storage unit 160, and uses the luminance of adipose tissue as a reference value to determine the evaluation range in which mammary tissue exists. It is for converting luminance as a relative value.

When the relative brightness conversion processing unit 141 receives the notification that the processing of the evaluation range brightness calculation unit 130 has been completed, the brightness of the reference adipose tissue stored in the information storage unit 160 and the mammary gland tissue to be evaluated exist. The brightness of each site in the evaluation range is called, and the brightness of each site in the evaluation range where the mammary gland tissue exists is converted as a relative value of the brightness of the adipose tissue with the brightness of the adipose tissue as a reference value.

Specifically, for example, when the luminance of the reference adipose tissue is set to 1, the luminance of the background is set to 0, and the luminance of the evaluation range in which mammary gland tissue exists is converted into a relative luminance that is a relative value. The converted relative brightness of the evaluation range in which mammary gland tissue exists is transmitted to the connected relative brightness ratio calculator 142 .

The relative brightness ratio calculation unit 142 is connected to the relative brightness conversion processing unit 141, the information storage unit 160, and the state determination unit 150, and calculates each relative value based on the relative brightness processed by the relative brightness conversion processing unit 141. It is for calculating percentages.

The relative brightness ratio calculation unit 142 aggregates the relative brightness of each part of the evaluation range in which the mammary gland tissue as the evaluation target transmitted by the relative brightness conversion processing unit 141 exists, for each constant relative brightness value, and calculates the constant relative brightness. A ratio for each luminance value is calculated and stored in the connected information storage unit 160 . Further, when the processing of the relative brightness ratio calculation unit 142 is completed, the relative brightness ratio calculation unit 142 notifies the connected state determination unit 150 that the processing is completed.

FIG. 6 is a block diagram showing details of the information storage unit.
As shown in FIG. 6, the information storage unit 160 includes a pre-masking image storage unit 161, a post-masking image storage unit 162, a reference tissue brightness storage unit 163, an evaluation range brightness storage unit 164, a relative brightness ratio storage unit 165, A judgment criterion storage unit 166 and a boundary identification information storage unit 167 are provided.

The pre-mask processing image storage unit 161 is for storing mammography images input from the medical imaging apparatus 200 and before the mask processing unit 110 performs mask processing. The information stored in the pre-masking image storage unit 161 is stored in association with, for example, identification information for identifying the pre-masking mammography image, examinee number, examination date, etc. Information is called.

The post-masking image storage unit 162 is for storing a mammography image input from the medical imaging apparatus 200 and subjected to masking by the masking unit 110, that is, an image in which only the range in which mammary tissue exists is extracted. It is. The information stored in the post-masking image storage unit 162 is stored in association with, for example, identification information for identifying the masked mammography image, an examinee number, an examination date, and the like. information is called.

The reference tissue brightness storage unit 163 is for storing the reference brightness of adipose tissue calculated by the reference tissue brightness calculation unit 120 . The information stored in the reference tissue brightness storage unit 163 is stored in association with, for example, identification information for identifying a mammography image for calculating the brightness of the reference tissue, an examinee number, an examination date, a brightness calculation date, and the like. and the desired information is called up as needed.

The evaluation range brightness storage unit 164 is for storing the brightness of each site in the evaluation range in which the mammary gland tissue to be evaluated exists, which is calculated by the evaluation range brightness calculation unit 130 . The information stored in the evaluation range luminance storage unit 164 is stored in association with, for example, identification information for identifying mammography images for calculating the luminance of the evaluation range, examinee number, examination date, luminance calculation date, and the like. and the desired information is called up as needed.

The relative brightness ratio storage unit 165 is for storing the relative brightness of each site in the evaluation range in which the mammary gland tissue, which is the evaluation target for the reference adipose tissue, is calculated by the reference brightness ratio calculation unit 140 . The information stored in the relative luminance ratio storage unit 165 is stored in association with, for example, identification information, examinee's number, examination date, relative luminance ratio calculation date, etc., and desired information is called as needed.

The determination criterion storage unit 166 determines that the image input from the medical imaging apparatus 200 is dense breast based on the ratio of the luminance of the mammary gland tissue when the luminance of the adipose tissue calculated by the reference luminance ratio calculation unit 140 is used as a reference value. This is for storing a reference value for the state determination unit 150 to determine whether or not the state is correct. The information stored in the criterion storage unit 166 is stored in association with identification information for identifying the criterion, and desired information is called as necessary.

Further, the criteria stored in the judgment criteria storage unit 166 are not only stored as fixed criteria, but can also be stored in association with age, sex, etc. so that the criteria can be appropriately changed according to age and sex. .

Furthermore, in the present embodiment, an example of judging the state of dense breast is shown, but when judging other symptoms, it is stored in association with the name of the symptom, etc., and the desired information such as the symptom is called as necessary. You may do so.

The boundary identification information storage unit 167 stores boundary identification information for identifying the boundary between the evaluation range in which the mammary gland tissue exists and another tissue, which is obtained by the learning unit 113 learning the boundary of the evaluation range in which the mammary tissue exists. It is for remembering.

Based on the information accumulated in the boundary identification information storage unit 167, the mask processing unit 112 can perform mask processing for excluding muscles, skin, papilla, etc. from the mammography image input from the medical imaging apparatus 200. In addition, the information storage unit 160 can store various information as needed.

FIG. 7 is a diagram showing a learning image that is learned by the learning unit by the learning information input means.
As shown in FIG. 7, FIGS. 7A and 7B are mammography images and their masking images for learning by the learning section 113 by the learning information input means 114. FIG.

FIG. 7A is a mammography image taken for breast cancer screening, for example. FIG. 7B is a masking image obtained by excluding muscle, skin, papilla, etc. from the image of FIG.

First, as images for the learning unit 113 to learn, a large number of mammography images such as those shown in FIG. 7A and masking images surrounding only the range where mammary gland tissue exists such as those shown in FIG. 7B are prepared. As for the masking images for learning, it is desirable that medical personnel or the like manually create them.

A large amount of the created mammography images for learning and the masking images surrounding only the range where mammary gland tissue exists are input to the learning unit 113 by the learning information input means 114, and from the information input by the learning information input means 114, The learning unit 113 performs learning to identify the boundary between the evaluation range in which mammary gland tissue exists and other tissues, and identifies the boundary between the evaluation range in which mammary gland tissue exists and other tissues obtained by learning. The learning unit 113 causes the information storage unit 160 to store the identification information.

FIG. 8 is a diagram showing a U-net structured neural network used in deep learning.
As shown in FIG. 8, the learning unit 113 uses a program using a deep learning technique called U-net to identify boundaries between evaluation ranges in which mammary tissue exists and other tissues. Information, ie the extent of the evaluation area in which the mammary tissue is present, can be learned.

A feature of U-net is that the input layer and the output layer have the same image size.
In image recognition performed by ordinary neural networks, the convolutional layer extracts the complex features of the object, and the pooling layer is responsible for allowing for the overall displacement of the object.

For this reason, the deeper the layer, the more complex features are extracted, but the positional information of the extracted features becomes ambiguous. However, in order to identify a boundary such as an evaluation range where mammary gland tissue exists and specify the range, complex extracted features and overall position information must be specified.

Therefore, by using a U-shaped neural network as shown in FIG. 8, the position information that has become ambiguous in the pooling layer can be restored on the original image.

The horizontal arrow in Fig. 8 is called "convolution layer", the downward arrow is called "pooling layer", and the upward arrow is called "convolution + upsampling". characteristics can be extracted.

Also, the deeper the "pooling layer" is stacked, the more positional information of the extracted features is allowed. Furthermore, by combining upsampling with information from the previous layer, the image can be reconstructed while preserving the features extracted by the "convolutional layer".

This down arrow, up arrow, and horizontal arrow process extracts the features of the input image and restores the original image details from deep layers while retaining the extracted features. A desired region can be extracted from an input image by using such a technique.

In this embodiment, by using a program using the above-mentioned deep learning technology called U-net, a breast mammography image is input to the input layer, and the image output from the output layer is compared with the masking image. By learning to align and match, it is possible to identify the boundary between the evaluation range in which mammary gland tissue exists and other tissues from mammography images of the breast, and extract only the range in which mammary gland tissue exists. .

In addition, the learning by the learning unit 113 can use not only programs using deep learning technology called U-net, but also various programs such as deep learning technology called FCN (Fully Convolutional Network).

FIG. 9 is a formula and a diagram showing a method of measuring the learning result in the process of learning by the learning unit.
As shown in FIG. 9, if the image output by learning is X and the manually masked image is Y, DICE coefficients are represented by DICE coefficients DSC as shown in FIG.

The DICE coefficient indicates the ratio of the average number of elements of two sets to the number of common elements, and the calculation result is a value between 0 and 1. As the DICE coefficient, that is, the shaded portion where X and Y overlap, approaches 1, it is determined that the similarity between the two sets is high.

FIG. 10 is a graph showing the results of learning by the learning unit using DICE coefficients.
As shown in FIG. 10, when the learning unit 113 performs learning using U-net, the DICE coefficient DSC is 0.8984, indicating a matching rate of nearly 90%. From this result, it can be seen that the learning of the learning unit 113 allows the mask processing unit 110 to automatically identify the boundary between the evaluation range in which mammary gland tissue exists and other tissue.

FIG. 11 is a diagram showing mask processing performed by the mask processing unit.
As shown in FIG. 11, the mask processing unit 110 performs mask processing as shown in FIGS. 11(A), (B), and (C).

FIG. 11A is a mammography image input by the medical imaging apparatus 200. FIG.
FIG. 11(B) shows a diagram in which a mammography image input to the mask processing unit 110 is processed by a boundary determination unit using boundary identification information that identifies a boundary between an evaluation range in which mammary gland tissue exists and another tissue learned by the learning unit 113. A masking image 111 encloses only the range in which the processed mammary gland tissue to be evaluated exists.

FIG. 11C shows an image processed by the image processing unit 112 . FIG. 11(A)
and the masked image of FIG. 11(B), mask processing is performed to exclude muscle, skin, papilla, etc. as shown in FIG. An image that extracts only is drawn.

FIG. 12 is a diagram showing a state in which the reference tissue brightness calculator detects the brightness of the reference tissue.
The image in FIG. 12 depicts the upper part of the breast from a mammography image. A lot of adipose tissue that serves as a reference exists in the upper part of the breast, and in the present embodiment, an example in which luminance is calculated using the adipose tissue that exists in the upper part of the breast as a reference will be described.

First, the scanning information acquiring unit 121 starts scanning at a certain position of the mammography image and acquires information while moving in the horizontal direction. The position to start scanning can be determined by the doctor while looking at the screen, or scanning can be started at a certain height.
The scanning information acquisition unit 121 scans while moving in units of pixels in the horizontal direction, and the boundary detection unit 122 detects the boundary between the background and the skin.

The reference tissue luminance measurement unit 123 measures the luminance of the adipose tissue at a position shifted horizontally by 10 to 100 pixels toward the breast from the boundary detection point where the boundary is detected by the boundary detection unit 122, and further measures the luminance for 100 pixels in the horizontal direction. The reference tissue luminance measurement unit 123 measures the luminance of the adipose tissue.

The average value of the measured brightness of 100 pixels of the adipose tissue is calculated by the average value calculation unit 124, and the calculated average value is stored in the information storage unit 160 as the reference brightness of the adipose tissue.

In addition, the measurement of the luminance of adipose tissue for 100 pixels by the reference tissue luminance measuring unit 123 can be repeated multiple times. Specifically, after the first measurement and calculation of the average value, the boundary between the background and the skin is again detected by the boundary detection unit 122 at a position 1 to 50 pixels below the breast, and the reference tissue luminance measurement unit 123 repeats the measurement of luminance at a position horizontally shifted by 10 to 100 pixels toward the breast side a plurality of times such as 100 times, and further averages the repeatedly calculated average values to store information as the luminance of adipose tissue that serves as a reference. It can also be stored in unit 160 .

FIG. 13 is an enlarged view of a part of the mammography image.
As shown in FIG. 13, when a mammography image is enlarged, it can be seen that when the image is handled by a computer or the like, it is displayed as a collection of pixels, which are the minimum units having luminance information.

A mammography image is displayed based on the luminance information of each pixel. The evaluation range scanning information acquisition unit 131 of the evaluation range luminance calculation unit 130 acquires the luminance information of each pixel, and the evaluation range luminance measurement unit 132 calculates one pixel. The brightness of each is measured.

FIG. 14 is an image diagram showing conversion processing performed by the relative luminance conversion processing unit.
As shown in FIG. 14, when the luminance of the reference adipose tissue calculated by the reference tissue luminance calculation unit 120 is set to 1, each part of the evaluation range in which the mammary gland tissue exists calculated by the evaluation range luminance calculation unit 130 is converted by the relative luminance conversion processing unit 141 into a relative luminance value of . In this way, the brightness of the evaluation range calculated in FIG. 13 is converted into a relative value with reference to the adipose tissue.

FIG. 15 is an image diagram showing a state processed by the relative luminance ratio calculator.
As shown in FIG. 15, the relative brightness ratio calculation unit 142 calculates and tabulates the brightness ratio based on the relative value with reference to the adipose tissue converted by the relative brightness conversion processing unit 141 .

The higher the value of the relative brightness, the whiter the image is drawn, and the lower the value, the blacker the adipose tissue and the background.
In FIG. 15, 1% of the total have a relative luminance of 0 or more and less than 1, 38% of the total have a relative luminance of 1 or more and less than 2, and 27 of the total have a relative luminance of 3 or more and less than 4. %, 12% of the total have a relative luminance of 4 or more and less than 5, 7% of the total have a relative luminance of 5 or more and less than 6, 5% of the total have a relative luminance of 6 or more and less than 7, 4% of the total have a relative luminance of 7 or more and less than 8, 4% of the total have a relative luminance of 8 or more and less than 9, 1% of the total have a relative luminance of 9 or more and less than 10, relative luminance is 10 or more is 0% of the total.

Based on this result and the criteria stored in the information storage unit 160, the state determination unit 150 determines whether dense breast is present or not.
For example, assuming that objects with a relative luminance of 3 or higher are drawn in white as a criterion, "Dense breast when the ratio of relative luminance of 3 or higher is 30% or higher, and dense breast when the ratio of relative luminance of 3 or higher is less than 30%" When the standard is set to "not", the result is 77% consistent with the result of human evaluation.

Note that when the results shown in FIG. 15 are applied to the above criteria, the total ratio of the relative luminance of 3 or higher is 27%+12%+7%+5%+4%+4%+1%=60%, so the relative luminance of 3 or higher. is 30% or more, it is judged to be dense breast.

Also, the relative luminance ratio calculator 142 can visualize the ratio of relative luminance processed by the relative luminance ratio calculator 142 in colors like a heat map. As a specific example, white for parts with a relative luminance of 0 or more and less than 1, yellow for parts with a relative luminance of 1 or more and less than 2, and yellow for parts with a relative luminance of 2 or more and less than 3 Pink, orange for areas with a relative brightness of 3 or more and less than 4, and red for areas with a relative brightness of 4 or more, may be reflected in mammography images before or after mask processing. good.

As described above, by visualizing the ratio of relative luminance processed by the relative luminance ratio calculation unit 142 in color, the doctor can evaluate dense breast not only from the numerical value but also from the visualized information.

[Second embodiment]
Next, a second embodiment of the invention will be described. The video processing apparatus of this embodiment has substantially the same configuration as that of the first embodiment, except that the structure and calculation method of the reference tissue luminance calculator are different. For this reason, substantially the same components as those of the first embodiment are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

FIG. 16 is a block diagram showing details of a reference tissue luminance calculator in the video processing device according to the second embodiment.
As shown in FIG. 16, the reference tissue brightness calculation unit 120 includes a scanning information acquisition unit 121, a boundary detection unit 122, a reference tissue brightness measurement unit 123, an average value calculation unit 124, a brightness measurement unit 125, a moving average calculation unit 126, and a moving average slope calculator 127 .

The scanning information acquisition unit 121 is connected to the mask processing unit 110, the luminance measurement unit 125, and the information storage unit 160, scans the mammography image input from the medical imaging apparatus 200, and acquires information on the image. It is for

The scanning information acquisition unit 121 can obtain information on the mammography image before mask processing at an arbitrary position and in an arbitrary range based on the position information on the mammography image before mask processing that is input by an input means (not shown).

In addition, the unit of scanning by the scanning information acquisition unit 121 can be set arbitrarily. For example, the mammography image before mask processing can be scanned pixel by pixel. It is also possible to obtain information on the mammography image before masking while scanning. Note that the present embodiment will be described assuming that the scanning information acquisition unit 121 scans in units of pixels.
Information on the mammography image before mask processing, which the scanning information acquiring unit 121 acquires while scanning, is transmitted to the luminance measuring unit 125 .

The luminance measurement unit 125 is connected to the scanning information acquisition unit 121, the moving average calculation unit 126, and the information storage unit 160, and measures the luminance from the information on the mammography image before mask processing received from the scanning information acquisition unit 121. It is for

The luminance information measured by the luminance measuring section 125 is stored in the information storage section 160 . Further, when the processing of the luminance measurement unit 125 is completed, the luminance measurement unit 125 notifies the connected moving average calculation unit 126 of the completion of the processing.

The moving average calculation unit 126 is connected to the luminance measurement unit 125, the moving average inclination calculation unit 127, and the information storage unit 160, and acquires the luminance measured by the luminance measurement unit 125 while the scanning information acquisition unit 121 moves. This is for calculating moving average information in which the average value for each fixed section of information to be obtained is obtained while shifting the section.

As a result, it is possible to draw a smooth curve graph representing the luminance when the information obtained by the scanning information acquiring unit 121 and the luminance measuring unit 125 is graphed. The moving average information calculated by the moving average calculation unit 126 is stored in the information storage unit 160 . Further, when the processing of the moving average calculation unit 126 is completed, the moving average calculation unit 126 notifies the connected moving average slope calculation unit 127 that the processing has been completed.

The moving average slope calculation unit 127 is connected to the moving average calculation unit 126, the boundary detection unit 122, and the information storage unit 160, and calculates the rate of change of the moving average line from the moving average information calculated by the moving average calculation unit 126. is for calculating the slope of

Specifically, the moving average slope calculating unit 127 calculates the rate of change of the moving average line by differentiating the moving average information based on the luminance information at a certain position of the mammography image calculated by the moving average calculating unit 126. be able to.

The slope information of the moving average line calculated by the moving average slope calculator 127 is stored in the information storage unit 160 . Further, when the processing of the moving average slope calculation unit 127 is completed, the moving average slope calculation unit 127 notifies the connected boundary detection unit 122 that the processing has been completed.

The boundary detection unit 122 is connected to the moving average slope calculation unit 127, the reference tissue luminance measurement unit 123, and the information storage unit 160, and obtains the slope information of the moving average line calculated by the moving average slope calculation unit 127 and the brightness measurement. This is for detecting the tissue boundary from the luminance information calculated by the unit 125 .

Specifically, in the chest imaged in mammography screening, the pectoralis major muscle, fat tissue, and skin tissue overlap in order from the inside of the body. The pectoralis major muscle is drawn white because it absorbs X-rays, whereas adipose tissue is drawn black because it transmits X-rays.

The scanning information acquisition unit 121 scans the positions where the pectoralis major muscle, the fat tissue, and the skin tissue overlap in order, the brightness measurement unit 125 measures the brightness, and the moving average calculation unit 126 calculates the moving average of the brightness. , the moving average slope calculator 127 calculates the slope of the moving average line, thereby obtaining the rate of change in luminance. From the difference in the rate of change, the boundaries with the pectoralis major muscle, adipose tissue, and skin tissue can be obtained.

More specifically, according to the inclination information of the moving average line calculated by the moving average inclination calculating unit 127, the point where the inclination first becomes 0 or more among the inclinations measured from the body side of the mammography image is the pectoralis major muscle. The boundary detection unit 122 detects the boundary of adipose tissue.

Further, as in the first embodiment, on the line graphing the luminance calculated by the luminance measuring unit 125, the point where the luminance becomes 0 when measured from the body side of the mammography image is the difference between the skin tissue and the background. The boundary detection unit 122 detects the boundary. As a result, it is possible to identify the reference adipose tissue site sandwiched between the pectoralis major muscle and the skin tissue. When the boundary detection unit 122 detects the boundary between the pectoralis major muscle and the fat tissue and the boundary between the skin tissue and the background, the boundary detection unit 122 transmits the position information to the reference tissue luminance measurement unit 123 .

The reference tissue luminance measurement unit 123 is connected to the boundary detection unit 122, the average value calculation unit 124, and the information storage unit 160. This is for calculating the brightness of the adipose tissue from the boundary information between the background and the background.

The luminance information measured by the reference tissue luminance measurement unit 123 is stored in the information storage unit 160 . Further, when the processing of the reference tissue brightness measurement unit 123 is completed, the reference tissue brightness measurement unit 123 notifies the connected average value calculation unit 124 of the completion of the processing.

The average value calculation unit 124 is connected to the reference tissue luminance measurement unit 123 , the information storage unit 160 , and the evaluation range luminance calculation unit 130 , and calculates fat by the processing of the reference tissue luminance measurement unit 123 stored in the information storage unit 160 . It is for calculating the average value of the brightness of adipose tissue based on the brightness of the tissue.

Upon receiving the notification that the reference tissue luminance measurement unit 123 has completed the processing, the average value calculation unit 124 calculates the luminance of adipose tissue in the specific range stored in the information storage unit 160, specifically, the pectoralis major muscle and the adipose tissue. and the boundary between the skin tissue and the background are called from the information storage unit 160, the average value is calculated from these luminances, and the calculated average value of the luminance of the adipose tissue is stored as information. Stored in unit 160 . Further, when the processing of the average value calculation unit 124 is completed, the average value calculation unit 124 notifies the connected evaluation range luminance calculation unit 130 that the processing has been completed.

In this way, the reference tissue brightness calculator 120 measures the brightness of adipose tissue at an arbitrary position where a large amount of reference adipose tissue exists, and uses the average value as a reference. can also be based on the average value.

In addition, the processing of the scanning information acquisition unit 121, the boundary detection unit 122, the reference tissue luminance measurement unit 123, the average value calculation unit 124, the luminance measurement unit 125, the moving average calculation unit 126, and the moving average inclination calculation unit 127 is performed as necessary. Accordingly, the scanning information acquiring unit 121 can repeat the processing while changing the scanning position.

Specifically, after the first measurement and calculation of the average value, the scanning information acquisition unit 121 moves downward by about 1 to 50 pixels below the breast, and the luminance measurement unit 125 and the moving average calculation unit 126 , the moving average slope calculator 127, and the boundary detector 122 detect the boundary between the pectoralis major muscle and the fat tissue and the boundary between the skin tissue and the background, and calculate the brightness of the fat tissue sandwiched between the boundaries. is measured by the reference tissue luminance measurement unit 123, and the average value calculation unit 124 repeats the process of calculating the average value of the luminance a plurality of times such as 100 times, and the repeatedly calculated average values are further averaged to become the reference. It can also be stored in the information storage unit 160 as the luminance of adipose tissue.

FIG. 17 is a mammography image showing positions scanned by the scanning information acquiring unit.
As shown in FIG. 17, a mammography image captured in a mammography examination is drawn with the pectoralis major muscle, fat tissue, and skin tissue overlapping in order from the inside of the body.

The reference tissue brightness calculation unit 120 scans the overlapping region of the pectoralis major muscle, the fat tissue, and the skin tissue in this way and calculates the brightness, so that the boundary between the pectoralis major muscle and the adipose tissue, and the skin tissue. A boundary with the background can be detected, and a reference adipose tissue site sandwiched between the boundaries can be specified.

FIG. 18 is a diagram showing a graph visualizing the luminance measured by the luminance measuring unit and a moving average line visualizing the moving average information calculated by the moving average calculating unit.
As shown in FIG. 18, the brightness measured by the brightness measurement unit 125 is indicated by the solid line in FIG. Eventually, the adipose tissue becomes a background with a brightness of 0.

The moving average line based on the moving average information calculated by the moving average calculating unit 126 is indicated by the dotted line in FIG. 18, and is represented by a smooth curve. The muscle side is represented by a high brightness, and as the brightness decreases, it becomes adipose tissue, and eventually the adipose tissue becomes a background with a brightness of 0.

FIG. 19 is a graph of the slope information of the moving average calculated by the moving average slope calculator.
As shown in FIG. 19, the slope information of the moving average line calculated by the moving average slope calculator 127 is displayed by the solid line in FIG. At , the slope changes around 0, and eventually converges to a constant value of 0 when it moves to the background.

The boundary detection unit 122 determines that the slope information obtained by differentiating the moving average line that first becomes 0 or more when measured from the pectoralis major muscle side is the boundary between the pectoralis major muscle and the fat tissue.
In the first embodiment and the second embodiment, the image processing apparatus 100 of the present invention measures the luminance ratio of the evaluation range in which the mammary gland tissue exists based on the adipose tissue from the mammography image. Various physical conditions can be quantified by changing the reference cells and the tissues to be evaluated.

For example, in the case of a fatty liver test, the luminance near the liver, for example, the kidney, which does not change much, is measured as a reference cell, and the percentage of fat in the entire liver after masking is calculated. It becomes possible to objectively evaluate the evaluation of fatty liver, which used to be a difficult judgment.

In addition, the reference tissue brightness calculation unit 120 specifies a reference tissue such as the diagnosis of metabolic syndrome, the progress of cerebral infarction in brain cells, calculates the brightness, and the mask processing unit 110 performs mask processing. By relatively comparing the brightness of the target tissue, the ambiguous evaluation can be objectively quantified and evaluated.

In the present embodiment, mammography images obtained by mammography screening are treated as medical images and processed by the image processing apparatus 100 of the present invention. However, medical images are not limited to mammography images. It is also possible to process images that have been visualized using endoscopic examinations and examination images that have been photographed under visible light, such as by endoscopic examinations and fundus cameras.

100 video processing device 110 mask processing unit 111 boundary determination unit 112 image processing unit 113 learning unit 114 learning information input means 120 reference tissue luminance calculation unit 121 scanning information acquisition unit 122 boundary detection unit 123 reference tissue luminance measurement unit 124 average value calculation unit 125 luminance measurement unit 126 moving average calculation unit 127 moving average inclination calculation unit 130 evaluation range luminance calculation unit 131 evaluation range scanning information acquisition unit 132 evaluation range luminance measurement unit 140 reference luminance ratio calculation unit 141 relative luminance conversion processing unit 142 relative luminance ratio Calculation unit 150 State determination unit 160 Information storage unit 161 Pre-masking image storage unit 162 Post-masking image storage unit 163 Reference tissue brightness storage unit 164 Evaluation range brightness storage unit 165 Relative brightness ratio storage unit 166 Criteria storage unit 167 Boundary identification Information storage unit 200 Medical imaging device 300 Display means

Claims (11)

In an image processing device that calculates the composition ratio of high-density mammary gland tissue in an evaluation range for evaluating the condition in an input medical image,
an adipose tissue brightness calculation means for calculating adipose tissue brightness, which is the brightness of the adipose tissue in the medical image ;
evaluation range luminance calculation means for sequentially scanning the evaluation range with a scanning frame in units of pixels , and calculating evaluation range luminance, which is the luminance of each pixel in the evaluation range scanned by the scanning frame;
relative brightness conversion means for converting the evaluation range brightness of each pixel in the evaluation range scanned by the scanning frame into a relative brightness, which is a relative value of each pixel, using the adipose tissue brightness as a reference;
relative luminance class division means for dividing each relative luminance converted by the relative luminance conversion means into relative luminance classes that are an arbitrary uniform range of relative luminance;
The high-density mammary gland in the evaluation range according to the ratio of the number of pixels with a relative luminance equal to or higher than a preset threshold class in the relative luminance classes divided by the relative luminance class dividing means to the number of pixels in the entire relative luminance class. high-concentration mammary gland tissue ratio calculating means for calculating the ratio of tissue;
A video processing device comprising: The high-concentration mammary gland tissue ratio calculation means includes:
Visualizing the evaluation range in the medical image with different colors according to the relative brightness existing in each of the divided relative brightness classes;
2. The video processing apparatus according to claim 1 , characterized by: The adipose tissue is adipose tissue in the breast, and the evaluation range is a range in which mammary gland tissue exists in the breast,
The adipose tissue brightness calculation means includes:
Boundary detection means for detecting a boundary between the inside of the breast and the outside of the breast;
Predetermined range luminance calculation means for calculating the luminance of a plurality of locations in a predetermined range region where the fatty tissue exists inside the breast from the boundary;
Predetermined range part luminance average value calculation means for calculating an average value of luminance of a plurality of points in the predetermined range part calculated by the predetermined range luminance calculation means;
with
making the average value of the brightness of a plurality of locations in the predetermined range site calculated by the predetermined range site brightness average value calculation means the adipose tissue brightness;
2. The video processing apparatus according to claim 1, characterized by: The predetermined range luminance calculation means is
calculating the brightness of the fatty tissue found above the breast and inside the skin;
4. The video processing apparatus according to claim 3 , characterized by: A mask processing unit that performs mask processing to exclude areas other than the evaluation range,
The video processing device according to claim 1, characterized by comprising: The mask processing unit
a learning unit that learns a range to be masked based on an input learning image; a range determination unit that determines an evaluation range based on the information learned by the learning unit;
6. The video processing apparatus according to claim 5 , comprising: The learning unit
Through deep learning using a neural network, learn the classification process that identifies only a certain range from the medical video for learning.
7. The video processing apparatus according to claim 6 , wherein: The identification process includes:
Image region extraction processing using a model of U-net structure,
8. The video processing apparatus according to claim 7 , wherein: a state judgment unit that judges the state of the evaluation range based on the ratio of the high-concentration mammary gland tissue in the evaluation range calculated by the high-concentration mammary gland tissue ratio calculating means;
The video processing device according to claim 1, characterized by comprising: In an image processing method for calculating the composition ratio of high-density mammary gland tissue in an evaluation range for evaluating a condition in an input medical image,
a step of calculating an adipose tissue luminance, which is the luminance of the adipose tissue in the medical image, by an adipose tissue luminance calculation means;
a step of sequentially scanning the evaluation range with a pixel-based scanning frame, and calculating evaluation range brightness, which is the brightness of each pixel in the evaluation range scanned by the scanning frame, by an evaluation range brightness calculation means;
a step of converting the evaluation range luminance of each pixel in the evaluation range scanned by the scanning frame into a relative luminance, which is a relative value thereof, by a relative luminance conversion means, using the adipose tissue luminance as a reference;
a step of dividing each relative luminance converted by the relative luminance conversion means into relative luminance classes that are an arbitrary uniform range of relative luminance, by a relative luminance class dividing means;
The high-density mammary gland in the evaluation range according to the ratio of the number of pixels with a relative luminance equal to or higher than a preset threshold class in the relative luminance classes divided by the relative luminance class dividing means to the number of pixels in the entire relative luminance class. a step of calculating the ratio of the tissue by the high-concentration mammary gland tissue ratio calculating means;
A video processing method characterized by comprising: In an image processing program that calculates the composition ratio of high-density mammary gland tissue in the evaluation range for evaluating the condition in the input medical image,
the computer,
an adipose tissue brightness calculation means for calculating adipose tissue brightness, which is the brightness of the adipose tissue in the medical image ;
Evaluation range luminance calculation means for sequentially scanning the evaluation range with a scanning frame in units of pixels , and calculating evaluation range luminance, which is the luminance of each pixel in the evaluation range scanned by the scanning frame;
relative brightness conversion means for converting the evaluation range brightness of each pixel in the evaluation range scanned by the scanning frame into a relative brightness, which is a relative value of each pixel, using the adipose tissue brightness as a reference;
Relative luminance class division means for dividing each relative luminance converted by the relative luminance conversion means into relative luminance classes that are an arbitrary uniform range of relative luminance;
The high-density mammary gland in the evaluation range according to the ratio of the number of pixels with a relative luminance equal to or higher than a preset threshold class in the relative luminance classes divided by the relative luminance class dividing means to the number of pixels in the entire relative luminance class. high-concentration mammary gland tissue ratio calculating means for calculating the tissue ratio;
A video processing program characterized by functioning as a

Images