KR20230033165A - Apparatus and method for automatically fitting and displaying region of interest in medical image to window - Google Patents

Abstract

The present invention relates to a device and method for automatically fitting a region of interest in a medical image to a window and displaying the same. When the size of an object of interest is significantly small compared to a full-sized medical image, a region of interest including the object of interest is displayed by fitting a window to the center and the size of the window. When the medical image includes a plurality of regions of interest, the medical image is separated by the number of regions of interest. By displaying each different region of interest in the separated medical image by fitting the center and the size to a separate window, convenience can be provided to a user.

Description

Apparatus and method for automatically fitting and displaying a region of interest in a medical image to a window

The present invention relates to an apparatus and method for automatically fitting and displaying a region of interest in a medical image to a window, and more particularly, when the size of an object of interest is significantly smaller than that of a full-size medical image, the corresponding object of interest The included region of interest is fitted and displayed according to the center and size of the window. If the medical image includes a plurality of regions of interest, the medical image is separated by the number of regions of interest, and each different region of interest of the separated medical images is placed at the center of the region. An apparatus and method for providing convenience to a user by fitting and displaying a size and a size in separate windows, respectively.

Due to the recent rapid development of medical technology and medical imaging apparatuses, it has become possible to obtain high-precision medical images for confirming a disease or condition of a patient.

These medical images are acquired through various medical imaging devices such as CT (computed tomography) imaging devices, MRI (magnetic resonance imaging) imaging devices, ultrasound imaging devices, etc., and images having various modalities depending on the type of medical imaging device is obtained

In addition, since the medical image visualizes the inside of the patient's body, it has the advantage of diagnosing or confirming the disease or condition of the patient through reading the medical image without opening the patient's body.

In order to accurately and accurately read such medical images, the function of a display device displaying the corresponding medical images is very important. Most display devices are implemented to reduce and display medical images according to the size of a window to be displayed.

However, while the full size of the medical image (more specifically, the active area with pixel values) maintains a very high resolution, the window size for displaying the medical image in the display device is relatively small, When a high-resolution medical image is scaled down and displayed according to the size of a window, an object of interest that has a significant effect on the reading of an actual medical image is displayed in a smaller size.

In this case, the user (medical expert such as a doctor or image reader) who reads the medical image has to zoom in on a certain area including the object of interest in the medical image several times, which causes a lot of time loss and requires a lot of reading time. There is a problem that reduces the ability to concentrate.

Accordingly, in the present invention, in a medical image captured by a medical imaging apparatus, a portion in which pixel values are stored according to an active area or an angle of view (FOV) is very large compared to a region of interest including an object of interest, so that a full-size medical image is displayed as a window. In the case of reducing and displaying according to the size, the region of interest is displayed relatively very small, and a method to overcome the inconvenience of the user having to enlarge the corresponding region of interest several times is proposed.

That is, the present invention matches the center of a region of interest included in a medical image with the center of a window and fits the region of interest to the window, thereby enlarging or reducing the medical image so that the region of interest is displayed on the window. It is intended to suggest a way to minimize it.

In addition, according to the present invention, as a result of matching the center of the region of interest and the center of the window, when the size of the region of interest is different from the size of the window, the size of at least one of the width or height of the region of interest is set to at least one of the width or height corresponding to the window. The region of interest is enlarged or reduced according to one size, or by selecting a smaller ratio among the horizontal and vertical ratios of the region of interest and the window, and expanding or reducing the region of interest according to the selected ratio. We would like to propose a method of fitting to the window.

Next, the prior art existing in the technical field of the present invention will be briefly described, and then the technical details to be achieved by the present invention to be differentiated from the prior art will be described.

First, Korean Patent Publication No. 2017-0126773 (November 20, 2017) relates to a method for outputting an ultrasound image and an ultrasound diagnosis apparatus, and includes a first text and a plurality of icons representing information on a region of interest included in an object of an ultrasound image. The present invention relates to a method for generating and displaying an ultrasound image and displaying second text corresponding to the icon in an area related to information indicated by the second text when a user selects the icon, and an ultrasound diagnosis apparatus.

That is, in Korean Patent Publication No. 2017-0126773, text indicating which organ or part of an organ is a region of interest included in an ultrasound image is displayed on the corresponding region. However, Korean Patent Publication No. 2017-0126773 displays the full size of the ultrasound image according to the size of the display, so when the size of the ROI is relatively very small compared to the full size of the ultrasound image, the size of the actually displayed ROI There is a problem that can be displayed smaller and smaller.

On the other hand, the present invention significantly improves the readability of the region of interest, which can be displayed very small compared to the full size of the medical image, by fitting the region of interest included in the medical image to the window, and the task for the medical image ( In order to increase the efficiency of reading, etc.), Korean Patent Publication No. 2017-0126773 does not describe these technical features of the present invention.

In addition, Korean Patent Publication No. 2017-0032671 (2017.03.23.) relates to an X-ray imaging device capable of providing an X-ray image displaying pain information, receiving an X-ray image and an infrared thermal image of a subject, and providing an infrared ray image. An X-ray imaging apparatus that detects a pain part of a subject according to the heat distribution of a zero laceration, displays pain information at a position of the X-ray image corresponding to the detected pain part, and provides an X-ray image displaying the pain information. it's about

That is, Korean Patent Publication No. 2017-0032671 (2017.03.23.) simply displays information obtained by infrared thermal imaging on an X-ray image, and displays the region of interest included in the medical image described in the present invention. The region of interest is not displayed on the window by matching the center of the center with the center of the window, and if the size of the region of interest differs from the size of the window during the matching, the region of interest is fitted to the size of the window and displayed. Neither do Therefore, there is a significant difference between Korean Patent Publication No. 2017-0032671 and the present invention.

As described above, most of the prior arts describe only the extent to which a specific medical image is displayed with specific information, and when displayed on a display with a screen size relatively small compared to the full size of the medical image, it has a significant effect on reading. A solution to the problem that the region of interest including the affected object of interest is displayed very small is not proposed at all.

On the other hand, in the present invention, when a photographing target (object of interest) is significantly smaller than the detector size of a medical imaging device or the field of view (FOV) is too large compared to the object of interest, a full-size medical image is reduced to fit the size of the window and displayed. This is to solve the problem of the prior art in which the object of interest is displayed in a very small size. By fitting the region of interest included in the medical image according to the center and size of the window, the region of interest is displayed on the window as the center, and the work related to the medical image is performed. It has the advantage of providing convenience to the user by increasing the efficiency of the device.

The present invention was created to solve the above problems, and displays an area of interest included in a medical image by fitting it to a window, thereby providing an object of interest when displaying a full-size medical image captured through a detector of a medical imaging apparatus. A region of interest including a region of interest is displayed relatively very small so that the user overcomes the inconvenience of repeatedly enlarging the corresponding region and automatically fits and displays the region of interest of a medical image in a window for the purpose of providing convenience to the user. Its object is to provide an apparatus and a method therefor.

Another object of the present invention is to provide an apparatus and method for displaying a center of a region of interest in a medical image on a window by matching and displaying the center of the region of interest with the center of the window.

In addition, when the center of the ROI and the center of the window are matched, the present invention displays the size of the ROI by fitting it to the size of the window when the size of the ROI is smaller or larger than the window size. Its object is to provide a device and method for improving readability.

Another object of the present invention is to provide an apparatus and method for automatically detecting a region of interest including an object of interest included in a medical image, fitting the region of interest to a window, and displaying the same.

In addition, when a medical image includes a plurality of regions of interest, the present invention can improve the readability of each region of interest by dividing the corresponding medical image by the number of regions of interest and fitting and displaying different regions of interest in individual windows. Its object is to provide a device and method for enabling it.

In addition, in the present invention, for the portion of the region of interest that exceeds the window size, when a portion of the window is selected and dragged in a direction including left, right, up, down, or a combination thereof, the excess portion in the corresponding direction is removed during fitting. Its object is to provide a display device and its method.

According to an embodiment of the present invention, an apparatus for automatically fitting and displaying a region of interest of a medical image to a window includes a region of interest detector that detects a region of interest including an object of interest in a medical image, and a window to display the region of interest. It is characterized in that it includes a region-of-interest fitting unit to be fitted and a display unit to display the fitted region-of-interest on the window.

In addition, the region of interest fitting unit matches the center of the region of interest with the center of the window to match the center of the region of interest to the center of the window, and the region of interest position fitting unit matches the center of the region of interest with the center of the window. or a size fitting unit for fitting at least one of the vertical sizes to at least one of the horizontal and vertical sizes of the window, respectively.

In addition, the region of interest fitting unit matches the center of the region of interest with the center of the window to match the center of the region of interest to the center of the window, and the region of interest position fitting unit matches the center of the region of interest with the center of the window. and a region-of-interest size fitting unit that selects a smaller ratio of the horizontal ratio and the vertical ratio of each, enlarges or reduces the region of interest according to the selected ratio, and fits the region of interest.

The apparatus further includes a garbage removal unit that removes a garbage area at an outer edge of a viewing angle of the medical image, wherein the garbage removal unit refers to a specification of a medical imaging device that has captured the medical image. It is characterized in that the garbage area is removed by removing the garbage area or by detecting a rectangular area of a certain size in which the pixel value does not change near the edge.

In addition, the region of interest is detected by referring to annotations, metadata, and user settings related to the region of interest added to the medical image, or is detected through pattern matching or artificial intelligence for the object of interest. .

The apparatus may further include a learning model generation step of generating a learning model by learning a plurality of learning medical images of the object of interest by modality or by object of interest, wherein the medical image is compared to the modality of the medical image and the purpose of use. and detecting the region of interest by detecting at least one object of interest by applying the generated learning model according to the method.

In addition, a method of automatically fitting and displaying a region of interest of a medical image to a window according to an embodiment of the present invention includes a region of interest detection step of detecting a region of interest including an object of interest in a medical image, and a method for displaying the region of interest. It is characterized by including a fitting step of fitting a region of interest to a window and a display step of displaying the fitted region of interest on the window.

In addition, the fitting of the region of interest may include a position fitting step of matching the center of the region of interest to the center of the window by matching the center of the region of interest with the center of the window, and the positioning of the region of interest based on the matched center. and a region-of-interest size fitting step of fitting at least one of the horizontal and vertical sizes to at least one of the horizontal and vertical sizes of the window, respectively.

In addition, the fitting of the region of interest may include a region of interest position fitting unit that matches the center of the region of interest with the center of the window to align the center of the region of interest with the center of the window, and the region of interest based on the matched center. and a region-of-interest size fitting unit that selects a smaller ratio among the horizontal and vertical ratios of the window, enlarges or reduces the region of interest according to the selected ratio, and fits the region of interest.

The method may further include a garbage removal step of removing a garbage area at an outer edge of a viewing angle of the medical image, wherein the garbage removal step refers to a specification of a medical imaging device that captures the medical image. to remove the garbage area, or by detecting a rectangular area of a certain size in which a pixel value does not change near the edge to remove the garbage area.

The method further includes a learning model generating step of generating a learning model by learning a plurality of medical images for training of the object of interest by modality or by object of interest, and using the medical image with the modality of the medical image. It is characterized in that it includes detecting the region of interest by detecting at least one or more objects of interest by applying the generated learning model according to a purpose.

As described above, the apparatus and method for automatically fitting and displaying a region of interest of a medical image to a window according to the present invention detects a region of interest of a medical image including an object of interest, and adjusts the region of interest to the center and size of the window. By fitting and displaying the medical image, it is possible to prevent a user from unnecessarily enlarging or reducing a medical image, thereby reducing time loss for the user.

That is, according to the present invention, as the user does not have to perform unnecessary operations such as enlargement or reduction, the speed of work can be increased and the degree of concentration of work can be increased so that medical images can be read efficiently.

In addition, the present invention has an effect of providing convenience to a user who reads a medical image by improving readability of the region of interest by fitting and displaying the region of interest according to the window so that the region of interest is displayed on the window.

1 is a diagram for explaining a method of displaying a medical image on a window according to the prior art.
2 is a conceptual diagram illustrating an apparatus and method for automatically fitting and displaying a region of interest of a medical image to a window according to an embodiment of the present invention.
3 is a diagram illustrating a method of displaying a region of interest by fitting it to a window according to an embodiment of the present invention.
4 is a diagram for explaining in detail a process of fitting a medical image to a window according to an embodiment of the present invention.
5 is a diagram for explaining in detail a process of fitting a medical image to a window according to another embodiment of the present invention.
6 is a diagram illustrating a method of displaying a plurality of regions of interest by fitting them to a window according to an embodiment of the present invention.
7 is a diagram for explaining a method of detecting a region of interest according to an embodiment of the present invention.
8 is a diagram for explaining a method of detecting a region of interest according to another embodiment of the present invention.
9 is a diagram shown to explain a method of removing a garbage area according to an embodiment of the present invention.
10 is a block diagram showing the configuration of an apparatus for automatically fitting and displaying a region of interest of a medical image to a window according to an embodiment of the present invention.
11 is a flowchart illustrating a procedure of automatically fitting and displaying a region of interest of a medical image to a window according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of an apparatus and method for automatically fitting and displaying a region of interest in a medical image to a window according to the present invention will be described in detail with reference to the accompanying drawings. Like reference numerals in each figure indicate like members. In addition, specific structural or functional descriptions of the embodiments of the present invention are merely exemplified for the purpose of explaining the embodiments according to the present invention, and unless otherwise defined, all of the technical or scientific terms used herein are included. Terms have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this specification, it should not be interpreted in an ideal or excessively formal meaning. It is preferable not to

1 is a diagram for explaining a method of displaying a medical image on a window according to the prior art.

As shown in FIG. 1, in the prior art of displaying a medical image in a window on a display to be displayed, the full size of the medical image is reduced to match the size of the window.

Meanwhile, although it is shown in FIG. 1 that there is not much difference between the overall size (full size) of the medical image and the window size, the overall size of the medical image is substantially several to several tens of times larger than the size of the window (or display) on the display.

In addition, an object of interest substantially including an abnormal part of a medical image (eg, a lesion such as inflammation or tumor) plays a very important role in reading a medical image.

In this case, the object of interest may be relatively small compared to the overall size of the medical image, and the object of interest may also have a size sufficient for reading.

However, in the case of the prior art, since the entire size of the medical image is reduced and displayed according to the size of the window, there is a problem in that the object of interest is also displayed very small.

In particular, in the case of the prior art, when the imaging target (ie, the object of interest) is significantly smaller than the size (size) of the detector of the medical imaging device, or the field of view (FOV) of the medical imaging device is too large, The object of interest is also displayed very small, which is inconvenient for the user to enlarge the corresponding object repeatedly several times, and there is a problem in that a lot of time is consumed in reading a medical image through the object of interest.

Therefore, the present invention is to overcome the inconveniences and problems of the prior art, rather than reducing and displaying the entire full-size medical image to fit the size of the window, rather than displaying an object of interest having an abnormal area compared to the full-size medical image. In preparation for a case where the size is remarkably small, the region of interest including the object of interest is automatically fitted and displayed according to the center and size of the window so that the region of interest is displayed on the window as the center. Through this, it is possible to provide convenience to the user by preventing repeated enlargement of the region of interest several times.

Hereinafter, the present invention will be described in detail with reference to each drawing.

2 is a conceptual diagram illustrating an apparatus and method for automatically fitting and displaying a region of interest of a medical image to a window according to an embodiment of the present invention.

As shown in FIG. 1, an apparatus 100 (hereinafter referred to as an automatic medical image fitting apparatus) automatically fitting and displaying a region of interest of a medical image to a window according to an embodiment of the present invention is an abnormal region from a medical image. A function of detecting a region of interest including an object of interest having , and automatically fitting and displaying the region of interest to a window of a display to be displayed.

That is, the automatic medical image fitting apparatus 100 fits a region of interest that is significantly smaller than that of a full-size medical image to a window, displays the region of interest in the window, and prevents the user from repeatedly enlarging the region of interest. It provides convenience to (medical experts) and allows them to efficiently perform tasks according to the purpose of using medical images (reading specific lesions on specific body parts).

In addition, the automatic medical image fitting apparatus 100 is implemented as an application program or physical hardware in a user terminal equipped by a user, and performs a function of automatically fitting and displaying a region of interest to a window.

On the other hand, medical images are captured through various medical imaging devices 200 including detectors that detect and image radiation or ultrasound, such as an x-ray imaging device, an ultrasound imaging device, a CT imaging device, etc., and have various modalities. ) is stored in the database 300.

That is, the automatic medical image fitting device 100 is implemented in a user terminal, and displays the area of interest detected from the medical image by fitting it to a window for the purpose of using the medical image (e.g., reading the progress of lung lesions and dialectomy). etc.) to perform tasks according to

On the other hand, the database 300 can be implemented as a picture archiving and communication system (PACS), and is in digital imaging and communication in medicine (DICOM) format, which is a standard for storing, transmitting, and outputting medical images. can be stored In this case, it is natural that the user terminal has a function capable of displaying medical images in DICOM format.

In addition, the automatic medical image fitting apparatus 100 receives a medical image of a specific patient from the database 300, detects a region of interest from the corresponding medical image, and displays the detected region of interest by fitting it to a window.

For example, when a user wants to read a medical image of a specific patient, the user accesses the database 300 through a web server through the automatic medical image fitting device 100 and requests and receives a medical image of the patient. there is.

That is, the database 300 does not simply perform a function of storing and managing medical images, but is equipped with a function as a web server and transmits and outputs medical images like a medical image storage and communication system (PACS). it will be implemented

In addition, the object of interest in the present invention means an abnormal part such as a lesion (eg, inflammation, tumor, cancer, etc.), or an object including the abnormal part (eg, lung, bone, etc.) or including the abnormal part It means a part of an object, and the area of interest means a certain area including the object of interest.

Meanwhile, detecting a region of interest in a medical image will be described with reference to FIGS. 7 and 8 , and fitting and displaying the region of interest in a window will be described with reference to FIGS. 3 to 6 .

Hereinafter, a process of displaying a region of interest by fitting it to a window will be described in detail.

3 is a diagram illustrating a method of displaying a region of interest by fitting it to a window according to an embodiment of the present invention.

As shown in FIG. 3 , when a medical image of a specific patient is input, the apparatus 100 for automatically fitting medical images according to an embodiment of the present invention detects at least one region of interest from the corresponding medical image.

At least one region of interest is detected according to the modality, body part, or purpose of use of the corresponding medical image.

Here, the purpose of use means to read a medical image including whether or not each body part included in the medical image is normal or abnormal, the progress of a disease, and the like. For example, the purpose of use may be to determine whether an abnormal part of the lung (eg, pneumonia, pneumothorax, pleural effusion, lung cancer, etc.) is confirmed and whether it is normal or abnormal.

However, the present invention is not limited to this, and medical images are used for various purposes for each body part (organ), such as reading the condition of a specific bone such as dislocation or bone tumor, or reading the condition of the brain such as brain tumor, brain hemorrhage, etc. can be utilized

In addition, the automatic medical image fitting apparatus 100 removes a garbage area at the outer edge of the viewing angle of the medical image so that the region of interest can be more efficiently fitted to the window and the medical image can be saved.

The garbage area is generated by a difference between an area exposed to radiation or ultrasound of the detector of the medical imaging apparatus 200 and an active area of the detector or a part where actual pixel values are stored according to the viewing angle.

At this time, the garbage area is a part of the medical image that is not required for reading at all, and is a rectangular area (black area) of a certain size in which the pixel value does not change at the outer edge of the medical image. It takes up a lot of space when storing .

Accordingly, the apparatus 100 for automatically fitting medical images can implement an eco-friendly ecosystem for storing medical images by reducing storage space by removing garbage areas and then storing the corresponding medical images to reduce power consumption required for storage.

In addition, the automatic medical image fitting apparatus 100 matches the center of the ROI with the center of the window, fits the center of the ROI to the center of the window, and displays the fitted ROI on the window. enhances readability.

Meanwhile, since the size of an actual medical image (A X B) is several to several tens of times larger than the size of a window (A' X B'), the size (size) of the region of interest may also be substantially similar to the size (size) of the window.

Accordingly, reading of the ROI can be performed sufficiently by fitting the center of the ROI detected in the medical image to the center of the window and displaying the result.

However, when the center of the region of interest is fitted to the center of the window, the size of the region of interest is smaller or larger than a predetermined ratio compared to the size of the window, preventing the user from expanding or reducing the region of interest. To do this, the automatic medical image fitting apparatus 100 further performs an additional process of fitting the size of the ROI according to the size of the window, thereby preventing the user's unnecessary action for the enlargement or reduction.

Here, fitting the size of the region of interest to the size of the window should not be understood as fitting the size of the region of interest to the size of the window, and the process of fitting the size of the region of interest to the size of the window is shown in FIG. And with reference to Figure 5 will be described in detail.

In addition, when a plurality of regions of interest are detected in the medical image, the automatic medical image fitting apparatus 100 separates the medical images as many as the number of regions of interest, and fits different regions of interest in each of the separated medical images to separate windows. can be displayed to increase the readability of individual areas of interest. Meanwhile, fitting a plurality of regions of interest to individual windows will be described in detail with reference to FIG. 6 .

Hereinafter, a process of fitting a region of interest to a window will be described in detail.

4 is a diagram for explaining in detail a process of fitting a medical image to a window according to an embodiment of the present invention.

As shown in FIG. 4 , the apparatus 100 for automatically fitting a medical image according to an embodiment of the present invention displays a region of interest detected from the medical image by fitting it to a window when a medical image is input.

In this case, the automatic medical image fitting apparatus 100 may remove the garbage area from the medical image, and the size (size) of the medical image is several to several tens of times greater than the size (size) of the window, as described above.

In addition, the automatic medical image fitting apparatus 100 performs position fitting of the region of interest by matching the center of the region of interest of the medical image with the center of the window to fit the position of the center of the region of interest to the center of the window.

The position fitting means fitting the center of the ROI to the center of the window according to a result of matching the center of the ROI with the center of the window. Also, the center of the region of interest is a pixel positioned at the center according to pixel coordinates of the region of interest.

In addition, the automatic medical image fitting apparatus 100 performs size fitting of a region of interest by fitting at least one of the horizontal and vertical sizes of the region of interest to at least one of the horizontal and vertical sizes of the window based on each matched center.

That is, the automatic medical image fitting apparatus 100 fits at least one of the horizontal or vertical sizes having a small ratio among the horizontal or vertical ratios of the region of interest to at least one of the horizontal or vertical sizes corresponding to the window.

The ROI size fitting is performed when the total size of the ROI relative to the total size of the window is smaller or larger than a predetermined ratio.

That is, as a result of matching the center of the region of interest and the center of the window, the automatic medical image fitting apparatus 100 selects the width or height of the region of interest having the smallest size among the horizontal or vertical sizes of the region of interest, and displays the window. Size fitting of the region of interest is performed by enlarging or reducing the size according to the corresponding horizontal or vertical size.

At this time, the automatic medical image fitting apparatus 100 determines that the remaining horizontal or vertical (ie, large horizontal or vertical) size of the ROI corresponds to the small horizontal or vertical size of the window when performing ROI size fitting. According to the horizontal or vertical size, the fitting is enlarged or reduced in proportion to the enlarged or reduced ratio.

This is because, when fitting the size of the ROI, if the size ratio to the original ROI is changed, the shape of the ROI itself may be changed and distorted, which may affect accurate reading. Accordingly, the present invention performs the size fitting of the ROI to have the same ratio to the size of the actual ROI in order to prevent the distortion.

For example, assuming that the horizontal and vertical sizes of the ROI are 4:3 and the horizontal and vertical sizes of the window are 30:40, the vertical size of the ROI becomes a small size. At this time, if the vertical size of the region of interest is fitted according to the vertical size corresponding to the window, the vertical size of the region of interest is enlarged by 10.33 times and the horizontal size of the remaining regions of interest is enlarged by 7.5 times.

Of course, the actual region of interest has a relatively small size compared to a full-size medical image, but there may be no significant difference from the actual window size. Accordingly, it is sufficient to only perform ROI position fitting with respect to the ROI.

However, in the present invention, when the size of the region of interest is larger or smaller than the window size by a preset ratio or more when performing the position fitting of the region of interest, size fitting of the region of interest is performed to further improve the readability of the region of interest. is to let

Also, when performing the position fitting of the region of interest, enlargement or reduction of the region of interest is performed only for the region of interest and not for the entire medical image. That is, the full size of the medical image is maintained.

In addition, the final result of displaying the ROI in the window by performing the ROI position fitting and the ROI size fitting indicated at the lower right of FIG. 4 is shown.

Meanwhile, when enlarging or reducing, a portion of a large size of a region of interest in the horizontal or vertical direction may exceed the window size. At this time, the automatic medical image fitting apparatus 100 handles a large portion of the region of interest in the horizontal or vertical direction exceeding the displayed region in the window and being covered when the region of interest is enlarged or reduced.

At this time, the automatic medical image fitting apparatus 100 selects an area within the window (holding a mouse click or holding a touch) and drags in a direction including up, down, left, right, or a combination thereof, in the dragged direction. Let the part that exceeds the window be displayed.

5 is a diagram for explaining in detail a process of fitting a medical image to a window according to another embodiment of the present invention.

As shown in FIG. 5 , when the automatic medical image fitting apparatus 100 according to another embodiment of the present invention fits a region of interest detected from a medical image to a window, the center of the region of interest and the center of the window Position fitting of the region of interest is performed by matching the position of the center of the region of interest to the center of the window. At this time, position fitting of the region of interest is performed through the same process as in FIG. 4 .

In addition, the automatic medical image fitting apparatus 100 selects a smaller ratio between the horizontal ratio and the vertical ratio of the region of interest based on each matched center, and selects a smaller ratio between the horizontal ratio and the vertical ratio of the window, and selects each of the selected ratios. Region-of-interest size fitting is performed to enlarge or reduce the entire region of interest according to the ratio.

That is, as in the ROI size fitting described in FIG. 4, after enlarging or reducing the smaller of the width or height of the ROI to fit the horizontal or vertical size corresponding to the window, the horizontal or vertical size of the remaining larger size is enlarged or reduced. While enlarging or reducing by the reduced ratio, the ROI size fitting described in FIG. 5 enlarges or reduces the entire ROI according to the selected ratio.

For example, assuming that the horizontal and vertical sizes of the region of interest are 4:3 and the horizontal and vertical sizes of the window are 30:40, the small ratio selected in the region of interest becomes the vertical ratio (ie, 3) , the small ratio selected in the window becomes the ratio of the width (i.e., 3). At this time, enlarging or reducing the entire region of interest according to the selected ratio is performed according to the ratio between the horizontal or vertical size for the small ratio selected in the region of interest and the horizontal or vertical size for the small ratio selected in the window.

That is, in the above example, since the vertical size selected at a small rate in the ROI is 3 and the horizontal size selected at a small percentage in the window is 30, the ratio between the vertical size of the ROI and the horizontal size of the window is 1 :10 becomes. Therefore, the automatic medical image fitting apparatus 100 performs ROI size fitting by enlarging the width and height of the ROI by 10 times (ie, the horizontal and vertical sizes of the ROI change from 4:3 to 40:30). is to carry out

Consequently, even in this case, the ratio of the size of the ROI size for which the ROI size fitting is performed has the same ratio to the size of the actual ROI.

In addition, the lower right corner of FIG. 5 shows the final result of displaying the region of interest in the window, and a case where a part of the region of interest exceeds the region displayed in the window may occur as described above.

6 is a diagram illustrating a method of displaying a plurality of regions of interest by fitting them to a window according to an embodiment of the present invention.

As shown in FIG. 6, the apparatus 100 for automatically fitting a medical image according to an embodiment of the present invention detects a region of interest including an object of interest from a medical image, and when a plurality of regions of interest are detected, the medical image An image is divided as many as the number of regions of interest. The separation is performed by copying medical images as many as the number of regions of interest.

In addition, the automatic medical image fitting apparatus 100 improves the readability of the individual regions of interest by fitting and displaying different regions of interest of the separated medical images in separate windows. Meanwhile, fitting and displaying each of the individual windows is performed through the same process as the method of fitting the region of interest to the window described with reference to FIGS. 3 to 5 .

For example, as shown in FIG. 6, the medical image is an x-ray image of the bones of the patient's feet, and the bone region of the left foot (region of interest first) and the bone region of the right foot (region of interest second) are included. When each region of interest is detected, the automatic medical image fitting apparatus 100 separates the x-ray image into two x-ray images.

In addition, the automatic medical image fitting apparatus 100 displays one of the two separated x-ray images by fitting it to a window centered on the first region of interest, and displaying the other x-ray image centered on the second region of interest. Display each by fitting to a window.

7 is a diagram for explaining a method of detecting a region of interest according to an embodiment of the present invention.

As shown in FIG. 7 , the apparatus 100 for automatically fitting a medical image according to an embodiment of the present invention performs a function of detecting at least one region of interest from a medical image.

Meanwhile, the medical image may be stored by adding information related to the region of interest in advance, and the information may be added by being annotated to one side of the medical image, added as separate metadata, or added by user setting. It may be provided together when the medical image is provided to the automatic medical image fitting device 100 .

Here, the information related to the region of interest includes the type of lesion (eg, pneumonia, lung cancer, etc.), coordinates of the region of interest, and the like.

In addition, the annotation refers to an annotation in which information related to a region of interest is inserted (overlaid) on one side of a medical image and added.

In addition, metadata means that information related to a region of interest is composed of separate data, and user setting means that a user sets the region of interest in a medical image in advance.

Therefore, the automatic medical image fitting apparatus 100 detects the ROI by reading the ROI by referring to annotations, metadata, user settings, or a combination thereof related to the ROI added to the medical image.

That is, if an annotation is added to the medical image, the region of interest is read by referring to the annotation, and if metadata is added, the region of interest is read by referring to the coordinates of the region of interest included in the metadata, and user setting If is added, the ROI is detected by reading the ROI according to the user setting.

8 is a diagram for explaining a method of detecting a region of interest according to another embodiment of the present invention.

As shown in FIG. 8, the automatic medical image fitting apparatus 100 according to an embodiment of the present invention, as described in FIG. At least one or more regions of interest may be detected.

At this time, the automatic medical image fitting device 100 detects the lesion by machine learning a plurality of learning medical images in which lesions (more specifically, regions of interest including objects of interest having lesions) are set for each body part. to create an artificial intelligence learning model for setting the area of interest.

At this time, machine learning is performed by classifying medical images for learning according to modalities and the purpose of using the medical images (ie, reading the type of lesion for each body part), and is performed through a convolutional neural network (CNN). However, in the present invention, it is not limited to CNN, and machine learning can be performed using various learning networks.

That is, as shown in FIG. 8, the artificial intelligence learning model reads the type of lesion for each modality of medical images such as x-ray images and CT images, for each body part included in each medical image, and for each body part. Each of the medical images is created and stored according to the purpose of use.

Meanwhile, machine learning may be performed by the automatic medical image fitting device 100 and stored in the database 300, or may be performed in a separate machine learning server (not shown) and stored in the database 300.

In addition, as shown in FIG. 8, when the modality of a medical image is an x-ray image and the purpose of using the medical image is to read a specific lung lesion (eg, pneumonia), the automatic medical image fitting device 100 The modality is x-ray, the body part is the lung, an artificial intelligence learning model for detecting pneumonia is loaded, and the medical image is input into the loaded artificial intelligence learning model to detect pneumonia (i.e., inflammation), and the detection The region of interest is detected by setting and outputting a region of interest including an object of interest having pneumonia.

That is, the input of the artificial intelligence learning model is a medical image, the output is at least one coordinate (pixel coordinate) of the lesion and the probability of the lesion, and the automatic medical image fitting device 100 of the artificial intelligence learning model A region of interest is detected by detecting at least one lesion by selecting a probability whose output exceeds a preset threshold among probabilities.

In addition, the automatic medical image fitting device 100 inputs the medical images to at least one artificial intelligence learning model corresponding to the modality of the medical image and the body part, and detects an abnormal part (lesion) for each artificial intelligence learning model. A region of interest may be detected by detecting at least one or more lesions.

For example, if the medical image is an x-ray image and the purpose of use is to read a lung lesion, the automatic medical image fitting device 100 has an x-ray image modality and generates lung cancer, pneumonia, Each of the medical images is input to an artificial intelligence learning model for detecting pleural fluid to detect abnormal parts including lung cancer, pneumonia, pleural effusion, or a combination thereof, and regions of interest for each abnormal part are respectively detected.

Meanwhile, the region of interest may be detected through pattern matching with respect to the object of interest, in addition to using an artificial intelligence learning model.

For example, by checking whether or not there is an abnormal part (lesion) through pattern matching between an object of interest for a medical image and a previously stored normal object of interest, a region of interest including the object of interest is set according to the check result. can be detected by

Meanwhile, as described with reference to FIGS. 7 and 8 , it is described that the region of interest is detected through information on the region of interest added to the medical image, an artificial intelligence learning model, or pattern matching, but a key technical feature of the present invention. Since is displayed by fitting the region of interest of the medical image to the window, the method of detecting the region of interest is not limited thereto.

9 is a diagram shown to explain a method of removing a garbage area according to an embodiment of the present invention.

As shown in FIG. 9 , the automatic medical image fitting device 100 according to an embodiment of the present invention receives specifications for the detector of the medical image capture device 200 that has taken a medical image and receives the specifications. A function of detecting and removing the garbage area included in the medical image is performed with reference to .

Meanwhile, when a medical image is stored in the database 300, the specification may be provided from the medical imaging apparatus 200 that captures the medical image and stored together with the medical image, and the medical image may be automatically fitted to the medical image. When provided as (100), it may be provided together.

Meanwhile, when specifications for the detector of the medical imaging apparatus 200 are not provided, the automatic medical image fitting apparatus 100 detects a rectangular area of a certain size in which the pixel value does not change at the outer edge of the medical image, It is also possible to remove a garbage area included in the medical image by removing the rectangular area.

In addition, the medical images removed from the garbage area are stored in the database 300 . At this time, the existing medical images stored in the database 300 are replaced with the medical images removed in the garbage area and stored and managed.

10 is a block diagram showing the configuration of an apparatus for automatically fitting and displaying a region of interest of a medical image to a window according to an embodiment of the present invention.

As shown in FIG. 10, the automatic medical image fitting apparatus 100 according to an embodiment of the present invention includes a user interface providing unit 110, a medical image input unit 120 receiving a medical image of a specific patient, A garbage area removal unit 130 that removes garbage areas included in a medical image, a learning model generator 140, a region of interest detector 150 that detects at least one region of interest including an object of interest from the medical image, The medical image separator 160 separates the medical image according to the result of detecting the region of interest, the region of interest fitting unit 170 fits the window to display the region of interest, and the fitted region of interest is displayed on the window. It is configured to include a display unit 180 for displaying.

In addition, the user interface unit 110 senses a user's manipulation (eg, dragging, etc.) of a window displayed by fitting a region of interest through interaction with the user, and moves and displays the region of interest according to the user's manipulation, or Performs a function that allows the display to be enlarged or reduced.

In addition, the medical image input unit 110 performs a function of receiving a medical image of a specific patient to be used by the user.

At this time, when a user designates a specific patient, the medical image input unit 110 requests a medical image of the patient from the database 300 and receives the medical image of the patient from the database 300 .

Meanwhile, information related to a region of interest may be added to the medical image, and the information is pre-annotated in the medical image, pre-set by a user (user setting), or pre-configured as separate metadata.

In addition, the garbage area removal unit 130 detects and removes a garbage area, which is an unnecessary area formed at the outer edge of the medical image, to perform a fitting and stores it in the database 300 . Meanwhile, since the removal of the garbage area has been described with reference to FIG. 9 , it will be omitted here.

In addition, the learning model generation unit 140 performs a function of generating an artificial intelligence learning model for detecting an object of interest by machine learning a plurality of medical images for learning according to modality, body part, and purpose of use.

As described above, AI learning models are created and managed according to each modality, each body part, and the purpose of use (eg, reading a specific lesion on a specific body part).

Also, the ROI detector 150 performs a function of detecting at least one ROI including an object of interest from the medical image.

Here, the detection of the region of interest is performed by referring to annotations, metadata, or user settings related to the region of interest added to the medical image, or by pattern matching or an artificial intelligence learning model (ie, artificial intelligence) for the object of interest. Detection through is as described above. Meanwhile, since the detection of the ROI has been described with reference to FIGS. 7 and 8 , further detailed description will be omitted.

Also, when the number of regions of interest detected by the region of interest detector 150 is plural, the medical image separator 160 separates medical images by the number of regions of interest detected. As described above, the separation is performed by copying the medical image as many times as the number of the ROI.

In addition, the ROI fitting unit 170 performs a function of fitting a window to display the ROI, and the ROI position fitting unit 171 performs center fitting to align the center of the ROI to the center of the window. and a region of interest size fitting unit 172 that performs size fitting of the region of interest to match the size of the region of interest aligned to the center of the window to the size of the window.

Also, the ROI position fitting unit 171 aligns the center of the ROI to the center of the window by matching the center of the ROI to the center of the window. That is, the ROI position fitting unit 171 fits the position of the ROI included in the medical image according to the window so that the ROI is displayed on the window.

In addition, the ROI size fitting unit 172, as a result of matching the centers, fits the ROI to the window according to the size of the ROI and the size of the window based on the matched centers.

At this time, the region of interest size fitting unit 172, as a result of matching each center, when the size of the region of interest is smaller or larger than a predetermined ratio compared to the size of the window (this is referred to as a size fitting condition), The smaller of the horizontal or vertical sizes of the ROI is enlarged or reduced to fit the horizontal or vertical size corresponding to the window, and the remaining horizontal or vertical size of the ROI (ie, the larger of the horizontal or vertical sizes) is the Fitting by expanding or reducing the small size in proportion to the enlarged or reduced ratio, or selecting a smaller ratio among the horizontal and vertical ratios of the region of interest and the window, and expanding or reducing the region of interest according to the selected ratio fit in At this time, as described above, a part of the region of interest may exceed the window according to the enlargement or reduction.

On the other hand, if the size fitting unit 172 of the region of interest does not satisfy the size fitting condition (ie, if the size of the region of interest as a result of matching and fitting each center falls within a predetermined ratio compared to the size of the window) ), size fitting for the region of interest is not performed.

In addition, when a plurality of regions of interest are detected and the medical images are separated into multiple regions through the medical image separator 160, the region of interest fitting unit 170 displays different regions of interest for the separated plurality of medical images in individual windows. Fit and mark each one. At this time, fitting the individual windows is performed by matching the center of each region of interest to the individual window in the region of interest position fitting unit 171, and matching the center of each region of interest to each window in the region of interest position fitting unit 171. This is performed by adjusting the size of the region of interest to the size of each individual window.

The display unit 180 displays the ROI by displaying a result of fitting the ROI to the window through the ROI fitting unit 170 on the window.

Meanwhile, when the user selects an area of the window through the user interface and drags in a direction including up, down, left, right, or a combination thereof, the display unit 180 detects the drag and performs the drag. It performs a function of displaying a region of interest (or region of a medical image) that exceeds the window in the performed direction.

11 is a flowchart illustrating a procedure of automatically fitting and displaying a region of interest of a medical image to a window according to an embodiment of the present invention.

As shown in FIG. 11, in the procedure of automatically fitting and displaying a region of interest of a medical image to a window according to an embodiment of the present invention, first, the automatic medical image fitting apparatus 100 includes the object of interest in the medical image. A region of interest detection step of detecting a region of interest to be detected is performed (S110).

Detecting the region of interest refers to annotations, metadata, or user settings for information related to the region of interest added to the medical image, or reads the object of interest through pattern matching or artificial intelligence for the object of interest to detect the object of interest. Detection by setting the region of interest to include is as described above.

On the other hand, when detecting a region of interest through artificial intelligence, the automatic medical image fitting device 100 performs machine learning on a plurality of medical images for learning and at least one artificial intelligence learning model according to the modality of the medical images, body parts, and purpose of use. Further comprising the step of generating a learning model to generate.

Next, the automatic medical image fitting apparatus 100 performs a region of interest fitting step of fitting the region of interest to a window to be displayed.

The region-of-interest fitting step may include a region-of-interest position fitting step (S120) of matching the center of the region of interest and the center of the window to align the center of the region of interest with the center of the window, and matching the center of the region of interest according to the matching result. Based on the center, match at least one of the horizontal or vertical sizes of the region of interest with at least one of the horizontal or vertical sizes of the window, or select a smaller ratio between the horizontal and vertical ratios of the region of interest and the window, respectively, and select each ratio. A region-of-interest size fitting step (S130) of performing region-of-interest size fitting according to . Since center fitting and ROI size fitting have been described with reference to FIGS. 3 to 5 and 10 , further detailed descriptions will be omitted.

Meanwhile, when a plurality of regions of interest are detected, the automatic medical image fitting apparatus 100 separates as many medical images as the number of regions of interest into a plurality of medical images, and separates different regions of interest from each separated medical image into a plurality of windows. A fitting step is further included, and the step of individually fitting each of the plurality of windows is performed through the same process as steps S120 and S130.

Next, the automatic medical image fitting apparatus 100 performs a display step of displaying the region of interest on the window according to the fitting result through the region of interest fitting step (S140).

Meanwhile, in the displaying step, when a plurality of ROIs are detected and each ROI is fitted to an individual window, each ROI is displayed on an individual window according to a fitting result.

In addition, although not shown in FIG. 10, the automatic medical image fitting apparatus 100 further performs a garbage removal step of detecting and removing a garbage area at the outer edge of the viewing angle of the medical image, and detecting and removing the garbage is shown in FIG. Since it has been described in detail in , it is omitted here.

In this way, the present invention fits and displays the region of interest detected from a medical image to a window to be displayed, so that the active region in the medical image or the part where the pixel value is stored according to the viewing angle of the detector of the medical imaging apparatus is much larger than the object of interest. Compared to a large, full-size medical image, when the region of interest including the object of interest is very small, the user is prevented from having to repeatedly enlarge the corresponding region of interest, thereby providing convenience to the user.

As described above, the present invention has been described with reference to the embodiments shown in the drawings, but this is only exemplary, and various modifications and other equivalent embodiments will be made by those skilled in the art in the field to which the technology belongs. You will understand that it is possible. Therefore, the technical protection scope of the present invention will be determined by the claims below.

100: Device for automatically fitting and displaying a region of interest in a medical image to a window
110: user interface providing unit 120: medical image input unit
130: garbage area removal unit 140: learning model generation unit
150: region of interest detection unit 160: medical image separation unit
170: region of interest fitting unit 171: region of interest position fitting unit
172: region of interest size fitting unit 180: display unit
200: medical imaging device 300: database

Claims (14)

a region of interest detector detecting a region of interest including an object of interest in a medical image;
a region of interest fitting unit fitting a window to display the region of interest; and
and a display unit to display the fitted region of interest on the window. The method of claim 1,
The region of interest fitting unit,
and a region-of-interest size fitting unit that fits at least one of the horizontal and vertical sizes of the region of interest to at least one of the horizontal and vertical sizes of the window, respectively; automatically sets the region of interest of the medical image to the window. A fitting display device. The method of claim 1,
The region of interest fitting unit,
and a region-of-interest size fitting unit that selects a smaller ratio between the horizontal and vertical ratios of the region of interest and the window, and enlarges or reduces the region of interest according to the selected ratio to fit the region of interest. A device that automatically fits and displays a region of interest to a window. The method of claim 1,
The region of interest fitting unit,
When there are a plurality of regions of interest, the method further comprising dividing the medical images into a plurality of pieces and fitting and displaying different regions of interest included in the separated medical images in a plurality of windows, respectively. A device that automatically fits and displays a region of interest to a window. The method of claim 1,
The device,
A garbage removal unit for removing a garbage area at an outer edge of the viewing angle of the medical image;
The garbage removal unit,
The garbage area is removed by referring to the specifications of the medical imaging apparatus that captured the medical image;
An apparatus for automatically fitting and displaying a region of interest of a medical image to a window, characterized in that by detecting a rectangular region of a certain size in the vicinity of the edge where the pixel value does not change, and removing the garbage region. The method of claim 1,
The area of interest is
Detected with reference to annotations, metadata, and user settings related to the region of interest added to the medical image;
An apparatus for automatically fitting and displaying a region of interest in a medical image to a window, characterized in that it is detected through pattern matching or artificial intelligence for the object of interest. The method of claim 1,
The device,
It further includes a learning model generation unit for generating a learning model by performing machine learning on the plurality of training medical images for the object of interest for each modality or each object of interest,
detecting the region of interest by detecting at least one object of interest by applying the medical image to the generated learning model according to the modality of the medical image and the purpose of use of the medical image; A device that automatically fits and displays. a region of interest detection step of detecting a region of interest including an object of interest in a medical image;
a region of interest fitting step of fitting the region of interest to a window to be displayed; and
and displaying the fitted region of interest on the window. The method of claim 8,
The region of interest fitting step,
and automatically fitting the region of interest of the medical image to the window by matching at least one of the horizontal and vertical sizes of the region of interest with at least one of the horizontal and vertical sizes of the window. How to display by fitting. The method of claim 8,
In the region of interest fitting step,
and a region-of-interest size fitting step of selecting a smaller ratio between the horizontal and vertical ratios of the region of interest and the window, and enlarging or reducing the region of interest according to the selected ratio. A method of automatically fitting and displaying a region of interest to a window. The method of claim 9,
In the region of interest fitting step,
When there are a plurality of regions of interest, the method further comprising dividing the medical images into a plurality of pieces and fitting and displaying different regions of interest included in the separated medical images in a plurality of windows, respectively. A method of automatically fitting and displaying a region of interest to a window. The method of claim 8,
The method,
A garbage removal step of removing a garbage area at an outer edge of the viewing angle of the medical image;
The garbage removal step,
The garbage area is removed by referring to the specifications of the medical imaging apparatus that captured the medical image;
The method of automatically fitting and displaying a region of interest in a medical image to a window, characterized in that by detecting a rectangular region of a certain size in the vicinity of the edge where the pixel value does not change, and removing the garbage region. The method of claim 8,
The area of interest is
Detected with reference to annotations, metadata, and user settings related to the region of interest added to the medical image;
A method of automatically fitting and displaying a region of interest of a medical image to a window, characterized in that it is detected through pattern matching or artificial intelligence for the object of interest. The method of claim 7,
The method,
Further comprising a learning model generating step of generating a learning model by performing machine learning on a plurality of training medical images of the object of interest for each modality or for each object of interest,
detecting the region of interest by detecting at least one object of interest by applying the medical image to the generated learning model according to the modality of the medical image and the purpose of use of the medical image; How to display by automatically fitting to .

Images