JP6785990B2 - Medical image processing equipment and endoscopic equipment - Google Patents

Description

The present invention relates to a medical image processing device and an endoscopic device that use the analysis results of medical images.

Conventionally, among devices related to medical treatment (hereinafter referred to as medical devices), those for acquiring an image including a subject (hereinafter referred to as a medical image) present the acquired medical image to a doctor. Then, the doctor uses the medical image obtained from the medical device as one of the judgment materials to make a diagnosis or the like. As a matter of course, the discrimination of the condition of the subject using the medical image at the time of diagnosis is based on the skill and experience of the doctor.

In recent years, since image analysis technology has advanced, it is possible to obtain objective information or quantitative information from medical images by analyzing medical images. For this reason, the number of medical devices that support discrimination, diagnosis, and the like by presenting the analysis results of medical images to doctors and the like is increasing. For example, the endoscope device described in Patent Document 1 is abnormal by using a fluorescence image taken by fluorescence or a narrow band optical image taken by using light having a specific narrow wavelength band (so-called narrow band light). Identify the location of an area. Then, the position of the region where the abnormality exists is displayed on the endoscopic image for display.

Japanese Unexamined Patent Publication No. 2006-198106

In a conventional medical device, a notable area (so-called area of interest, area of interest) including one or more parts (lesion part or part having a possibility of lesion) in which an abnormality such as a lesion is observed by analyzing a medical image is included. , Or an area called an abnormal area, etc., hereinafter referred to as an abnormal area), and presenting the position or the like to support diagnosis or the like. Then, in the detection of the abnormal region, it is usual to use a medical image taken under specific imaging conditions in which it is easy to detect the type of lesion to be detected. That is, in the conventional medical device, the abnormal region is detected on the premise that a specific lesion or the like is easily detected under a specific imaging condition. However, the lesions and the like may not be accurately discriminated because the imaging conditions that can be observed well differ depending on the type or properties (progression, etc.) of the lesions and the type of the organ having the lesions and the like.

An object of the present invention is to provide a medical image processing device and an endoscopic device capable of discriminating an abnormal region more accurately than before.

The medical image processing apparatus of the present invention includes a medical image acquisition unit that acquires a medical image including a subject, a first abnormal area determination unit that determines an abnormal region for each medical image using a plurality of medical images, and a medical image. A medical image obtained by photographing a subject using a characteristic discriminator that discriminates the characteristics of the abnormal region using the discrimination result of each abnormal region and an illumination light having a specific spectrum controlled according to the characteristics of the abnormal region. It is provided with a second abnormal area discriminating unit for discriminating an abnormal region.

The first abnormal region discrimination unit preferably uses a medical image taken by using special light having a spectrum different from that of white light to discriminate the abnormal region.

As a characteristic of the abnormal region, the characteristic discriminating unit preferably discriminates the depth from the surface of the object of the abnormality in the abnormal region.

It is preferable that the characteristic discrimination unit discriminates the characteristics of the abnormal region by using the accuracy of the discrimination result by the first abnormal region discriminating unit.

It is preferable that the second abnormal region discrimination unit determines a medical image used for discriminating the abnormal region by using a predetermined correspondence relationship between the characteristics of the abnormal region and the specific spectrum.

It is preferable that the second abnormal region determination unit calculates a specific spectrum by using the characteristics of the abnormal region.

It is preferable that the illumination light having a specific spectrum includes violet light and blue light, and the amount of violet light is larger than the amount of blue light.

Illumination light having a specific spectrum includes blue light, green light, and red light, the amount of blue light is larger than the amount of green light, and the amount of green light is larger than the amount of red light. Is preferable.

Illumination light having a specific spectrum includes blue light, green light, and red light, the amount of green light is larger than the amount of blue light, and the amount of blue light is larger than the amount of red light. Is preferable.

A medical image used in the first abnormal region discrimination unit, a medical image used in the second abnormal region discrimination unit, or a medical image used in the first abnormal region discrimination unit and a medical image used in the second abnormal region discrimination unit. It is preferable to include a display unit that displays at least one of the other acquired medical images as a medical image for display.

It is preferable that the first abnormal region discriminating unit and the second abnormal region discriminating unit discriminate each abnormal region while displaying the medical image for display.

Display control that superimposes the abnormal area determined by the second abnormal area discrimination unit on the medical image for display and displays it, or displays that the second abnormal area discrimination unit has determined the abnormal area together with the medical image for display. It is preferable to provide a part.

The endoscope device of the present invention is an endoscope that acquires an endoscope image including a subject by photographing a subject using a light source unit that emits a plurality of types of illumination light having different spectra and the illumination light. An abnormality is used by using an image acquisition unit, a first abnormal region discrimination unit that discriminates an abnormal region for each endoscopic image using a plurality of endoscopic images, and a discrimination result of an abnormal region for each endoscopic image. A subject is photographed using a characteristic discriminator that discriminates the characteristics of a region, a light source control unit that switches the illumination light to an illumination light having a specific spectrum according to the characteristics of an abnormal region, and an illumination light having a specific spectrum. A second abnormal region discriminating unit for discriminating an abnormal region is provided by using the endoscopic image obtained.

The medical image processing device and the endoscopic device of the present invention can discriminate an abnormal region more accurately than before.

It is a block diagram of a medical image processing apparatus. It is a block diagram of an endoscope device. It is a block diagram of a medical image analysis processing unit. It is a flowchart which shows the operation of a medical image processing apparatus. It is a medical image for display. It is an endoscopic image for the first abnormal region discrimination processing. It is an endoscopic image for the second abnormal region discrimination processing. It is an endoscopic image for display which showed the discrimination result. It is a block diagram of the medical image analysis processing part which includes the motion discriminating part. It is a block diagram of an endoscope apparatus including a medical image processing apparatus. It is a flowchart in the case of performing the discrimination processing of an abnormal region in real time in an endoscope apparatus. It is explanatory drawing of the diagnosis support apparatus including a medical image processing apparatus. It is explanatory drawing of the medical business support apparatus including a medical image processing apparatus.

[First Embodiment]
As shown in FIG. 1, the medical image processing device 10 includes a medical image acquisition unit 11, a medical image analysis processing unit 12, a display unit 13, a display control unit 15, an input / reception unit 16, a general control unit 17, and a storage unit. 18 is provided.

The medical image acquisition unit 11 obtains a medical image including a subject directly from an endoscope device 21 or the like which is a medical device, or via a management system such as a PACS (Picture Archiving and Communication System) 22 or another information system. get. The medical image is a still image or a moving image (so-called inspection moving image). When the medical image is a moving image, the medical image acquisition unit 11 can acquire a frame image constituting the moving image as a still image after the examination. When the medical image is a moving image, the display of the medical image includes displaying a still image of one representative frame constituting the moving image and playing the moving image one or more times. Further, in the medical image acquired by the medical image acquisition unit 11, in addition to the image taken by the doctor using the medical device such as the endoscope device 21, the medical device such as the endoscope device 21 is instructed by the doctor to take a picture. Includes images taken automatically regardless.

When the medical image acquisition unit 11 can acquire a plurality of medical images, the medical image acquisition unit 11 can selectively acquire one or a plurality of these medical images. In addition, the medical image acquisition unit 11 can acquire a plurality of medical images acquired in a plurality of different tests. For example, one or both of a medical image acquired by a past examination and a medical image acquired by the latest examination can be acquired. That is, the medical image acquisition unit 11 can arbitrarily acquire a medical image.

In the present embodiment, the medical image processing device 10 is connected to the endoscope device 21 and acquires a medical image from the endoscope device 21. That is, in the present embodiment, the medical image is an endoscopic image.

Further, when discriminating an abnormal region, the medical image acquisition unit 11 acquires at least one or a plurality of endoscopic images (medical images) having different imaging conditions. Specifically, in the present embodiment, the medical image acquisition unit 11 has the first endoscope image 111, the second endoscope image 112, the third endoscope image 113, and the third endoscope image 113 in order to discriminate the abnormal region. , 4 types of endoscopic images of the 4th endoscopic image 114 (see FIG. 6) are acquired. Further, the medical image acquisition unit 11 acquires an endoscopic image 121 (see FIG. 7) that is more suitable for discriminating an abnormal region. The first endoscope image 111, the second endoscope image 112, the third endoscope image 113, and the fourth endoscope image 114 are endoscope images taken by using special light. The special light refers to light having a spectrum different from that of white light, and includes so-called narrow band light having a specific very narrow wavelength band.

The imaging conditions are conditions related to the imaging of medical images, such as the spectrum of illumination light, the presence / absence or intensity of image processing when generating a medical image, and the like. The spectrum of illumination light is an intensity distribution for each wavelength, and includes the concept of wavelength band and center wavelength. The image processing when generating a medical image is, for example, a process related to adjustment of a color or the like that emphasizes a specific tissue or lesion. In addition to the above, in the present embodiment, the medical image acquisition unit 11 displays both the portion that determines the abnormal region and the portion that does not determine the abnormal region on the display unit 13 for observation. The endoscopic image 101 for use is acquired (see FIG. 5). In many cases, the endoscopic image 101 for display has different imaging conditions from the endoscopic image used for discriminating the abnormal region. However, when an endoscopic image suitable for display is used when discriminating the abnormal region, the endoscopic image used for discriminating the abnormal region can also be used for the endoscopic image 101 for display. In the present embodiment, the display endoscope image 101 is an endoscope image taken with white light.

The medical image acquired by the medical image acquisition unit 11 in the present embodiment is a medical image taken in one specific examination. In addition, as a general rule, the medical image acquired for the part for determining the abnormal region is within the time range in which the angle of view or the shape of the subject does not change significantly (in the image processing, the parts can be associated with each other). It is a series of medical images taken in (within range).

As shown in FIG. 2, the endoscope device 21 to which the medical image processing device 10 is connected in the present embodiment irradiates at least one of light in a white wavelength band or light in a specific wavelength band to photograph a subject. The endoscope 31, the light source device 32 that irradiates the subject with illumination light through the endoscope 31, the processor device 33, and the monitor 34 that displays an endoscope image taken by using the endoscope 31. Has.

The endoscope 31 includes an image sensor 41 that captures a subject by using illumination light reflected or scattered by the subject, fluorescence emitted by the subject or a drug administered to the subject, or the like. The image sensor 41 is, for example, a CMOS (Complementary Metal-Oxide-Semiconductor) color sensor (sensor having a color filter).

The light source device 32 includes a light source unit 42 and a light source control unit 47. The light source unit 42 emits a plurality of types of illumination light having different spectra. The light source unit 42 includes, for example, a light emitting device such as an LED (Light Emitting Diode), an LD (Laser Diode), or a xenon lamp. Further, the light source unit 42 includes a prism, a mirror, an optical fiber, an optical filter for adjusting the wavelength band, the amount of light, or the like, if necessary. In the present embodiment, the light source unit 42 includes a V-LED43 that emits purple light having a central wavelength of about 405 nm, a B-LED44 that emits blue light having a central wavelength of about 450 nm, and a green light having a central wavelength of about 540 nm. It includes a G-LED 45 that emits light and an R-LED 46 having a center wavelength of about 630 nm.

The light source control unit 47 controls the light emitting source included in the light source unit 42, and generates the illumination light used by the endoscope 31 for photographing the subject. Further, when the light source unit 42 includes a plurality of light emitting devices, the light source control unit 47 can control the light emitting timing and the light emitting amount of each light emitting device. Therefore, the light source device 32 can supply a plurality of types of illumination light having different spectra to the endoscope 31 at an arbitrary timing and at an arbitrary intensity. For example, in the present embodiment, the light source device 32 emits two or more of purple light, blue light, green light, red light, or light of each of these colors, in addition to white light, under the control performed by the light source control unit 47. Light or the like mixed at an arbitrary intensity ratio can be emitted as illumination light at an arbitrary timing and at an arbitrary intensity. In addition, the light source device 32 can emit light having a specific narrow wavelength band (so-called narrow band light) as illumination light depending on the characteristics of the light emitting device or by using an optical filter. For example, light in a wavelength band shorter than the green wavelength band, particularly light in the blue band or purple band in the visible region can be emitted.

The processor device 33 acquires an endoscopic image from the image sensor 41, or an endoscopic image generation unit 48 that generates an endoscopic image obtained by applying image processing to the endoscopic image acquired from the image sensor 41. Be prepared. The image sensor 41 and the endoscopic image generation unit 48 constitute an "endoscopic image acquisition unit" in the endoscope device 21. The endoscopic image acquisition unit acquires an endoscopic image including the subject by photographing the subject using the illumination light. The medical image processing device 10 is connected to the processor device 33. Then, the medical image acquisition unit 11 acquires the endoscopic image directly from the endoscopic image generation unit 48 of the endoscope device 21.

The medical image analysis processing unit 12 performs analysis processing using an endoscopic image (hereinafter, simply referred to as an endoscopic image) which is a medical image acquired by the medical image acquisition unit 11. Specifically, as shown in FIG. 3, it includes a first abnormal region determination unit 52, a characteristic determination unit 53, and a second abnormal region determination unit 54.

The first abnormal region discrimination unit 52 discriminates the abnormal region 119 (see FIG. 6) for each medical image by using a plurality of medical images acquired by the medical image acquisition unit 11 (hereinafter, referred to as a first abnormal region discriminating process). ). When the medical image acquisition unit 11 acquires an endoscopic image, the abnormal region is an region having one or more lesions, a region having a color or shape characteristic different from that of surrounding tissues, a region where a drug is sprayed, or the like. , Subjects who have undergone treatment (biopsy, endoscopic mucosal resection (EMR), or endoscopic submucosal dissection (ESD), etc.) Is an area containing. When the medical image is an endoscopic image, the lesion or the like is, for example, a polyp (raised lesion), and more specifically, a hyperplastic polyp (HP) or SSA / P (sessile serrated adenoma). / polyp), adenoma, cancer, etc. Further, the region having a color or shape characteristic different from that of the surrounding tissue or the like is redness, atrophy, diverticulum, treatment scar, or the like of the subject.

The first abnormal region determination unit 52 determines whether or not a part or all of the medical image is an abnormal region 119. That is, the first abnormal region discrimination unit 52 discriminates the abnormal region 119 for each pixel, for each small region when the endoscopic image is divided into small regions, or for the entire endoscopic image. Can be done. In the present embodiment, the first abnormal region determination unit 52 determines whether or not the abnormal region is 119 for each small region composed of a predetermined number of pixels.

In addition, the first abnormal region determination unit 52 may determine one or a plurality of locations of each medical image as the abnormal region 119. In addition, the first abnormal region discrimination unit 52 uses a medical image taken by using special light having a spectrum different from that of white light to discriminate the abnormal region 119. In addition, the first abnormal region discrimination unit 52 includes at least the presence / absence of the abnormal region 119 and the “probability” indicating the certainty of the discrimination in the discrimination result.

The characteristic discrimination unit 53 discriminates the characteristics of the abnormal region 119 by using the discrimination result of the first abnormal region discrimination unit 52, that is, the discrimination result of the abnormal region 119 for each medical image (hereinafter, referred to as characteristic discrimination processing). The characteristics of the abnormal region 119 include the position, size, range (area, etc.), shape, thickness, length, depth from the surface of the mucous membrane, and the abnormal region of the abnormal target in the abnormal region 119. It refers to the density of tissues, etc., or other characteristics. In the present embodiment, the characteristic determination unit 53 determines the characteristics of the abnormal region 119 as "surface layer type" or "middle layer" according to the depth of the abnormal target in the abnormal region 119 (depth based on the mucosal surface). It is discriminated into one of three types of "type" and "deep type".

Further, the characteristic discrimination unit 53 discriminates the characteristics of the abnormal region 119 by using the accuracy of the discrimination result by the first abnormal region discriminating unit 52. For example, in an endoscopic image in which an abnormality at a relatively shallow position from the mucosal surface or the mucosal surface is often captured, if the accuracy of the discrimination result of the first abnormal region discrimination unit 52 is the highest, it is determined to be "surface layer type". .. On the contrary, in the endoscopic image in which the abnormality at a relatively deep position under the mucous membrane is often captured, if the accuracy of the discrimination result of the first abnormal region discrimination unit 52 is the highest, it is determined to be the "deep layer type".

The second abnormal region determination unit 54 discriminates the abnormal region by using a medical image taken using an illumination light having a specific spectrum controlled according to the characteristics of the abnormal region (hereinafter, the second abnormal region determination). Processing). While the first abnormal area discrimination process is a general-purpose discrimination process for discriminating an abnormal area, the second abnormal area discrimination process is performed by matching the characteristics of the abnormal area on the premise that there is an abnormal area. This is a discrimination process for discriminating the abnormal region more accurately than the first abnormal region discriminating process. The "specific spectrum" is a spectrum controlled according to the characteristics of the anomalous region 119. Therefore, in many cases, the above-mentioned "specific spectrum" includes white light used for photographing a medical image for display (endoscopic image 101 for display) and a medical image used in the first abnormal region discrimination process. It is different from the spectrum of special light used for photographing (first endoscope image 111, second endoscope image 112, third endoscope image 113, and fourth endoscope image 114). However, depending on the characteristics of the abnormal region, the spectrum may be the same as the spectrum of the illumination light used for capturing the medical image for display or the medical image used for the first abnormal region discrimination process.

In the present embodiment, when the characteristic of the abnormal region 119 is "surface layer type", the illumination light having a specific spectrum includes violet light and blue light, and the amount of violet light is larger than the amount of blue light. large. When the characteristic of the anomalous region 119 is "middle layer type", the illumination light having a specific spectrum includes blue light, green light, and red light, and the amount of blue light is larger than the amount of green light. , The amount of green light is larger than the amount of red light. When the characteristic of the anomalous region 119 is "deep layer type", the illumination light having a specific spectrum includes blue light, green light, and red light, and the amount of green light is larger than the amount of blue light. , The amount of blue light is larger than the amount of red light.

The second abnormal region discriminating unit 54 uses a predetermined correspondence relationship between the characteristics of the abnormal region 119 discriminated by the characteristic discriminating unit 53 and the above-mentioned "specific spectrum" to use a medical image for discriminating the abnormal region. Can be determined. In this case, the second abnormal region determination unit 54 holds the correspondence relationship in advance. However, the second abnormal area determination unit 54 can be stored in, for example, a storage unit (not shown) configured by a predetermined memory or the like.

The second abnormal region determination unit 54 determines a medical image to be used for the second abnormal region determination process according to the characteristics of the abnormal region 119 determined by the characteristic determination unit 53, and acquires the medical image from the medical image acquisition unit 11. To do. The medical image acquisition unit 11 acquires in advance a medical image that may be requested by the second abnormal region determination unit 54. Further, when the medical image acquisition unit 11 has not acquired the medical image requested by the second abnormal area determination unit 54, the medical image acquisition unit 11 uses the medical image for the second abnormal area determination process in accordance with the request of the second abnormal area determination unit 54. Get an image.

While the display control unit 15 is displaying the display endoscopic image 101, which is a medical image for display, on the display unit 13, the first abnormal area determination unit 52 and the second abnormal area determination unit 54 are displayed. , Each abnormal region is discriminated. The same applies to the characteristic determination process performed by the characteristic determination unit 53. However, the display control unit 15 updates the display endoscope image 101 to be displayed on the display unit 13 while the first abnormal area discrimination process, the characteristic discrimination process, or the second abnormal area discrimination process is being performed. May be done. That is, "while the display control unit 15 is displaying the display endoscope image 101, which is a medical image for display, on the display unit 13, each abnormal region is determined" is for display. It means that the first abnormal area discrimination process, the characteristic discrimination process, and the second abnormal area discrimination process are executed in the background of the display control for displaying the endoscope image 101 on the display unit 13.

The display unit 13 is a display that displays the medical image acquired by the medical image acquisition unit 11 and the determination result of the abnormal region. That is, the display unit 13 uses the medical image used in the first abnormal area determination unit, the medical image used in the second abnormal area determination unit, or the medical image used in the first abnormal area determination unit and the second abnormal area determination. In addition to the medical image used in the department, at least one of the acquired medical images is displayed as a medical image for display. The monitor or display included in the device or the like to which the medical image processing device 10 is connected can be shared and used as the display unit 13 of the medical image processing device 10.

The display control unit 15 controls the display mode of the medical image and the analysis result on the display unit 13. Specifically, the abnormal region 131 determined by the second abnormal region discriminating unit 54 is superimposed on the display endoscopic image 101, which is a medical image for display, or is displayed, or is a medical image for display. Along with the display endoscope image 101, the second abnormal area determination unit 54 displays that the abnormal area has been determined (message or the like). For example, in the present embodiment, the display control unit 15 displays the endoscope image 101 for display on the display unit 13. In addition, the display control unit 15 emphasizes the outline of the abnormal region 131 (see FIG. 8), which is the discrimination result of the second abnormal region discriminating process, in the display endoscope image 101, and the display is inside the display. The location of the abnormal region 131 is shown in the endoscopic image 101. Depending on the setting, the display control unit 15 can adjust the color of the abnormal region 131 or the region other than the abnormal region 131 to indicate the location of the abnormal region 131. Further, depending on the setting, the display control unit 15 may use any other method such as sound (including voice), light (partial blinking of the endoscope image 101 for display, etc.), coordinate display, and the like to display an abnormal area. The presence or absence of 131 or the location of the abnormal region 131 can be indicated.

The input receiving unit 16 receives input from a mouse, keyboard, or other operating device connected to the medical image processing device 10. The operation of each part of the medical image processing device 10 can be controlled by using these operating devices.

The integrated control unit 17 comprehensively controls the operation of each unit of the medical image processing apparatus 10. When the input receiving unit 16 receives the operation input using the operation device, the overall control unit 17 controls each unit of the medical image processing device 10 according to the operation input.

The storage unit 18 causes an abnormality in a storage device (not shown) such as a memory included in the medical image processing device 10, a medical device such as an endoscope device 21 or a storage device (not shown) included in the PACS 22 as necessary. Save the area discrimination result, etc.

The operation flow of the medical image processing apparatus 10 will be described below. As shown in FIG. 4, the medical image acquisition unit 11 automatically or manually selects a plurality of endoscopic images 101 for display (step S110), and the display control unit 15 acquires the medical image. The display endoscope images 101 acquired by the acquisition unit 11 are sequentially displayed on the display unit 13 (step S111). In the present embodiment, for example, the endoscope image 101 for display shown in FIG. 5 is acquired and displayed.

On the other hand, in the background, an endoscopic image for the first abnormal region discrimination process is acquired (step S112). Specifically, as shown in FIG. 6, the first endoscope image 111 taken with the purple special light, the second endoscope image 112 taken with the blue special light, and the green one. The third endoscope image 113 photographed by using the special light and the fourth endoscope image 114 photographed by using the red special light are acquired.

When the medical image acquisition unit 11 acquires the first endoscopic image 111, the second endoscopic image 112, the third endoscopic image 113, and the fourth endoscopic image 114 for the first abnormal region discrimination process, The first abnormal area determination unit 52 uses these to perform the first abnormal area determination process (step S113). Specifically, the first abnormal region discrimination unit 52 uses the first endoscope image 111, the second endoscope image 112, and the third endoscope image 113, respectively, using the fourth endoscope image 114. To standardize. As a result, the influence of the illuminance of the first endoscope image 111, the second endoscope image 112, and the third endoscope image 113 is removed. Then, the first abnormal region determination unit 52 is abnormal from the standardized first endoscope image 111, the standardized second endoscope image 112, and the standardized third endoscope image 113, respectively. The presence or absence of the region 119 is determined. For example, the first abnormal region determination unit 52 discriminates a local low signal value region (a region having a small pixel value) as an abnormal region 119 in each of the standardized endoscopic images 111, 112, 113.

When the first abnormal region discrimination unit 52 outputs the presence / absence of the abnormal region 119 and the accuracy of discrimination for each of the standardized endoscopic images 111, 112, 113, the characteristic discriminating unit 53 uses this to output the abnormal region 119. (Step S114).

Specifically, when there is an abnormal region 119 and the accuracy of the discrimination result of the standardized first endoscope image 111 is the highest, the characteristic discriminating unit 53 determines that the characteristic of the abnormal region 119 is "surface layer type". Determine if there is. Since the first endoscopic image 111 is an endoscopic image taken using special blue light, lesions and the like at a relatively shallow position (so-called surface layer) on the mucosal surface or under the mucosa can be easily captured. Therefore, when the lesion or the like is near the surface layer, the lesion or the like can be more accurately discriminated in the first endoscopic image 111 than in the second endoscopic image 112 and the third endoscopic image 113, and as a result. This is because the accuracy of the first abnormal region determination process is high. Further, when there is an abnormal region 119 and the accuracy of the discrimination result of the standardized second endoscope image 112 is the highest, the characteristic discriminating unit 53 determines that the characteristic of the abnormal region is "middle layer type". .. Similarly, when there is an abnormal region 119 and the accuracy of the discrimination result of the standardized third endoscope image 113 is the highest, the characteristic discriminating unit 53 determines that the characteristic of the abnormal region is "deep layer type". To do. In this way, the characteristic determination unit 53 determines the depth from the surface of the abnormal target in the abnormal region 119 as the characteristic of the abnormal region.

When the characteristic discrimination unit 53 discriminates the characteristics of the abnormal region 119, the second abnormal region discriminating unit 54 determines one or a plurality of endoscopes used for the second abnormal region discriminating process shown in FIG. 7, for example, according to the characteristics of the abnormal region 119. The mirror image 121 is determined. The endoscope image 121 for the second abnormal region discrimination process is an endoscopic image taken by using illumination light having a specific spectrum controlled according to the characteristics of the abnormal region 119, and is a characteristic of the abnormal region 119. When is "surface layer type", the endoscopic image 121 is an endoscopic image taken by using illumination light in which features such as lesions near the surface layer of the mucous membrane are easily captured. When the characteristic of the abnormal region 119 is "middle layer type", the endoscopic image 121 is an endoscopic image taken by using illumination light in which features such as lesions in the vicinity of the middle layer of the mucous membrane are easily captured. When the characteristic of the abnormal region 119 is "deep layer type", the endoscopic image 121 is an endoscopic image taken by using illumination light in which features such as lesions in the vicinity of the deep layer of the mucous membrane are easily captured.

The second abnormal region discrimination unit 54 acquires the endoscopic image 121 for the second abnormal region discrimination processing from the medical image acquisition unit 11 (step S115), and uses the endoscopic image 121 again to obtain the abnormal region 131 again. Is determined (step S116). As a result, as shown in FIG. 8, the display control unit 15 displays the abnormal region 131 determined by the second abnormal region discriminating unit 54 using the endoscopic image 121 on the display unit 13 for display endoscopy. It is displayed on the mirror image 101, and its location is shown to a doctor or the like. As a result, the medical image processing device 10 supports diagnosis and the like.

As described above, the medical image processing apparatus 10 performs the first abnormal region discrimination processing to discriminate the abnormal region for the time being, and discriminates the characteristics of the discriminated abnormality. Then, according to the characteristics of the abnormal region, the abnormal region is determined again by using a medical image in which the characteristics of the abnormality in the abnormal region are easily captured. Therefore, the determination of the abnormal region performed by the medical image processing apparatus 10 is less dependent on the type or property of the lesion or the like, or the type of the organ having the lesion or the like than before. As a result, the medical image processing apparatus 10 can discriminate an abnormal region more stably and accurately than before.

In the first embodiment, the second abnormal region discrimination unit 54 determines the medical image used for discriminating the abnormal region by using the characteristics of the abnormal region and the correspondence with the “specific spectrum”, but the second abnormality Instead of using the correspondence relationship, the region discrimination unit 54 can calculate the “specific spectrum” by using the characteristics of the abnormal region discriminated by the characteristic discrimination unit 53. For example, the second abnormal region discrimination unit 54 contains purple light with an amount of light proportional to the accuracy of the first abnormal region discrimination processing of the standardized first endoscope image 111, and the standardized second endoscope image 112. Contains blue light with an amount of light proportional to the accuracy of the first abnormal region discrimination process of the above, and also contains green light with an amount of light proportional to the accuracy of the first abnormal region discrimination process of the standardized third endoscope image 113. A "specific spectrum" can be calculated. If illuminating light containing purple light, blue light, and green light is used at this ratio, features such as lesions included in the abnormal region can be easily captured as a result, so that an endoscopic image suitable for the second abnormal region discrimination process is used. It becomes 121. Therefore, if the endoscopic image 121 taken by using the illumination light having the specific spectrum calculated as described above is used, the second abnormal region determination unit 54 is more accurate than the conventional one as in the first embodiment. The abnormal region 131 can be discriminated well.

Each unit constituting the medical image analysis processing unit 12 such as the first abnormal region determination unit 52, the characteristic determination unit 53, and the second abnormal region determination unit 54 in the first embodiment has been learned by machine learning, deep learning, or the like. , So-called AI (Artificial Intelligence) program can be used for configuration.

In the first embodiment, the second abnormal region determination unit 54 performs the second abnormal region determination process using one endoscopic image 121, but the second abnormal region determination unit 54 has a plurality of abnormal region determination units 54. The second abnormal region discrimination process can be performed using the medical image.

As shown in FIG. 9, the medical image analysis processing unit 12 includes a first abnormal region determination unit 52, a characteristic determination unit 53, a second abnormal region determination unit 54, and a motion determination unit 151. Can include. When a plurality of endoscope images are used for discriminating an abnormal region, the motion discrimination unit 151 determines the movement of the subject image (relative movement between the subject and the endoscope 31) reflected in the plurality of endoscope images. Determine. For example, when discriminating the movement of an endoscope image for two consecutive frames, the motion discriminating unit 151 determines the degree of movement of the subject image based on the correlation of pixel values. More specifically, when two consecutive frames of G images (endoscope images taken using green light) are used as G1 images and G2 images, each image is in each of the G1 image and the G2 image. A plurality of small square areas (for example, about 20 places) are set in the area, the average value of the pixel values in each small square area is calculated, and the small square areas of these small square areas are separated from each other between the G1 image and the G2 image. Obtain the correlation coefficient of the average pixel value. Then, when the correlation coefficient is equal to or more than a predetermined threshold value, it is determined that there is no movement of the subject image, and when the correlation coefficient is less than the predetermined threshold value, it is determined that there is movement of the subject image. To do.

When the medical image analysis processing unit 12 includes the motion discrimination unit 151, when performing the first abnormal region discrimination processing or when using a plurality of endoscopic images in the second abnormal region discrimination processing, these plurality of As the endoscopic image, a combination of endoscopic images determined by the motion determination unit 151 that the subject image does not move can be used. As a result, each of these discrimination processes can be performed with high accuracy.

[Second Embodiment]
In each of the above embodiments, the medical image processing device 10 and the endoscope device 21 are separate devices, but the endoscope device 21 can include the medical image processing device 10. In this case, as in the endoscope device 510 shown in FIG. 10, each part 520 constituting the medical image processing device 10 is provided in the processor device 33. However, the display unit 13 can share the monitor 34 of the endoscope device 21. Further, the medical image acquisition unit 11 corresponds to the "endoscopic image acquisition unit" formed by the image sensor 41 and the endoscopic image generation unit 48. Therefore, it is sufficient to provide the processor device 33 with each unit other than the medical image acquisition unit 11 and the display unit 13. The configuration of each other part is the same as that of the first embodiment. In addition, a new endoscope device can be configured as a whole of the medical image processing device 10 of each of the above-described embodiments and other modifications, and the endoscope device 21 of FIG.

The endoscope device 510 including the medical image processing device 10 is basically a device for observing a subject in real time. Therefore, the endoscope device 510 includes the acquisition of the endoscopic image which is a medical image, the first abnormal region discrimination processing, the characteristic discrimination processing, the second abnormal region discrimination processing, the discrimination result display processing, and the like. It can be executed in real time while taking an endoscopic image, or at any timing caused by the operation of various operation units.

Hereinafter, the flow of operation when the endoscope device 21 including the medical image processing device 10 performs the abnormality region discrimination process in real time will be described. As shown in FIG. 11, the endoscope device 510 sequentially photographs a subject using, for example, white light, acquires an endoscope image 101 for display (step S210), and obtains the acquired endoscope for display. The mirror image 101 is displayed on the monitor 34 (step S211). The endoscope device 510 constantly repeats these steps, for example, at a predetermined frame rate. Therefore, a doctor or the like can observe the subject in real time.

As described above, while the display endoscope image 101 is being displayed, in the background, the general control unit 17 is performing the display during the shooting of the series of display endoscope images 101. At a predetermined timing that does not interfere with the continuous display of the endoscopic image 101, the imaging for determining the first abnormal region is interrupted intermittently. That is, the light source control unit 47 intermittently switches the illumination light from white light to purple special light, blue special light, green special light, or red special light. Then, the image sensor 41 takes a picture of the subject with the special light of each color between the pictures taken with the white light, so that the endoscopic image for the first abnormal region discrimination processing similar to the first embodiment, that is, , The first endoscope image 111, the second endoscope image 112, the third endoscope image 113, and the fourth endoscope image 114 are acquired (step S212).

The first abnormal region determination unit 52 obtains the first endoscope image 111, the second endoscope image 112, the third endoscope image 113, and the fourth endoscope image 114 acquired in real time as described above. The first abnormal region determination process is performed (step S213), and the characteristic determination unit 53 determines the characteristics of the abnormal region 119 determined in the first abnormal region determination process (step S214). These are the same as in the first embodiment.

After that, when the second abnormal region determination unit 54 determines a specific spectrum according to the characteristics of the abnormal region 119, the integrated control unit 17 displays the endoscope images 101 for display during the acquisition of the series of display endoscopic images 101. At a predetermined timing that does not interfere with the continuous display of the endoscopic image 101, the imaging for determining the first abnormal region is interrupted intermittently. At this time, the light source control unit 47 switches the illumination light to the illumination light having a specific spectrum suitable for the second abnormal region discrimination process according to the characteristics of the abnormal region. Specifically, the light source control unit 47 switches the illumination light from the white light to the illumination light having a specific spectrum determined by the second abnormal region determination unit 54 (step S215), and the image sensor 41 uses each of these colors. By photographing the subject with the special light of the above, the endoscope image 121 for the second abnormal region discrimination processing similar to that of the first embodiment is acquired (step S216). Then, the second abnormal region determination unit 54 performs the second abnormal region determination process using the endoscope image 121 acquired in real time as described above (step S217), and the display control unit 15 performs the second abnormal region determination process. The abnormal region 131 determined by the discrimination unit 54 using the endoscope image 121 is displayed on the display endoscope image 101 displayed on the display unit 13, and the location thereof is shown to a doctor or the like.

As described above, the endoscope device 510 including the medical image processing device 10 performs the first abnormal region discrimination processing, the characteristic discrimination processing, the second abnormal region discrimination processing, and the like in real time to perform diagnosis and the like in real time. Can be assisted.

In the second embodiment, the illumination light is switched from white light to purple special light, blue special light, green special light, or red special light when taking a picture for determining the first abnormal region. , The first endoscope image 111, the second endoscope image 112, the third endoscope image 113, and the fourth endoscope image 114 are acquired one by one, and two or more of them are acquired. It can be acquired at the same time (in one frame). For example, when the image sensor 41 is a color sensor, when the light source control unit 47 takes a picture for determining the first abnormal region, it is red in the same shooting frame as the purple special light or the blue special light. By emitting special light, the first endoscope image 111 or the second endoscope image 112 and the fourth endoscope image 114 can be acquired in one frame. This is because there is no (or little) color mixing even if the purple special light or the blue special light and the red special light are emitted at the same time. In this way, for example, when the first endoscope image 111 or the second endoscope image 112 and the fourth endoscope image 114 are acquired in one frame, the first abnormal region determination process is performed in a total of three frames. It is efficient because the necessary endoscopic image can be acquired, and the influence on the acquisition and display of the endoscopic image 101 for display can be further reduced.

In the second embodiment, the second abnormal region determination unit 54 determines a specific spectrum from the characteristics of the abnormal region 119 using a predetermined correspondence relationship as in the first embodiment. When the spectrum of the illumination light can be changed in real time and flexibly as in the embodiment, the second abnormal region determination unit 54 uses the abnormal region 119 determined by the characteristic determination unit 53 instead of using a predetermined correspondence relationship. Specific spectrum (specifically, light emission of each light emitting source for realizing a specific spectrum) of the illumination light for capturing the endoscopic image 121 suitable for the second abnormal region discrimination process by using the characteristics of It is preferable to calculate the ratio, etc.). This is because the spectrum (specific spectrum) of the illumination light to be used can be determined more accurately than the spectrum defined in the correspondence relationship, and as a result, the accuracy of the second abnormal region discrimination process is further improved.

In the second embodiment, the endoscope device 510 includes the medical image processing device 10, but as shown in FIG. 12, the diagnostic support device 610 used in combination with the endoscope device 21 and other modalities. Can include the medical image processing apparatus 10 of the above embodiment and other modifications. Further, as shown in FIG. 13, various inspection devices such as the first inspection device 621, the second inspection device 622, ..., The Nth inspection device 623, including the endoscope device 21, and an arbitrary network 626 are used. The medical business support device 630 to be connected can include the medical image processing device 10 of the above embodiment and other modifications.

In addition, the medical image processing device 10, various devices including the medical image processing device 10, and various devices or systems including the functions of the medical image processing device 10 can be used by making the following various changes and the like. ..

As the medical image, a white band light or a normal light image obtained by irradiating light of a plurality of wavelength bands as white band light can be used.

When an image obtained by irradiating light in a specific wavelength band is used as the medical image, the specific wavelength band can be narrower than the white wavelength band.

The specific wavelength band is, for example, a blue band or a green band in the visible region.

When the specific wavelength band is the blue band or the green band in the visible region, the specific wavelength band includes the wavelength band of 390 nm or more and 450 nm or less or 530 nm or more and 550 nm or less, and the light of the specific wavelength band is 390 nm or more. It is preferable to have a peak wavelength in the wavelength band of 450 nm or less or 530 nm or more and 550 nm or less.

The specific wavelength band is, for example, the red band in the visible range.

When the specific wavelength band is the red band in the visible region, the specific wavelength band includes a wavelength band of 585 nm or more and 615 nm or 610 nm or more and 730 nm or less, and the light of the specific wavelength band is 585 nm or more and 615 nm or less or 610 nm. It is preferable to have a peak wavelength in the wavelength band of 730 nm or less.

The specific wavelength band includes, for example, a wavelength band in which the absorption coefficient differs between the oxidized hemoglobin and the reduced hemoglobin, and the light in the specific wavelength band has a peak wavelength in the wavelength band in which the absorption coefficient differs between the oxidized hemoglobin and the reduced hemoglobin. Can have.

A specific wavelength band includes a wavelength band in which the absorption coefficient differs between the oxidized hemoglobin and the reduced hemoglobin, and light in the specific wavelength band has a peak wavelength in the wavelength band in which the absorption coefficient differs between the oxidized hemoglobin and the reduced hemoglobin. In this case, the specific wavelength band includes 400 ± 10 nm, 440 ± 10 nm, 470 ± 10 nm, or 600 nm or more and 750 nm or less, and the light in the specific wavelength band is 400 ± 10 nm, 440 ± 10 nm, It is preferable to have a peak wavelength in a wavelength band of 470 ± 10 nm or 600 nm or more and 750 nm or less.

When the medical image is an in-vivo image showing the inside of a living body, the in-vivo image can have information on fluorescence emitted by a fluorescent substance in the living body.

Further, as the fluorescence, fluorescence obtained by irradiating the living body with excitation light having a peak wavelength of 390 nm or more and 470 nm or less can be used.

When the medical image is an in-vivo image of the inside of a living body, the wavelength band of infrared light can be used as the specific wavelength band described above.

When the medical image is an in-vivo image of the inside of a living body and the wavelength band of infrared light is used as the above-mentioned specific wavelength band, the specific wavelength band is a wavelength band of 790 nm or more and 820 nm or 905 nm or more and 970 nm or less. Light in a specific wavelength band preferably has a peak wavelength in a wavelength band of 790 nm or more and 820 nm or less or 905 nm or more and 970 nm or less.

The medical image acquisition unit 11 acquires a special optical image having a signal in a specific wavelength band based on a normal light image obtained by irradiating light in a white band or light in a plurality of wavelength bands as light in the white band. It can have a special optical image acquisition unit. In this case, a special light image can be used as a medical image.

A signal in a specific wavelength band can be obtained by an operation based on RGB or CMY color information included in a normal optical image.

By calculation based on at least one of a normal light image obtained by irradiating light in a white band or light in a plurality of wavelength bands as light in a white band and a special light image obtained by irradiating light in a specific wavelength band. A feature amount image generation unit for generating a feature amount image can be provided. In this case, a feature image can be used as a medical image.

As for the endoscope device 21, a capsule endoscope can be used as the endoscope 31. In this case, the light source device 32 and a part of the processor device 33 can be mounted on the capsule endoscope.

In each of the above embodiments and modifications, the medical image acquisition unit 11, the medical image analysis processing unit 12 (each unit constituting the medical image analysis processing unit 12), the display control unit 15, the input receiving unit 16, the integrated control unit 17, and the control unit 17 The hardware structure of the processing unit that executes various processes such as the endoscope image generation unit 48 of the endoscope device 21 is various processors as shown below. For various processors, the circuit configuration is changed after manufacturing the CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), which is a general-purpose processor that executes software (program) and functions as various processing units. It includes a programmable logic device (PLD), which is a possible processor, a dedicated electric circuit, which is a processor having a circuit configuration specially designed for performing various processes, and the like.

One processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of a CPU and an FPGA). May be done. Further, a plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a client or a server, one processor is configured by a combination of one or more CPUs and software. There is a form in which this processor functions as a plurality of processing units. Secondly, as typified by System On Chip (SoC), there is a form in which a processor that realizes the functions of the entire system including a plurality of processing units with one IC (Integrated Circuit) chip is used. is there. As described above, the various processing units are configured by using one or more of the above-mentioned various processors as a hardware-like structure.

Further, the hardware structure of these various processors is, more specifically, an electric circuit in the form of combining circuit elements such as semiconductor elements.

10 Medical image processing device 11 Medical image acquisition unit 12 Medical image analysis processing unit 13 Display unit 15 Display control unit 16 Input / reception unit 17 General control unit 18 Storage unit 21,510 Endoscope device 22 PACS
31 Endoscope 32 Light source device 33 Processor device 34 Monitor 41 Image sensor 42 Light source unit 43 V-LED
44 B-LED
45 G-LED
46 R-LED
47 Light source control unit 48 Endoscopic image generation unit 52 1st abnormal area discrimination unit 53 Characteristic judgment unit 54 2nd abnormal region discrimination unit 101 Endoscopic image for display 111 1st endoscopic image 112 2nd endoscope Image 113 Third endoscopic image 114 Fourth endoscopic image 119, 131 Abnormal area 121 Endoscopic image for discriminating the second abnormal area 151 Motion discriminating part 520 Each part constituting the medical image processing device 610 Diagnosis support device 621 1st inspection device 622 2nd inspection device 623 Nth inspection device 626 Network 630 Medical business support device S110-S217 Operation steps

Claims (13)

A medical image acquisition unit that acquires medical images including the subject,
A first abnormal region determination unit that discriminates an abnormal region for each medical image using a plurality of the medical images,
Using the discrimination result of the abnormal region for each medical image, a characteristic discriminating unit for discriminating the characteristics of the abnormal region,
A second abnormal region discriminating unit that discriminates the abnormal region by using the medical image obtained by photographing the subject using illumination light having a specific spectrum controlled according to the characteristics of the abnormal region.
A medical image processing device comprising. The medical image processing apparatus according to claim 1, wherein the first abnormal region discrimination unit uses the medical image taken by using special light having a spectrum different from that of white light to discriminate the abnormal region. The medical image processing apparatus according to claim 1 or 2, wherein the characteristic determination unit determines the depth from the surface of an abnormality target in the abnormal region as a characteristic of the abnormal region. The medical image processing apparatus according to any one of claims 1 to 3, wherein the characteristic discrimination unit discriminates the characteristics of the abnormal region by using the accuracy of the discrimination result by the first abnormal region discrimination unit. The second abnormal region discriminating unit determines the medical image used for discriminating the abnormal region by using a predetermined correspondence relationship between the characteristics of the abnormal region and the specific spectrum. The medical image processing apparatus according to any one of 4. The medical image processing apparatus according to any one of claims 1 to 4, wherein the second abnormal region determination unit calculates the specific spectrum by using the characteristics of the abnormal region. The medical image processing apparatus according to claim 5 or 6, wherein the illumination light having a specific spectrum includes violet light and blue light, and the amount of violet light is larger than the amount of blue light. The illumination light having the specific spectrum includes blue light, green light, and red light, the amount of blue light is larger than the amount of green light, and the amount of green light is larger than the amount of red light. The medical image processing apparatus according to claim 5 or 6. The illumination light having the specific spectrum includes blue light, green light, and red light, the amount of green light is larger than the amount of blue light, and the amount of blue light is larger than the amount of red light. The medical image processing apparatus according to claim 5 or 6. The medical image used in the first abnormal region discrimination unit, the medical image used in the second abnormal region discrimination unit, or the medical image used in the first abnormal region discrimination unit and the second abnormal region discrimination. The medical image according to any one of claims 1 to 9, further comprising a display unit for displaying at least one of the acquired medical images in addition to the medical image used in the unit as the medical image for display. Processing equipment. The medical image processing apparatus according to claim 10, wherein the first abnormal region determination unit and the second abnormal region determination unit determine each of the abnormal regions while displaying the medical image for display. .. The abnormal region determined by the second abnormal region discriminating unit is superimposed on the medical image for display and displayed, or the second abnormal region discriminating unit discriminates the abnormal region together with the medical image for display. The medical image processing apparatus according to claim 10 or 11, further comprising a display control unit for displaying the effect. A light source unit that emits multiple types of illumination light with different spectra,
An endoscopic image acquisition unit that acquires an endoscopic image including the subject by photographing the subject using the illumination light.
A first abnormal region determination unit that discriminates an abnormal region for each endoscopic image using a plurality of the endoscopic images,
A characteristic discrimination unit that discriminates the characteristics of the abnormal region by using the discrimination result of the abnormal region for each endoscopic image,
A light source control unit that switches the illumination light to illumination light having a specific spectrum according to the characteristics of the abnormal region.
A second abnormal region discrimination unit that discriminates the abnormal region by using the endoscopic image obtained by photographing the subject using the illumination light having the specific spectrum.
Endoscope device equipped with.

Images