WO2019083019A1

WO2019083019A1 - Medical image processing device and endoscope device

Info

Publication number: WO2019083019A1
Application number: PCT/JP2018/039914
Authority: WO
Inventors: 孝明齋藤
Original assignee: 富士フイルム株式会社
Priority date: 2017-10-26
Filing date: 2018-10-26
Publication date: 2019-05-02
Also published as: JPWO2019083019A1; JP6785990B2

Abstract

Provided are a medical image processing device and an endoscope device that can discriminate an abnormal region with greater accuracy than was previously possible. A medical image processing device (10) comprises: a medical image acquiring unit (11) that acquires a medical image containing a subject; a first abnormal region discriminating unit (52) that discriminates abnormal regions in each medical image using a plurality of medical images; a characteristic discriminating unit (53) that discriminates characteristics of the abnormal regions using the discrimination results of the abnormal regions of the medical images; and a second abnormal region discriminating unit (54) that discriminates abnormal regions using a medical image captured using illumination light having a specific spectrum controlled in accordance with the characteristics of the abnormal regions.

Description

MEDICAL IMAGE PROCESSING DEVICE AND ENDOSCOPIC DEVICE

The present invention relates to a medical image processing apparatus using an analysis result of a medical image, and an endoscope apparatus.

Conventionally, among devices related to medical treatment (hereinafter referred to as medical devices), those that acquire images (hereinafter referred to as medical images) including a subject present the acquired medical images to the doctor. Then, the doctor uses the medical image obtained from the medical device as one of the determination materials to make a diagnosis and the like. As a matter of course, the discrimination of the condition of the subject or the like using the medical image at the time of diagnosis is based on the skill and experience of the doctor.

In recent years, as image analysis technology has advanced, objective information or quantitative information can be obtained from medical images by analyzing medical images. For this reason, medical devices that support discrimination, diagnosis, and the like are increasing by presenting analysis results of medical images to a doctor or the like. For example, the endoscope apparatus described in Patent Document 1 is abnormal using a fluorescence image taken using fluorescence or a narrow band light image taken using light having a specific narrow wavelength band (so-called narrow band light). Locate the area where And the position of the area | region which has abnormality is displayed on the endoscopic image for a display.

Japanese Patent Application Publication No. 2006-198106

In a conventional medical device, a notable area (so-called attention area, area of interest) including one or more parts where abnormality such as a lesion is recognized by analyzing a medical image (lesion part or a part having a possibility of a lesion) Or, it is an area called an abnormal area etc. Hereinafter, it is called an abnormal area, and the position etc. is presented to support diagnosis and the like. And in the detection of an abnormal area | region, it is normal to use the medical image image | photographed on the specific imaging condition which is easy to detect the kind etc. of the lesion made into detection object. That is, in the conventional medical device, detection of an abnormal area is performed on the premise that a specific lesion or the like is easily detected under a specific imaging condition. However, depending on the type or nature of the lesion or the like (progression degree or the like), or the type of organ or the like in which the lesion or the like is present, the imaging conditions that can be observed well differ.

An object of the present invention is to provide a medical image processing apparatus and an endoscope apparatus capable of determining an abnormal area with higher accuracy than in the prior art.

A medical image processing apparatus according to the present invention comprises a medical image acquisition unit for acquiring a medical image including a subject, a first abnormal area discrimination unit for discriminating an abnormal area for each medical image using a plurality of medical images, and a medical image A medical image of a subject taken using an illumination light having a specific spectrum controlled according to the characteristics of the abnormal area, and a characteristic judgment unit for judging the characteristics of the abnormal area using the judgment result of the abnormal area for each And a second abnormal area determination unit that determines an abnormal area by using the second abnormal area determination unit.

The first abnormal area determination unit preferably uses a medical image captured using special light having a spectrum different from that of white light for determining an abnormal area.

The characteristic determining unit preferably determines the depth from the surface of the target of the abnormality in the abnormal area as the characteristic of the abnormal area.

The characteristic determination unit preferably determines the characteristic of the abnormal area using the accuracy of the determination result by the first abnormal area determination unit.

The second abnormal area determination unit preferably determines a medical image used to determine the abnormal area, using a predetermined correspondence relationship between the characteristics of the abnormal area and a specific spectrum.

The second abnormal area determination unit preferably calculates a specific spectrum using the characteristics of the abnormal area.

The illumination light having a specific spectrum preferably includes violet light and blue light, and the light amount of the violet light is preferably larger than the light amount of the blue light.

The illumination light having a specific spectrum includes blue light, green light and red light, and the light amount of blue light is larger than the light amount of green light, and the light amount of green light is larger than the light amount of red light Is preferred.

The illumination light having a specific spectrum includes blue light, green light, and red light, and the amount of green light is larger than the amount of blue light, and the amount of blue light is larger than the amount of red light Is preferred.

The medical image used in the first abnormal area judging unit, the medical image used in the second abnormal area judging unit, or the medical image used in the first abnormal area judging unit and the medical image used in the second abnormal area judging unit It is preferable to have a display unit that displays at least one of the other acquired medical images as a medical image for display.

It is preferable that the first abnormal area judging unit and the second abnormal area judging unit judge each abnormal area while displaying the medical image for display.

Display control which superimposes and displays the abnormal area which the 2nd abnormal area judgment part judged on a medical image for display, or displays that the 2nd abnormal area judgment part judged the abnormal area with a medical image for display It is preferable to provide a part.

The endoscope apparatus according to the present invention is an endoscope that acquires an endoscope image including a subject by photographing a subject using the illumination light and a light source unit that emits a plurality of types of illumination lights having different spectra. Using the image acquisition unit, a first abnormal area determination unit for determining an abnormal area for each endoscopic image using a plurality of endoscopic images, and a determination result for an abnormal area for each endoscopic image A subject is photographed using a light source control unit that switches illumination light to illumination light having a specific spectrum according to the characteristic determination unit that determines the characteristics of the region, and a characteristic of the abnormal region, and illumination light having a specific spectrum And a second abnormal area determination unit that determines an abnormal area using the endoscope image.

The medical image processing apparatus and the endoscope apparatus of the present invention can determine an abnormal area with higher accuracy than in the related art.

It is a block diagram of a medical image processing device. It is a block diagram of an endoscope apparatus. It is a block diagram of a medical image analysis processing part. It is a flowchart which shows an effect | action of a medical image processing apparatus. It is a medical image for display. It is an endoscope image for the 1st unusual field distinction processing. It is an endoscopic image for 2nd abnormal area | region discrimination | determination processing. It is an endoscope image for display which showed a distinction result. It is a block diagram of a medical image analysis processing part provided with a motion distinction part. It is a block diagram of an endoscope apparatus containing a medical image processing device. It is a flowchart in the case of performing the discrimination | determination process of an abnormal area | region in real time in an endoscope apparatus. It is an explanatory view of a diagnosis support device containing a medical image processing device. It is an explanatory view of a medical operation support device including a medical image processing device.

First Embodiment
As shown in FIG. 1, the medical image processing apparatus 10 includes a medical image acquisition unit 11, a medical image analysis processing unit 12, a display unit 13, a display control unit 15, an input reception unit 16, an overall control unit 17, and a storage unit. It has eighteen.

The medical image acquisition unit 11 directly transmits a medical image including a subject via a management system such as a PACS (Picture Archiving and Communication System) 22 or other information system directly from the endoscope apparatus 21 or the like which is a medical apparatus. get. The medical image is a still image or a moving image (so-called test moving image). When the medical image is a moving image, the medical image acquiring unit 11 can acquire a frame image constituting the moving image as a still image after the examination. When the medical image is a moving image, displaying the medical image includes reproducing the moving image one or more times in addition to displaying a still image of one representative frame constituting the moving image. In addition, in the medical image acquired by the medical image acquisition unit 11, the medical device such as the endoscope device 21 or the like receives the imaging instruction of the doctor in addition to the image photographed by the doctor using the medical device such as the endoscope device 21 or the like. Includes images taken automatically regardless of

When the medical image acquisition unit 11 can acquire a plurality of medical images, it can selectively acquire one or more medical images among these medical images. Further, the medical image acquisition unit 11 can acquire a plurality of medical images acquired in a plurality of different examinations. For example, one or both of a medical image acquired by an examination performed in the past and a medical image acquired by the latest examination can be acquired. That is, the medical image acquisition unit 11 can arbitrarily acquire a medical image.

In the present embodiment, the medical image processing apparatus 10 is connected to the endoscope apparatus 21 and acquires a medical image from the endoscope apparatus 21. That is, in the present embodiment, the medical image is an endoscopic image.

Further, in the case of determining the abnormal area, the medical image acquisition unit 11 acquires endoscopic images (medical images) in which at least one or a plurality of imaging conditions are different. Specifically, in the present embodiment, the medical image acquisition unit 11 uses the first endoscopic image 111, the second endoscopic image 112, the third endoscopic image 113, and the like to determine the abnormal area. , And four types of endoscopic images of the fourth endoscopic image 114 (all refer to FIG. 6). Furthermore, the medical image acquisition unit 11 acquires an endoscopic image 121 (see FIG. 7) more suitable for the determination of the abnormal area. The first endoscopic image 111, the second endoscopic image 112, the third endoscopic image 113, and the fourth endoscopic image 114 are endoscopic images taken using special light. Special light is light having a spectrum different from white light, and includes so-called narrow band light having a specific narrow wavelength band.

The imaging condition is a condition relating to imaging of a medical image, and is, for example, the spectrum of illumination light or the presence or absence or intensity of image processing at the time of generating a medical image. The spectrum of illumination light is an intensity distribution for each wavelength, and includes the concepts of wavelength band and central wavelength. The image processing at the time of generating a medical image is, for example, processing relating to adjustment of a color or the like that emphasizes a specific tissue or a lesion. In addition to the above, in the present embodiment, the medical image acquisition unit 11 is a display for displaying on the display unit 13 for observation of both the part for determining the abnormal area and the part for not determining the abnormal area. The endoscope image 101 for the subject is acquired (see FIG. 5). In many cases, the endoscopic image 101 for display is different in imaging condition from the endoscopic image used to determine the abnormal area. However, when using an endoscopic image suitable for display when determining an abnormal area, an endoscopic image used for determining an abnormal area can also be used for the endoscopic image 101 for display. In the present embodiment, the endoscopic image 101 for display is an endoscopic image captured using white light.

In addition, the medical image which the medical image acquisition part 11 acquires in this embodiment is a medical image image | photographed in one specific test | inspection. In addition, in principle, the medical image acquired for the part that determines the abnormal area is within a temporal range in which the angle of view or the shape of the subject does not change significantly (the temporal extent to which the parts can be associated with each other in image processing) Range of medical images taken within the

As shown in FIG. 2, in the present embodiment, the endoscope apparatus 21 to which the medical image processing apparatus 10 is connected shoots a subject by irradiating at least one of light of a white wavelength band or light of a specific wavelength band. Endoscope 31, a light source device 32 for emitting illumination light into a subject through the endoscope 31, a processor device 33, and a monitor 34 for displaying an endoscopic image or the like captured using the endoscope 31. Have.

The endoscope 31 includes an image sensor 41 for imaging a subject using illumination light in which the subject is reflected or scattered, or fluorescence from which the subject or a drug or the like administered to the subject emits light. The image sensor 41 is, for example, a complementary metal-oxide-semiconductor (CMOS) color sensor (a sensor having a color filter).

The light source device 32 includes a light source unit 42 and a light source control unit 47. The light source unit 42 emits a plurality of types of illumination light having different spectra. The light source unit 42 includes, for example, a light emitting device such as a light emitting diode (LED), a laser diode (LD), or a xenon lamp. In addition, the light source unit 42 is provided with a prism, a mirror, an optical fiber, an optical filter for adjusting a wavelength band, a light amount, and the like as needed. In the present embodiment, the light source unit 42 includes a V-LED 43 that emits violet light with a center wavelength of about 405 nm, a B-LED 44 that emits blue light with a center wavelength of about 450 nm, and a green light with a center wavelength of about 540 nm. It comprises a G-LED 45 emitting light and an R-LED 46 with a center wavelength of about 630 nm.

The light source control unit 47 controls the light emission source included in the light source unit 42, and generates illumination light used by the endoscope 31 for photographing an object. Further, when the light source unit 42 includes a plurality of light emitting devices, the light source control unit 47 can control the light emission timing and the light emission amount of each light emitting device. Therefore, the light source device 32 can supply the endoscope 31 with a plurality of types of illumination light having different spectra, at any timing and at any intensity. For example, in the present embodiment, the light source device 32 controls two or more of purple light, blue light, green light, red light, or light of each of these colors in addition to white light under the control of the light source control unit 47. Light mixed at an arbitrary intensity ratio can be emitted as illumination light at an arbitrary timing and an arbitrary intensity. In addition to this, the light source device 32 can emit light having a specific narrow wavelength band (so-called narrow band light) as illumination light according to the characteristics of the light emitting device or the use of an optical filter. For example, it is possible to emit light in a wavelength band shorter than the green wavelength band, in particular, light in the blue band or purple band in the visible range.

The processor device 33 acquires an endoscopic image from the image sensor 41, or generates an endoscopic image by performing image processing on the endoscopic image acquired from the image sensor 41. Prepare. The image sensor 41 and the endoscope image generation unit 48 constitute an “endoscope image acquisition unit” in the endoscope apparatus 21. The endoscopic image acquisition unit acquires an endoscopic image including the subject by photographing the subject using the illumination light. The medical image processing apparatus 10 is connected to the processor 33. Then, the medical image acquisition unit 11 acquires an endoscope image directly from the endoscope image generation unit 48 of the endoscope apparatus 21.

The medical image analysis processing unit 12 performs analysis processing using an endoscopic image (hereinafter, simply referred to as an endoscopic image) which is a medical image acquired by the medical image acquisition unit 11. Specifically, as shown in FIG. 3, it includes a first abnormal area determination unit 52, a characteristic determination unit 53, and a second abnormal area determination unit 54.

The first abnormal area determination unit 52 determines an abnormal area 119 (see FIG. 6) for each medical image using a plurality of medical images acquired by the medical image acquisition unit 11 (hereinafter referred to as a first abnormal area identification process) ). When the medical image acquisition unit 11 acquires an endoscopic image, the abnormal area refers to an area having characteristics of color or shape different from one or more lesions, surrounding tissues, etc., an area where a drug is dispersed, or the like. , Areas subject to treatment (biopsy, endoscopic mucosal resection (EMR), or endoscopic submucosal dissection (ESD), etc. Is an area containing When the medical image is an endoscopic image, the lesion or the like is, for example, a polyp (a raised lesion), more specifically, a hyperplastic polyp (HP), an SSA / P (sessile serrated adenoma) / polyp), adenomas, cancer, etc. In addition, the area having characteristics of color or shape different from the surrounding tissue or the like is, for example, the subject's redness, atrophy, diverticulum, or a treatment scar.

The first abnormal area determination unit 52 determines whether or not a part or all of the medical image is an abnormal area 119. That is, the first abnormal area determination unit 52 determines the determination of the abnormal area 119 for each pixel, for each small area when dividing the endoscopic image into small areas, or for the entire endoscopic image. Can be done. In the present embodiment, the first abnormal area determination unit 52 determines whether or not the area is an abnormal area 119 for each small area consisting of a predetermined number of pixels.

In addition, the first abnormal area determination unit 52 may determine one or more places of each medical image as the abnormal area 119. In addition, the first abnormal area determination unit 52 uses, for the determination of the abnormal area 119, a medical image captured using special light having a spectrum different from white light. In addition to this, the first abnormal area determination unit 52 includes in the determination result at least the presence or absence of the abnormal area 119 and the "probability" indicating the certainty of the determination.

The characteristic judging unit 53 judges the characteristic of the abnormal area 119 using the judgment result of the first abnormal area judging unit 52, that is, the judgment result of the abnormal area 119 for each medical image (hereinafter referred to as the characteristic judgment processing). The characteristics of the abnormal area 119 include the position, size, range (area, etc.), shape, thickness, length, depth from the surface of the mucous membrane, etc., and the abnormal area in the abnormal area 119. It refers to the density of tissue etc. or other features. In the present embodiment, the characteristic discrimination unit 53 determines the characteristic of the abnormal area 119 as “surface type” or “middle layer” according to the depth at which the abnormality target in the abnormal area 119 is located (depth based on the mucosal surface). It is determined into one of three types of "type" or "deep type".

Further, the characteristic judging unit 53 judges the characteristic of the abnormal area 119 using the accuracy of the judgment result by the first abnormal area judging unit 52. For example, in an endoscopic image in which an abnormality at a relatively shallow position from the mucous membrane surface or the mucous membrane surface is often captured, if the certainty result of the first abnormal area discriminating unit 52 is the highest, it is discriminated as "surface type". . On the contrary, in the endoscopic image in which an abnormality at a relatively deep position under the submucosa is often captured, if the accuracy of the determination result of the first abnormal area determination unit 52 is the highest, it is determined to be "deep".

The second abnormal area determination unit 54 determines an abnormal area using a medical image captured using illumination light having a specific spectrum controlled according to the characteristics of the abnormal area (hereinafter, second abnormal area determination Processing). While the first abnormal area determination process is a general-purpose determination process for determining an abnormal area, the second abnormal area determination process is performed on the premise that there is an abnormal area, by matching it with the characteristics of the abnormal area. This is a determination process for determining an abnormal area more accurately than the first abnormal area determination process. The “specific spectrum” is a spectrum controlled according to the characteristics of the abnormal area 119. For this reason, the “specific spectrum” is often white light used to capture a medical image for display (endoscopic image 101 for display) and a medical image used in the first abnormal area discrimination processing This is different from the spectrum of special light used for capturing (the first endoscopic image 111, the second endoscopic image 112, the third endoscopic image 113, and the fourth endoscopic image 114). However, depending on the characteristics of the abnormal area, it may be the same as the spectrum of the illumination light used for photographing the medical image for display or the medical image used for the first abnormal area discrimination processing.

In the present embodiment, when the characteristic of the abnormal region 119 is “surface type”, the illumination light having a specific spectrum includes violet light and blue light, and the light amount of the violet light is higher than the light amount of the blue light. large. When the characteristic of the abnormal area 119 is “middle layer type”, the illumination light having a specific spectrum includes blue light, green light, and red light, and the light amount of blue light is larger than the light amount of green light and The amount of green light is larger than the amount of red light. When the characteristic of the abnormal area 119 is “deep layer type”, the illumination light having a specific spectrum includes blue light, green light and red light, and the amount of green light is larger than the amount of blue light, and The amount of blue light is larger than the amount of red light.

The second abnormal area judging unit 54 uses a predetermined correspondence relationship between the characteristic of the abnormal area 119 judged by the characteristic judging unit 53 and the above “specific spectrum”, and uses the medical image to be used for judging the abnormal area. Can be determined. In this case, the second abnormal area determination unit 54 holds the correspondence relationship in advance. However, the second abnormal area determination unit 54 can be stored, for example, in a storage unit (not shown) configured by a predetermined memory or the like.

The second abnormal area judging unit 54 determines a medical image to be used for the second abnormal area judging process according to the characteristic of the abnormal area 119 judged by the characteristic judging unit 53, and acquires the medical image from the medical image acquiring unit 11. Do. The medical image acquisition unit 11 acquires in advance a medical image that the second abnormal area determination unit 54 may request. Further, when the medical image acquired by the second abnormal area judging unit 54 is not acquired, the medical image acquiring unit 11 uses the medical treatment for the second abnormal area judging processing according to the request from the second abnormal area judging unit 54. Get an image.

Note that while the first abnormal area determination unit 52 and the second abnormal area determination unit 54 are displaying the endoscopic image 101 for display, which is a medical image for display, on the display unit 13, the display control unit 15 , To determine each abnormal area. The characteristic determination processing performed by the characteristic determination unit 53 is the same. However, the display control unit 15 updates the endoscopic image 101 for display displayed on the display unit 13 while performing the first abnormal area determination process, the characteristic determination process, or the second abnormal area determination process. May. That is, “the determination of each abnormal area is performed while the display control unit 15 is displaying the display endoscopic image 101 which is a display medical image on the display unit 13” means “for display”. This means that the first abnormal area determination process, the characteristic determination process, and the second abnormal area determination process are performed in the background of display control in which the endoscope image 101 is displayed on the display unit 13.

The display unit 13 is a display that displays the medical image acquired by the medical image acquisition unit 11, the determination result of the abnormal area, and the like. That is, the display unit 13 determines the medical image used in the first abnormal area determination unit, the medical image used in the second abnormal area determination unit, or the medical image used in the first abnormal area determination unit and the second abnormal area determination At least one of the medical image acquired in addition to the medical image used in the department is displayed as a medical image for display. A monitor or display included in a device or the like connected to the medical image processing apparatus 10 can be shared and used as the display unit 13 of the medical image processing apparatus 10.

The display control unit 15 controls the display mode of the medical image and the analysis result on the display unit 13. Specifically, the abnormal area 131 determined by the second abnormal area determination unit 54 is superimposed and displayed on the display endoscopic image 101 which is a display medical image, or a display medical image A message (such as a message) indicating that the second abnormal area determination unit 54 has determined an abnormal area is displayed together with the endoscopic image 101 for display. For example, in the present embodiment, the display control unit 15 displays the endoscopic image 101 for display on the display unit 13. Further, in the endoscopic image 101 for display, the display control unit 15 emphasizes the outline of the abnormal area 131 (see FIG. 8), which is the determination result of the second abnormal area determination processing, and the like. The location of the abnormal area 131 is shown in the endoscopic image 101. The display control unit 15 can adjust the color of an area other than the abnormal area 131 or the abnormal area 131 to indicate the location of the abnormal area 131 depending on the setting. Further, depending on the settings, the display control unit 15 may use sound (including sound), light (partial blinking of the endoscopic image 101 for display, etc.), display of coordinates, or any other abnormal method such as display of coordinates. The presence or absence of 131 or the location of the abnormal area 131 can be indicated.

The input reception unit 16 receives input from a mouse, a keyboard, and other operation devices connected to the medical image processing apparatus 10. The operation of each part of the medical image processing apparatus 10 can be controlled using these operation devices.

The integrated control unit 17 integrally controls the operation of each unit of the medical image processing apparatus 10. When the input reception unit 16 receives an operation input using the operation device, the overall control unit 17 controls each unit of the medical image processing apparatus 10 according to the operation input.

The storage unit 18 is abnormal according to need to a storage device (not shown) such as a memory included in the medical image processing apparatus 10, or a storage device (not shown) included in a medical device such as the endoscope apparatus 21 or the PACS 22. Save the judgment result of the area etc.

The flow of the operation of the medical image processing apparatus 10 will be described below. As shown in FIG. 4, the medical image acquisition unit 11 acquires a plurality of endoscopic images 101 for display automatically or by manual selection (step S110), and the display control unit 15 performs medical image processing. The display endoscope image 101 acquired by the acquiring unit 11 is sequentially displayed on the display unit 13 (step S111). In the present embodiment, for example, an endoscopic image 101 for display shown in FIG. 5 is acquired and displayed.

On the other hand, in the background, an endoscopic image for the first abnormal area discrimination processing is acquired (step S112). Specifically, as shown in FIG. 6, a first endoscope image 111 captured using purple special light, a second endoscope image 112 captured using blue special light, and a green color A third endoscopic image 113 captured using special light and a fourth endoscopic image 114 captured using red special light are acquired.

When the medical image acquisition unit 11 acquires the first endoscopic image 111, the second endoscopic image 112, the third endoscopic image 113, and the fourth endoscopic image 114 for the first abnormal area discrimination processing, The first abnormal area determination unit 52 performs a first abnormal area determination process using these (step S113). Specifically, the first abnormal area determination unit 52 uses the fourth endoscopic image 114 for each of the first endoscopic image 111, the second endoscopic image 112, and the third endoscopic image 113. Standardize. Thereby, the influence of the illuminance of the first endoscopic image 111, the second endoscopic image 112, and the third endoscopic image 113 is removed. Then, the first abnormal area judging unit 52 generates an abnormality from the normalized first endoscope image 111, the normalized second endoscope image 112, and the normalized third endoscope image 113. The presence or absence of the area 119 is determined. For example, the first abnormal area determination unit 52 determines a local low signal value area (an area having a small pixel value) as an abnormal area 119 in each of the standardized

endoscopic images

111, 112, and 113.

When the first abnormal area judging unit 52 outputs the presence / absence of the abnormal area 119 and the accuracy of the judgment for each of the standardized

endoscopic images

111, 112, 113, the characteristic judging unit 53 uses this to output the abnormal area 119. The characteristic of is determined (step S114).

Specifically, when there is an abnormal area 119 and the accuracy of the judgment result of the normalized first endoscope image 111 is the highest, the characteristic judging unit 53 determines that the characteristic of the abnormal area 119 is “surface type”. Determine that there is. The first endoscopic image 111 is an endoscopic image captured using special blue light, so that a lesion or the like at a relatively shallow position (so-called surface layer) on the mucous membrane surface or submucous membrane is easily reflected. Therefore, when the lesion or the like is in the vicinity of the surface layer, the lesion or the like can be accurately determined in the first endoscopic image 111 rather than the second endoscopic image 112 and the third endoscopic image 113. As a result, the accuracy of the first abnormal area determination processing is increased. In addition, when there is an abnormal area 119 and the certainty result of the judgment result of the normalized second endoscope image 112 is the highest, the characteristic judging unit 53 judges that the characteristic of the abnormal area is “middle layer type”. . Similarly, when there is an abnormal area 119 and the certainty result of the judgment result of the normalized third endoscope image 113 is the highest, the characteristic judging unit 53 judges that the characteristic of the abnormal area is “deep layer type”. Do. As described above, the characteristic judging unit 53 judges the depth from the surface of the abnormal target in the abnormal area 119 as the characteristic of the abnormal area.

When the characteristic judging unit 53 judges the characteristic of the abnormal area 119, the second abnormal area judging unit 54, according to the characteristic of the abnormal area 119, for example, one or more endoscopes used in the second abnormal area judging process shown in FIG. A mirror image 121 is determined. The endoscopic image 121 for the second abnormal area discrimination processing is an endoscopic image taken using illumination light having a specific spectrum controlled according to the characteristics of the abnormal area 119, and the characteristic of the abnormal area 119 When the “surface type” is the “surface type”, the endoscopic image 121 is an endoscopic image taken using illumination light in which a feature such as a lesion near the surface of the mucous membrane is easily reflected. When the characteristic of the abnormal area 119 is “middle layer type”, the endoscope image 121 is an endoscope image photographed using illumination light in which a feature such as a lesion near the middle layer of the mucous membrane is easily reflected. Further, when the characteristic of the abnormal area 119 is “deep layer type”, the endoscope image 121 is an endoscope image photographed using illumination light in which a feature such as a lesion near the deep layer of the mucous membrane is easily reflected.

The second abnormal area judging unit 54 acquires the endoscope image 121 for the second abnormal area judging processing from the medical image acquiring unit 11 (step S115), and again using the endoscopic image 121, the abnormal area 131. Is determined (step S116). As a result, as shown in FIG. 8, the display control unit 15 displays, on the display unit 13, the abnormal area 131 determined by the second abnormal area determination unit 54 using the endoscope image 121. It is displayed on the mirror image 101 and shows the whereabouts to a doctor or the like. Thereby, the medical image processing apparatus 10 supports diagnosis and the like.

As described above, the medical image processing apparatus 10 firstly performs the first abnormal area determination processing to determine the abnormal area, and determines the characteristic of the determined abnormality. Thereafter, in accordance with the characteristics of the abnormal area, the abnormal area is again determined using a medical image in which the characteristic of the abnormality in the abnormal area is likely to be captured. Therefore, the determination of the abnormal area performed by the medical image processing apparatus 10 is less dependent on the type or the property of a lesion or the like, or the type of an organ having a lesion or the like than in the related art. As a result, the medical image processing apparatus 10 can determine the abnormal area more stably and accurately than in the related art.

In the first embodiment, the second abnormal area judging unit 54 determines the medical image to be used for judging the abnormal area by using the characteristic of the abnormal area and the “specific spectrum” and the correspondence relationship. The area determination unit 54 can calculate the “specific spectrum” using the characteristics of the abnormal area determined by the characteristic determination unit 53 instead of using the correspondence relationship. For example, the second abnormal area judging unit 54 includes the purple light at a light amount proportional to the accuracy of the first abnormal area judging processing of the normalized first endoscope image 111, and the standardized second endoscope image 112. "Includes blue light at a light quantity proportional to the accuracy of the first abnormal area discrimination process of the first, and includes green light at a light quantity proportional to the accuracy of the first abnormal area discrimination process of the standardized third endoscope image 113" A particular spectrum can be calculated. When illumination light including purple light, blue light and green light is used at this ratio, as a result, features such as a lesion included in the abnormal area are easily reflected, so an endoscopic image suitable for the second abnormal area discrimination processing It will be 121. Therefore, if the second abnormal area judging unit 54 uses the endoscope image 121 photographed using the illumination light having the specific spectrum calculated as described above, the accuracy is higher than that in the first embodiment, as in the first embodiment. The abnormal area 131 can be determined well.

Each unit constituting the medical image analysis processing unit 12 such as the first abnormal area judging unit 52, the characteristic judging unit 53, and the second abnormal area judging unit 54 in the first embodiment is learned by machine learning or deep learning. , So-called AI (Artificial Intelligence) program can be used.

In the first embodiment, the second abnormal area judging unit 54 performs the second abnormal area judging process using one endoscopic image 121. However, the second abnormal area judging unit 54 includes a plurality of second abnormal area judging units. The second abnormal area determination process can be performed using the medical image.

As shown in FIG. 9, the medical image analysis processing unit 12 includes the motion determination unit 151 in addition to the first abnormal region determination unit 52, the characteristic determination unit 53, and the second abnormal region determination unit 54. Can be included. When a plurality of endoscope images are used to determine an abnormal area, the movement determination unit 151 detects the movement of the subject image (the relative movement between the subject and the endoscope 31) in the plurality of endoscope images. Determine. For example, in the case of determining the movement of two consecutive frames of endoscopic images, the movement determining unit 151 determines the degree of movement of the subject image based on the correlation of pixel values. More specifically, when G images (endoscope images taken using green light) of two consecutive frames are G1 images and G2 images, each of the G1 images and G2 images A plurality of small square areas (for example, about 20 places) are set inside, the average value of pixel values in each small square area is calculated, and these small square areas are formed between the G1 image and the G2 image. The correlation coefficient of the average pixel value is determined. Then, when the correlation coefficient is equal to or more than a predetermined threshold value, it is determined that there is no movement of the subject image, and when the correlation coefficient is less than a predetermined threshold value, it is judged that the movement of the subject image is present. Do.

When the medical image analysis processing unit 12 includes the motion determination unit 151, when performing the first abnormal area determination process or when using a plurality of endoscopic images in the second abnormal area determination process, a plurality of these plural As the endoscopic image, a combination of endoscopic images determined by the motion determination unit 151 that there is no movement of the subject image can be used. As a result, each of the determination processes can be performed with high accuracy.

Second Embodiment
In each of the above embodiments, the medical image processing apparatus 10 and the endoscope apparatus 21 are separate apparatuses, but the endoscope apparatus 21 can include the medical image processing apparatus 10. In this case, as in the endoscope apparatus 510 shown in FIG. 10, each unit 520 constituting the medical image processing apparatus 10 is provided in the processor apparatus 33. However, the display unit 13 can share the monitor 34 of the endoscope apparatus 21. Moreover, the medical image acquisition part 11 is corresponded to the "endoscope image acquisition part" which the image sensor 41 and the endoscope image generation part 48 form. For this reason, it is sufficient to provide the processor 33 with components other than the medical image acquisition unit 11 and the display unit 13. The configuration of the other components is the same as that of the first embodiment. In addition, a new endoscope apparatus can be configured as a whole of the medical image processing apparatus 10 of each of the above-described embodiments and the other modifications and the endoscope apparatus 21 of FIG. 2.

An endoscope apparatus 510 including the medical image processing apparatus 10 is basically an apparatus for observing a subject in real time. For this reason, the endoscope apparatus 510 performs acquisition of an endoscopic image which is a medical image, first abnormal area discrimination processing, characteristic discrimination processing, second abnormal area discrimination processing, display processing of discrimination results, and the like. It can be performed in real time while photographing an endoscope image or at any timing caused by the operation of various operation units and the like.

Hereinafter, the flow of the operation when the endoscope apparatus 21 including the medical image processing apparatus 10 performs the process of determining an abnormal area in real time will be described. As shown in FIG. 11, the endoscope apparatus 510 sequentially captures an object using, for example, white light, and acquires an endoscopic image 101 for display (step S210), and acquires the acquired display endoscope The mirror image 101 is displayed on the monitor 34 (step S211). The endoscope apparatus 510 executes these steps constantly and repeatedly, for example, at a predetermined frame rate. Therefore, a doctor or the like can observe the subject in real time.

As described above, while displaying the endoscopic image 101 for display, in the background, the general control unit 17 displays the endoscopic image 101 for display during the series of imaging of the endoscopic image 101 for display. The imaging for determining the first abnormal area is intermittently interrupted at a predetermined timing that does not affect continuous display of the endoscopic image 101. That is, the light source control unit 47 switches the illumination light intermittently from white light to special light of purple, special light of blue, special light of green, or special light of red. Then, the image sensor 41 captures an image of the subject with the special light of each of the colors during the imaging using the white light, so that an endoscope image for the first abnormal area discrimination processing similar to the first embodiment, that is, The first endoscope image 111, the second endoscope image 112, the third endoscope image 113, and the fourth endoscope image 114 are acquired (step S212).

The first abnormal area discrimination unit 52 uses the first endoscope image 111, the second endoscope image 112, the third endoscope image 113, and the fourth endoscope image 114 acquired in real time as described above. The first abnormal area determination process is performed (step S213), and the characteristic determination unit 53 determines the characteristic of the abnormal area 119 determined in the first abnormal area determination process (step S214). These are the same as in the first embodiment.

After that, when the second abnormal area judging unit 54 determines a specific spectrum in accordance with the characteristics of the abnormal area 119, the general control unit 17 determines that the display for the display is performed during the imaging of the series of display endoscopic images 101. The imaging for determining the first abnormal area is intermittently interrupted at a predetermined timing that does not affect continuous display of the endoscopic image 101. At this time, the light source control unit 47 switches the illumination light to illumination light having a specific spectrum suitable for the second abnormal area determination processing in accordance with the characteristics of the abnormal area. Specifically, the light source control unit 47 switches the illumination light from white light to illumination light having a specific spectrum determined by the second abnormal area determination unit 54 (step S215), and the image sensor 41 selects one of these colors By imaging the subject with the special light of the above, the endoscope image 121 for the second abnormal area determination processing similar to that of the first embodiment is acquired (step S216). Then, the second abnormal area judging unit 54 performs the second abnormal area judging process using the endoscope image 121 acquired in real time as described above (step S217), and the display control unit 15 performs the second abnormal area. The abnormal area 131 determined by the determination unit 54 using the endoscopic image 121 is displayed in the endoscopic image 101 for display displayed on the display unit 13, and the location is indicated to a doctor or the like.

As described above, the endoscope apparatus 510 including the medical image processing apparatus 10 performs the first abnormal area determination process, the characteristic determination process, the second abnormal area determination process, and the like in real time, and performs the diagnosis, etc. in real time. Can help.

In the second embodiment, the illumination light is switched from white light to purple special light, blue special light, green special light, or red special light when photographing for the first abnormal area discrimination. The first endoscopic image 111, the second endoscopic image 112, the third endoscopic image 113, and the fourth endoscopic image 114 are acquired one by one each, but two or more of them are It can be acquired simultaneously (in one frame). For example, when the image sensor 41 is a color sensor, when the light source control unit 47 performs imaging for determining the first abnormal area, it is red in the same imaging frame as purple special light or blue special light. By emitting special light, the first endoscope image 111 or the second endoscope image 112 and the fourth endoscope image 114 can be acquired in one frame. The violet special light or blue special light and the red special light do not (or do not mix well) at the same time when they emit light simultaneously. Thus, for example, when the first endoscopic image 111 or the second endoscopic image 112 and the fourth endoscopic image 114 are acquired in one frame, the first abnormal area discrimination processing is performed in a total of three frames. The required endoscopic image can be acquired, which is efficient, and the influence on the acquisition and display of the endoscopic image 101 for display can be further reduced.

In the second embodiment, the second abnormal area determination unit 54 determines a specific spectrum from the characteristics of the abnormal area 119 using a predetermined correspondence relationship, as in the first embodiment. When the spectrum of the illumination light can be flexibly changed in real time as in the embodiment, the second abnormal area judging unit 54 does not use the predetermined correspondence, but the abnormal area 119 judged by the characteristic judging unit 53. Specific spectrum of the illumination light for photographing the endoscope image 121 suitable for the second abnormal area discrimination processing using the characteristics of (specifically, light emission of each light emission source for realizing the specific spectrum) It is preferable to calculate the ratio etc.). This is because the spectrum (specific spectrum) of the illumination light to be used can be determined more accurately than the spectrum defined in the correspondence, and as a result, the accuracy of the second abnormal area discrimination processing is further improved.

In the second embodiment, the endoscope apparatus 510 includes the medical image processing apparatus 10. However, as shown in FIG. 12, the diagnostic support apparatus 610 used in combination with the endoscope apparatus 21 and other modality. Can include the medical image processing apparatus 10 of the above-described embodiment and other variations. In addition, as shown in FIG. 13, various inspection devices such as the first inspection device 621, the second inspection device 622,..., The N-th inspection device 623 and the like including the endoscope device 21, for example, via an arbitrary network 626 The medical service support device 630 to be connected can include the medical image processing device 10 of the above-described embodiment and other modified examples.

In addition, the medical image processing apparatus 10, various apparatuses including the medical image processing apparatus 10, and various apparatuses or systems including the functions of the medical image processing apparatus 10 can be used with various modifications described below. .

As a medical image, a normal light image obtained by irradiating light of a plurality of wavelength bands as light of a white band or light of a white band can be used.

When an image obtained by irradiating light of a specific wavelength band is used as a medical image, a specific wavelength band can use a band narrower than the white wavelength band.

The specific wavelength band is, for example, a blue band or a green band in the visible range.

When the specific wavelength band is a blue band or a green band in the visible range, the specific wavelength band includes a wavelength band of 390 nm to 450 nm or 530 nm to 550 nm, and light of the specific wavelength band is 390 nm or more It is preferable to have a peak wavelength within a wavelength band of 450 nm or less or 530 nm or more and 550 nm or less.

The specific wavelength band is, for example, a red band in the visible range.

When the specific wavelength band is a red band in the visible range, the specific wavelength band includes a wavelength band of 585 nm to 615 nm or 610 nm to 730 nm, and light of the specific wavelength band is 585 nm to 615 nm or 610 nm It is preferable to have a peak wavelength within the wavelength band of not less than 730 nm.

The specific wavelength band includes, for example, wavelength bands in which the absorption coefficient is different between oxygenated hemoglobin and reduced hemoglobin, and light of a specific wavelength band is a peak wavelength in the wavelength band where absorption coefficients are different between oxygenated hemoglobin and reduced hemoglobin You can have

A specific wavelength band includes wavelength bands whose absorption coefficients are different between oxyhemoglobin and reduced hemoglobin, and light of a specific wavelength band has peak wavelengths in wavelength bands whose absorption coefficients are different between oxyhemoglobin and reduced hemoglobin In the case, the specific wavelength band includes the wavelength bands of 400 ± 10 nm, 440 ± 10 nm, 470 ± 10 nm, or 600 nm to 750 nm, and the light of the specific wavelength band is 400 ± 10 nm, 440 ± 10 nm, It is preferable to have a peak wavelength in a wavelength band of 470 ± 10 nm, or 600 nm or more and 750 nm or less.

When the medical image is an in-vivo image obtained by copying the inside of a living body, this in-vivo image can have information of fluorescence emitted from a fluorescent substance in the living body.

Further, as fluorescence, fluorescence obtained by irradiating excitation light having a peak wavelength of 390 nm or more and 470 nm or less into a living body can be used.

When the medical image is an in-vivo image obtained by copying the inside of a living body, the above-mentioned specific wavelength band can utilize the wavelength band of infrared light.

When the medical image is an in-vivo image obtained by copying the inside of a living body and a wavelength band of infrared light is used as the above-mentioned specific wavelength band, the specific wavelength band is a wavelength band of 790 nm to 820 nm or 905 nm to 970 nm And light of a specific wavelength band preferably has a peak wavelength in a wavelength band of 790 nm or more and 820 nm or less or 905 nm or more and 970 nm or less.

The medical image acquisition unit 11 acquires a special light image having a signal of a specific wavelength band based on a normal light image obtained by irradiating light of a plurality of wavelength bands as light of the white band or light of the white band. It can have a special light image acquisition unit. In this case, a special light image can be used as a medical image.

A signal of a specific wavelength band can be obtained by an operation based on RGB or CMY color information included in a normal light image.

By an operation based on at least one of a normal light image obtained by irradiating light of a plurality of wavelength bands as light of a white band or light of a white band, and a special light image obtained by irradiating light of a specific wavelength band A feature amount image generation unit that generates a feature amount image can be provided. In this case, the feature amount image can be used as a medical image.

For the endoscope apparatus 21, a capsule endoscope can be used as the endoscope 31. In this case, the light source device 32 and part of the processor device 33 can be mounted on the capsule endoscope.

In each of the above-described embodiments and modifications, the medical image acquisition unit 11, the medical image analysis processing unit 12 (each unit constituting the medical image analysis processing unit 12), the display control unit 15, the input reception unit 16, the general control unit 17, and The hardware-like structure of a processing unit that executes various processes such as the endoscope image generation unit 48 of the endoscope apparatus 21 is various processors as shown below. For various processors, change the circuit configuration after manufacturing CPU (Central Processing Unit), which is a general-purpose processor that executes software (program) and functions as various processing units, FPGA (Field Programmable Gate Array), etc. A programmable logic device (Programmable Logic Device: PLD), which is a possible processor, and a dedicated electric circuit, which is a processor having a circuit configuration specially designed to execute various processes, are included.

One processing unit may be configured of one of these various processors, or configured of a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of a CPU and an FPGA) It may be done. In addition, a plurality of processing units may be configured by one processor. As an example in which a plurality of processing units are configured by one processor, first, one processor is configured by a combination of one or more CPUs and software as represented by computers such as clients and servers; There is a form in which this processor functions as a plurality of processing units. Second, as typified by a system on chip (SoC) or the like, there is a form using a processor that realizes the functions of the entire system including a plurality of processing units in one integrated circuit (IC) chip. is there. Thus, the various processing units are configured using one or more of the above-described various processors as a hardware structure.

Furthermore, the hardware-like structure of these various processors is, more specifically, an electrical circuit (circuitry) in the form in which circuit elements such as semiconductor elements are combined.

Reference Signs List 10 medical image processing apparatus 11 medical image acquisition unit 12 medical image analysis processing unit 13 display unit 15 display control unit 16 input reception unit 17 general control unit 18 storage unit 21, 510 endoscope apparatus 22 PACS
Reference Signs List 31 endoscope 32 light source device 33 processor device 34 monitor 41 image sensor 42 light source portion 43 V-LED
44 B-LED
45 G-LED
46 R-LED
47 light source control unit 48 endoscope image generation unit 52 first abnormal area determination unit 53 characteristic determination unit 54 second abnormal area determination unit 101 endoscopic image for display 111 first endoscopic image 112 second endoscope Image 113 Third endoscopic image 114 Fourth

endoscopic image

119, 131 Abnormal area 121 Endoscopic image for second abnormal area discrimination 151 Motion discrimination unit 520 Each part constituting medical image processing apparatus 610 Diagnosis support device 621 First inspection device 622 Second inspection device 623 Nth inspection device 626 Network 630 Medical service support device S110 to S217 Operation steps

Claims

A medical image acquisition unit for acquiring a medical image including a subject;
A first abnormal area determination unit that determines an abnormal area for each of the medical images using a plurality of the medical images;
A characteristic discrimination unit that discriminates the characteristic of the abnormal area using the discrimination result of the abnormal area for each medical image;
A second abnormal area judging unit for judging the abnormal area using the medical image obtained by photographing the subject using illumination light having a specific spectrum controlled according to the characteristics of the abnormal area;
Medical image processing apparatus comprising:
The medical image processing apparatus according to claim 1, wherein the first abnormal area determination unit uses the medical image captured using special light having a spectrum different from that of white light for determining the abnormal area.
The medical image processing apparatus according to claim 1, wherein the characteristic determination unit determines a depth from a surface of an abnormality target in the abnormal area as the characteristic of the abnormal area.
The medical image processing apparatus according to any one of claims 1 to 3, wherein the characteristic judging unit judges the characteristic of the abnormal area using the accuracy of the judgment result by the first abnormal area judging unit.
The second abnormal area determination unit determines the medical image to be used for the determination of the abnormal area using a predetermined correspondence relationship between the characteristics of the abnormal area and the specific spectrum. The medical image processing device according to any one of 4.
The medical image processing apparatus according to any one of claims 1 to 4, wherein the second abnormal area judging unit calculates the specific spectrum by using the characteristic of the abnormal area.
The medical image processing apparatus according to claim 5, wherein the illumination light having the specific spectrum includes violet light and blue light, and the light amount of the violet light is larger than the light amount of the blue light.
The illumination light having the specific spectrum includes blue light, green light, and red light, the amount of blue light is larger than the amount of green light, and the amount of green light is larger than the amount of red light The medical image processing apparatus according to claim 5.
The illumination light having the specific spectrum includes blue light, green light, and red light, and the amount of green light is larger than the amount of blue light, and the amount of blue light is larger than the amount of red light The medical image processing apparatus according to claim 5.
The medical image used in the first abnormal area judging unit, the medical image used in the second abnormal area judging unit, or the medical image used in the first abnormal area judging unit and the second abnormal area judging The medical image according to any one of claims 1 to 9, further comprising: a display unit configured to display at least one of the medical image acquired in addition to the medical image used in the department as the medical image for display. Processing unit.
The medical image processing apparatus according to claim 10, wherein the first abnormal area judging unit and the second abnormal area judging unit judge each of the abnormal areas while displaying the medical image for display. .
The abnormal area determined by the second abnormal area determination unit is superimposed on the medical image for display and displayed, or the second abnormal area determination unit determines the abnormal area together with the medical image for display. The medical image processing apparatus according to claim 10, further comprising a display control unit that displays a message.
A light source unit that emits plural types of illumination light having different spectra;
An endoscopic image acquisition unit configured to acquire an endoscopic image including a subject by capturing the subject using the illumination light;
A first abnormal area determination unit that determines an abnormal area for each of the endoscopic images using a plurality of the endoscopic images;
A characteristic discrimination unit that discriminates the characteristic of the abnormal area using the discrimination result of the abnormal area for each endoscopic image;
A light source control unit which switches the illumination light to illumination light having a specific spectrum according to the characteristics of the abnormal area;
A second abnormal area judging unit for judging the abnormal area using the endoscope image obtained by photographing the subject using illumination light having the specific spectrum;
Endoscope apparatus provided with.