JP7257544B2 - Information display system and information display method - Google Patents

Description

The present invention relates to an information display system and an information display method for collectively displaying medical images and diagnostic information of a subject by various modalities.

In the medical field, various modalities such as endoscopic diagnostic equipment and ultrasonic diagnostic equipment are used to capture medical images of subjects, and doctors make diagnoses based on the medical images captured by various modalities. It is

For example, in an endoscopic diagnostic apparatus, an object to be observed in a body cavity of a subject is irradiated with illumination light from an endoscope, and the object to be observed irradiated with the illumination light is imaged by an imaging element of the endoscope. endoscopic image is generated and displayed on the monitor. The doctor observes the endoscopic image displayed on the monitor, picks up a region of interest, which is a possible abnormal site, and distinguishes whether or not this region of interest is an abnormal site. Treat the identified area and create a diagnostic report.

Prior art documents related to the present invention include Patent Documents 1 to 4.

In Patent Document 1, in an endoscopic surgical operation device, a monitor screen consists of a main screen displaying an image observed by an endoscope and an image indicating the hardness of living tissue input from a hardness detection unit. It is described that the screen is divided into three parts, a first sub-screen for display and a second sub-screen for displaying an image representing the impedance of the living tissue input from the impedance detection unit.

In Patent Document 2, in an endoscopic surgery support system, photographed image data taken in from a camera attached to the tip of a flexible endoscope and tomography around a region to be treated by the flexible endoscope. It is described that the image data is superimposed and displayed on the display.

Patent Document 3 describes that an ultrasonic image and an endoscopic image as real-time images are separately displayed in an ultrasonic/endoscopic combined system.

In Patent Document 4, in a system having a medical image recording device, an electronic endoscope device, and the like, endoscopic observation image data, inspection result reports, etc. are stored in the medical image recording device 1 and connected. It states that the data held by other medical devices

JP-A-11-347043 JP 2006-320427 A JP-A-08-010263 JP 2004-350734 A

In the case of an endoscopic diagnostic apparatus, endoscopic images can be displayed on the monitor of the endoscopic diagnostic apparatus, but medical images and diagnostic information of the subject obtained by other modalities cannot be displayed on the monitor of the endoscopic diagnostic apparatus. could not be displayed at the same time. The same is true for other modalities.

Therefore, when observing an endoscopic image, a doctor uses the monitor of the endoscopic diagnostic apparatus to observe the endoscopic image, and when observing the ultrasonic image, uses the monitor of the ultrasonic diagnostic apparatus. are observing ultrasound images. In addition, when viewing diagnostic information of a subject, such as patient information and a diagnostic report of a subject, in an external system that stores and manages the diagnostic information of the subject, the monitor of the external system is used to view and read the diagnostic report. is creating

In this way, a doctor cannot collectively observe medical images, diagnostic information, etc. of a subject by various modalities, for example, on the monitor of an endoscopic diagnostic apparatus. In addition, before observing the endoscopic image of the subject, the doctor confirms the diagnostic information of the subject on the monitor in advance, and depending on the progress of the examination of the subject, the endoscopic diagnostic apparatus It was not possible to change the contents displayed on the monitor during the inspection by .

A first object of the present invention is to aggregate and display medical images and diagnostic information of a subject by various modalities in one place, and to change the contents to be displayed according to the progress of the examination of the subject. To provide an information display system and an information display method that can
A second object of the present invention is, in addition to the above-mentioned first object, to be able to display important information so that the user can easily recognize it even when a lot of information is aggregated and displayed. An object of the present invention is to provide an information display system and an information display method.

In order to achieve the above object, the present invention provides a medical image acquisition unit that acquires medical images of a subject from one or more modalities;
a monitor;
a display control unit that displays a medical image of the subject in at least one of two or more divided areas obtained by dividing the display screen of the monitor;
The display control unit provides an information display system that changes the contents of the medical image of the subject to be displayed on the monitor according to the progress of the examination of the subject.

further comprising a diagnostic information acquiring unit for acquiring diagnostic information of the subject from an external system,
The display control unit displays the diagnostic information of the subject on the monitor, or displays at least one of the medical image of the subject and the diagnostic information of the subject in at least one of the two or more divided areas, and displays the subject It is preferable to change the contents of the medical image of the subject and the diagnostic information of the subject to be displayed on the monitor according to the progress of the examination.

Also, the medical image of the subject includes a current endoscopic image of the subject,
Examination of a subject preferably includes a preparation stage, an endoscope insertion stage, a region of interest pick-up stage, a region of interest differentiation and treatment stage, and a diagnostic report generation stage.

Moreover, it is preferable that the display control unit causes the monitor to display at least one of the subject's patient information and the subject's past diagnostic report in the preparation stage.

Further, the display control unit displays the current endoscopic image of the subject in one of the two or more divided areas at the stage of inserting the endoscope, and displays the past endoscopic image of the subject. , the biological information of the subject, the insertion shape of the endoscope in the body cavity of the subject, and the support information for moving the endoscope in the body cavity of the subject, at least one of the two or more divided areas. is preferably displayed in at least one of the other divided areas.

In addition, the display control unit displays the biological information of the subject including the heartbeat value and the blood pressure value in one of the other divided regions, and during the examination of the subject, at least one of the heartbeat value and the blood pressure value reaches the upper limit value. When the value exceeds or falls below the lower limit, an arrow pointing to the subject's biological information is displayed in a divided area other than one of the other divided areas in which the subject's biological information is displayed, out of the two or more divided areas. It is preferable to display

further comprising a region-of-interest estimating unit for estimating the presence or absence of the region of interest and the position of the region of interest from the current endoscopic image of the subject;
The display control unit displays the current endoscopic image of the subject in one of the two or more divided areas in the stage of picking up the region of interest, and displays the current endoscopic image of the subject in one of the two or more divided areas and a normal part in the current endoscopic image of the subject generated by performing color enhancement processing on the current endoscopic image of the subject. It is preferable to display at least one of the color-enhanced images in which the difference in color from the abnormal portion is emphasized in at least one of the other divided regions of the two or more divided regions.

Furthermore, from a blue narrow-band light image in which the vascular structure of the surface layer of the mucous membrane is emphasized in the current endoscopic image of the subject, which is captured by irradiating the subject with white light and blue narrow-band light, the region of interest and An abnormal part discrimination unit that discriminates whether the estimated part is estimated to be an abnormal part, the position of the part estimated to be an abnormal part, and the probability that it is estimated to be an abnormal part,
The display control unit causes the current endoscopic image of the subject to be displayed in one of the two or more divided regions in the region-of-interest discrimination/treatment stage, and the abnormal region is estimated to be discriminated. At least an image representing the position of the part, the probability that the part identified as abnormal is estimated to be abnormal, and the length measurement result representing the length of the part identified as abnormal It is preferable to display one in at least one of the other divided areas of the two or more divided areas.

In addition, the display control unit switches the current endoscopic image of the subject to an image representing the oxygen saturation in the current endoscopic image of the subject in response to an instruction from the user, and divides the image into one divided area. Display is preferred.

Further, the display control unit displays an image representing the position of the site that is estimated to be an abnormal part when it is determined to be an abnormal part or in response to an instruction from the user. It is preferably displayed overlaid on the current endoscopic image.

In addition, after a certain period of time has elapsed or in response to an instruction from the user, the display control unit stops overlaying the image representing the position of the site that is estimated to be abnormal and displays the current state of the subject. is preferably displayed.

In addition, it is preferable that the display control unit displays an image showing the position of the part that is estimated to be an abnormal part by enclosing it with a frame line.

Moreover, it is preferable that the display control unit blinks the frame line at regular time intervals.

In addition, the display control unit displays an arrow pointing to an image representing the position of the part that is estimated to be an abnormal part, and the position of the part that is estimated to be an abnormal part out of the two or more divided regions. It is preferable to display it in a divided area other than one of the other divided areas where the representative image is displayed.

Moreover, it is preferable that the display control unit blinks the arrow at regular time intervals.

In addition, the display control unit performs image processing on the image signal of the endoscope, which outputs an image signal obtained by capturing an image of the subject using illumination light, in response to an instruction to start an examination from the user, and displays the current state of the subject. When connected to at least one of a processor device that outputs an endoscopic image and a light source device that outputs illumination light, the content displayed at the preparation stage automatically changes to the content displayed at the endoscope insertion stage It is preferable to switch to

Furthermore, it is preferable to provide an examination information feedback section that feeds back examination information including the diagnosis result of the subject, examination time, and used equipment to the external system at the stage of creating the diagnosis report.

Further, the display control unit divides the display screen of the monitor into a first divided area and one or more second divided areas having a smaller area than the first divided area, and acquires from one modality out of the one or more modalities. The obtained medical image of the subject is displayed in the first divided area, and at least one of the medical image of the subject obtained from one or more modalities and the diagnostic information of the subject obtained from an external system is displayed. is preferably displayed in at least one of the second divided regions of the .

Further, the display control unit divides the display screen of the monitor into a first divided area and two second divided areas, displays a medical image of the subject acquired from one modality in the first divided area, It is preferable to display at least one of the medical image of the subject acquired from the above modality and the diagnostic information of the subject acquired from the external system in at least one of the two second divided areas.

Further, the aspect ratio of the display screen of the monitor is 16:9, the aspect ratio of the first divided area and the two second divided areas is 5:4,
It is preferable that the first divided area is arranged from the left to the center of the display screen of the monitor, and the two second divided areas are arranged respectively in the upper right and lower right parts of the display screen of the monitor.

The present invention also provides a step in which a medical image acquisition unit acquires a medical image of a subject from one or more modalities;
a step of causing the display control unit to display the medical image of the subject in at least one of two or more divided areas obtained by dividing the display screen of the monitor;
To provide an information display method for changing the content of a medical image of a subject to be displayed on a monitor according to the progress of examination of the subject.

Furthermore, the diagnostic information acquisition unit acquires diagnostic information of the subject from an external system,
Display diagnostic information of the subject on a monitor, or display at least one of a medical image of the subject and diagnostic information of the subject in at least one of two or more divided areas, and the progress of the examination of the subject It is preferable to change the contents of the medical image of the subject and the diagnostic information of the subject to be displayed on the monitor in accordance with the conditions.

The medical image acquisition unit, diagnostic information acquisition unit, attention area estimation unit, abnormal area discrimination unit, display control unit, and examination information feedback unit are preferably hardware or processors that execute programs.

According to the present invention, medical images and diagnostic information of a subject by various modalities can be collectively observed on a monitor possessed by one modality. In addition, according to the present invention, before observing a medical image of the subject, the diagnostic information of the subject can be confirmed on the monitor in advance, and according to the progress of the examination of the subject, during the examination by the modality, , the content displayed on the monitor can be changed. Furthermore, according to the present invention, even when a large amount of information is aggregated and displayed, important information can be displayed so that the user can easily recognize it.

1 is a conceptual diagram of one embodiment showing the configuration of an endoscopic diagnostic apparatus to which the information display system of the present invention is applied; FIG. 1 is a block diagram of an embodiment showing internal configurations of an endoscope, a light source device, and a processor device included in an endoscope diagnostic apparatus; FIG. FIG. 10 is a graph of one embodiment showing emission spectra of violet light V, blue light B, green light G, and red light R emitted from four color LEDs; FIG. 1 is a conceptual diagram of one embodiment representing a display screen of a monitor; FIG. 1 is a conceptual diagram of an embodiment showing a procedure for examination of a subject by an endoscopic diagnostic apparatus; FIG. FIG. 4 is a conceptual diagram of one embodiment showing a display screen of a monitor in a preparatory stage; FIG. 4 is a conceptual diagram of an embodiment showing a display screen of a monitor during the insertion stage of an endoscope; FIG. 3 is a conceptual diagram of one embodiment representing a display screen of a monitor during the region-of-interest pick-up stage; FIG. 11 is a conceptual diagram of one embodiment showing a display screen of a monitor in a region of interest differentiation/treatment stage; FIG. 11 is a conceptual diagram of another embodiment showing a display screen of a monitor during the differentiation/treatment stage of a region of interest; FIG. 10 is a conceptual diagram of one embodiment showing a display screen of a monitor during the diagnostic report creation stage; FIG. 3 is a conceptual diagram of one embodiment representing a display screen of a monitor for alerting a physician. FIG. 4 is a conceptual diagram of another embodiment representing a display screen of a monitor for alerting a physician; FIG. 4 is a conceptual diagram of another embodiment representing a display screen of a monitor for alerting a physician;

The information display system and information display method of the present invention will be described in detail below based on preferred embodiments shown in the accompanying drawings.

FIG. 1 is a conceptual diagram of one embodiment showing the configuration of an endoscopic diagnostic apparatus to which the information display system of the present invention is applied. The endoscope diagnostic apparatus 10 shown in FIG. 1 has an endoscope 12 , a light source device 14 , a processor device 16 , a monitor 18 and a console 19 .

The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16 . The endoscope 12 outputs an image signal obtained by capturing an image of a subject using illumination light. It has an operation portion 12b, and a curved portion 12c and a distal end portion 12d provided on the distal end side of the insertion portion 12a. By operating the angle knob 12e of the operation portion 12b, the bending portion 12c is bent. Along with this bending action, the distal end portion 12d is directed in a desired direction.

In addition to the angle knob 12e, the operation section 12b is provided with a mode switching SW (switch) 13a. The mode switching SW 13a is used for switching operation between four types of modes: a normal observation mode, a first special observation mode, a second special observation mode, and a third special observation mode.

The normal observation mode is a mode for observing a normal image captured by irradiating the subject with white light as the current endoscopic image of the subject.
In the first special observation mode, the current endoscopic image of the subject is captured by irradiating the subject with white light and blue narrowband light, and the surface layer of the mucosa in the current endoscopic image of the subject is captured. This is the mode for observing the first special image (blue narrowband light image) in which the blood vessel structure is emphasized.
The blue narrow-band light is not limited to narrow-band light in the wavelength range of blue, and may be narrow-band light in the wavelength range from blue to violet.
The second special observation mode is a current endoscopic image of the subject, for example, a normal part and an abnormal part in the current endoscopic image of the subject, which is generated by performing color enhancement processing on the first special image. This is a mode for observing a second special image (color-enhanced image) in which the difference in color from is emphasized.
The third special observation mode is a mode for observing a third special image (oxygen saturation image) representing the oxygen saturation of blood hemoglobin in the normal image as the current endoscopic image of the subject.
Note that one or more special observation modes other than the first to third special observation modes may be included.

FIG. 2 is a block diagram of one embodiment showing internal configurations of an endoscope, a light source device, and a processor device included in the endoscope diagnostic apparatus. The light source device 14 outputs illumination light, and as shown in FIG. 4-color LEDs (light sources) including Diode) 20c and R-LED (Red Light Emitting Diode) 20d, a light source control unit 21 that controls the driving of these 4-color LEDs 20a to 20d, and a dichroic mirror, etc. and an optical path coupling portion 23 for coupling the optical paths of the four color lights emitted from the four color LEDs 20a to 20d.

FIG. 3 is a graph of one embodiment showing emission spectra of violet light V, blue light B, green light G, and red light R from four color LEDs. As shown in FIG. 3, the V-LED 20a emits violet light V with a central wavelength of 405±10 nm and a wavelength range of 380-420 nm. The B-LED 20b generates blue light B with a central wavelength of 460±10 nm and a wavelength range of 420-500 nm. The G-LED 20c generates green light G with a wavelength range of 480-600 nm. The R-LED 20d emits red light R with a central wavelength of 620-630 nm and a wavelength range of 600-650 nm.

In the case of the first special observation mode and the second special observation mode, the light source control section 21 turns on all of the V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d, for example. At that time, the emission amount of the violet light V, which has a high absorption rate in the surface layer blood vessels, is set to be relatively high, and the emission amount of the green light G, which has a high absorption rate in the middle layer blood vessels, is set to be relatively low. , violet light V, blue light B, green light G, and red light R are irradiated so that the light amount ratio of four colors of light becomes a predetermined ratio.
In the case of the third special observation mode, the light source control section 21 turns on the B-LED 20b, the G-LED 20c, and the R-LED 20d, for example. At that time, irradiation of first blue light B1, green light G1 and red light R1 having a center wavelength of 473±10 nm generated by limiting the wavelength band of blue light B using a first optical filter, Irradiation of second blue light B2, green light G2, and red light R2 having a center wavelength of 445±10 nm generated by limiting the wavelength band of blue light B using a filter is alternately repeated at regular intervals. .

As shown in FIG. 2, the light guide (LG) 41 is built in the endoscope 12 and the universal cord (the cord connecting the endoscope 12, the light source device 14 and the processor device 16). A multimode fiber can be used as the light guide 41 .

A distal end portion 12d of the endoscope 12 is provided with an illumination optical system 30a and an imaging optical system 30b.
The illumination optical system 30a has an illumination lens 45, and the light coupled by the optical path coupling section 23 is propagated to the distal end portion 12d of the endoscope 12 via the light guide 41 and the illumination lens 45, and the inside of the subject is observed. Target is irradiated.
The imaging optical system 30 b has an objective lens 46 and an imaging sensor 48 . Reflected light from the observation target enters the imaging sensor 48 via the objective lens 46 , and a reflected image of the observation target is formed on the imaging sensor 48 .

The imaging sensor 48 is a color imaging sensor that captures a reflected image of the subject and outputs an image signal. As the imaging sensor 48, a CCD (Charge Coupled Device) imaging sensor, a CMOS (Complementary Metal-Oxide Semiconductor) imaging sensor, or the like can be used. The imaging sensor 48 is a color imaging sensor for obtaining RGB image signals of three colors of R (red), G (green), and B (blue), that is, R pixels provided with R filters and G filters. It is an RGB image sensor having G pixels provided with a filter and B pixels provided with a B filter.

A CDS (Correlated Double Sampling)/AGC (Auto Gain Control) circuit 50 and an A/D (Analog/Digital) converter 52 are provided inside the endoscope 12 . An image signal output from the imaging sensor 48 is input to the CDS/AGC circuit 50 . The CDS/AGC circuit 50 performs correlated double sampling (CDS), automatic gain control (AGC), etc. on the image signal, which is an analog signal.

The image signal that has passed through the CDS/AGC circuit 50 is converted from an analog image signal to a digital image signal by an A/D converter 52 . The A/D converted digital image signal is input to the processor device 16 .

The processor device 16 is electrically connected to the monitor 18 and the console 19 as well as various modalities 80 and an external system 82 as shown in FIG. The processor device 16 image-processes the image signal to generate a current endoscopic image of the subject, and outputs medical images of various modalities 80 including the current endoscopic image of the subject, Reception unit 53, DSP (Digital Signal Processor) 56, noise removal unit 58, image processing switching unit 60, normal image processing unit 62, special image processing unit 64, video signal generation unit 66, medical image It has an acquisition unit 68 , a diagnostic information acquisition unit 70 , a region-of-interest estimation unit 72 , an abnormal part discrimination unit 74 , a display control unit 76 , and an examination information feedback unit 78 .

The medical image acquisition unit 68, the diagnostic information acquisition unit 70, the attention area estimation unit 72, the abnormal part discrimination unit 74, the display control unit 76, the examination information feedback unit 78, and the monitor 18 constitute an embodiment of the information display system of the present invention. Configure.
The information display system acquires medical images of each subject (patient) from various modalities 80, further acquires diagnostic information (medical information) of each subject from various external systems 82, and obtains each display at least one of medical images and diagnostic information for a subject.

The receiving unit 53 receives RGB image signals, which are digital signals, from the endoscope 12 .

The DSP 56 performs various signal processing such as defect correction processing, offset processing, gain correction processing, linear matrix processing, gamma conversion processing and demosaicing processing on the received image signal. In the defect correction process, signals of defective pixels of the imaging sensor 48 are corrected. In the offset processing, dark current components are removed from the RGB image signals subjected to the defect correction processing, and an accurate zero level is set. In the gain correction process, the signal level is adjusted by multiplying the RGB image signal after the offset process by a specific gain. The RGB image signals after gain correction processing are subjected to linear matrix processing for enhancing color reproducibility. After that, the brightness and saturation are adjusted by a gamma conversion process. The RGB image signals after linear matrix processing are subjected to demosaic processing (also referred to as isotropic processing or simultaneous processing), and signals of missing colors in each pixel are generated by interpolation. This demosaicing process causes all pixels to have RGB signals.

The noise removal unit 58 removes noise from the RGB image signal by performing noise removal processing such as a moving average method and a median filter method on the RGB image signal that has undergone gamma correction or the like in the DSP 56 . The noise-removed RGB image signal is input to the image processing switching section 60 .

When the normal observation mode is set by the mode switching SW 13a, the image processing switching unit 60 inputs the RGB image signal to the normal image processing unit 62, and switches between the first special observation mode, the second special observation mode, or the When the third special observation mode is set, switching is performed so that the RGB image signal is input to the special image processing section 64 .

The normal image processing unit 62 performs color conversion processing, color enhancement processing, structure enhancement processing, and the like on the RGB image signal. In the color conversion processing, 3×3 matrix processing, tone conversion processing, three-dimensional LUT processing, and the like are performed on digital RGB image signals to convert them into color-converted RGB image signals. Next, various color enhancement processes are applied to the RGB image signals that have undergone the color conversion process. Structural enhancement processing such as spatial frequency enhancement is performed on the color-enhanced RGB image signals. The RGB image signal subjected to structure enhancement processing is input from the normal image processing unit 62 to the video signal generation unit 66 as the RGB image signal of the normal image.

The special image processing section 64 operates when the first special observation mode, the second special observation mode, or the third special observation mode is set. The special image processing unit 64 includes a first special image processing unit 64a that performs image processing for the first special image on the RGB image signal to generate the first special image, and a first special image processing unit 64a that generates the first special image. It has a second special image processing unit 64b that performs image processing to generate a second special image, and a third special image processing unit 64c that performs image processing for the third special image to generate a third special image. .

The first special image processing unit 64a processes the RGB image signals of the first special image captured by irradiating the subject with white light and blue narrowband light, for example, similarly to the normal image processing unit 62. Conversion processing, color enhancement processing, structure enhancement processing, etc. are performed to generate a blue narrow-band light image (first special image) in which the vascular structure of the mucosal surface layer of the subject is emphasized.

The second special image processing unit 64b calculates, for example, the signal ratio B/G between the G image signal and the B image signal of the RGB image signals of the first special image, and calculates the signal ratio G between the R image signal and the G image signal. /R ratio is calculated, and with respect to the signal ratio B / G and the signal ratio G / R, the saturation and hue of the first range where the normal mucosa is distributed are the same, and the first range and the abnormal part is distributed, and at least one of the difference in hue between the first range and the third range in which the deep-layer blood vessels existing under the abnormal portion is distributed, for example By performing color enhancement processing so that reddish colors become redder and whitish colors become whiter, the difference in color between normal and abnormal areas in the current endoscopic image of the subject is emphasized. A color-enhanced image (second special image) is generated.

The third special image processing unit 64c alternately irradiates the first blue light B1, the green light G1, and the red light R1, and the second blue light B2, the green light G2, and the red light R2. For example, the signal ratio B1/G2 between the B1 image signal and the G2 image signal, the signal ratio R2/G2 between the R2 image signal and the G2 image signal, and the G2 image signal and B2 Calculate the signal ratio G2/B2 of the image signal, refer to the LUT (Look Up Table) that stores the correlation between these signal ratios and the oxygen saturation, and from the signal ratio, the blood of the subject The oxygen saturation of hemoglobin is calculated, and an oxygen saturation image (third special image) is generated by imaging the oxygen saturation.
The RGB image signals of the first special image, the second special image, and the third special image generated by the special image processing section 64 are input to the video signal generation section 66 .

The video signal generation unit 66 converts the RGB image signal input from the normal image processing unit 62 or the special image processing unit 64 into a video signal for display as an image that can be displayed on the monitor 18 . The video signal is input to the medical image acquisition unit 68 as an endoscopic image (medical image).

The medical image acquisition unit 68 acquires medical images (still images or moving images) of the subject, including the current endoscopic image of the subject, from one or more modalities 80 including the endoscopic diagnostic apparatus 10 . The medical image acquisition unit 68 may acquire medical images of the subject from only one modality, or may acquire medical images of the subject from two or more modalities. A medical image of the subject output from the medical image acquisition unit 68 is input to a region-of-interest estimation unit 72 , an abnormal part discrimination unit 74 , a display control unit 76 and an examination information feedback unit 78 .

The modality 80 is various medical devices that take medical images of a subject, such as an endoscope diagnostic device, an ultrasonic diagnostic device, an ultrasonic endoscope diagnostic device, an X-ray imaging device, a computed tomography device (CT ) and magnetic resonance imaging (MRI). The modality 80 includes a device for acquiring biological information of the subject, a device for acquiring the insertion shape of the endoscope 12 in the body cavity of the subject, and a device for moving the endoscope 12 in the body cavity of the subject. A device or the like for acquiring support information for the purpose may be included.

A device that acquires biological information of a subject is, for example, a heart rate monitor that acquires the heart rate value of the subject and a sphygmomanometer that acquires the blood pressure value of the subject.
The device for acquiring the insertion shape of the endoscope 12 is, for example, an endoscope insertion shape observation device that acquires the three-dimensional insertion shape of the endoscope 12 in the body cavity of the subject in real time using magnetism or the like. is.
A device that acquires support information includes, for example, a learning endoscopic image, the name of an organ displayed in this learning endoscopic image, the range of the organ (also referred to as a region or boundary), the tip of the endoscope 12 Based on a learned model in which the relationship between the position and orientation of the part 12d and a plurality of learning ultrasonic images has been learned, a diagnostic endoscopic image of the subject is displayed on this diagnostic endoscopic image. This apparatus estimates the name of the organ in which the patient is present, the range of the organ, the position and orientation of the distal end portion 12d of the endoscope 12, and acquires such information as the aforementioned support information.

The modality 80 includes, for example, a medical image management system (PACS: Picture Archiving and Communication Systems) for storing and managing medical images of the subject captured by various modalities 80, and the insertion portion 12a of the endoscope 12. Modalities other than the above may be included, such as a device that acquires the state of inflation and deflation of a double-balloon that is attached to and used.

A learning method is not particularly limited as long as it can learn the above relationship from a plurality of learning endoscopic images and generate a trained model.
As a learning method, for example, deep learning using a hierarchical neural network is used as an example of machine learning (machine learning), which is one of artificial intelligence (AI) technologies. be able to.
Note that machine learning other than deep learning may be used, artificial intelligence technology other than machine learning may be used, and learning methods other than artificial intelligence technology may be used. The same applies to the learning method of other trained models to be described later.

The diagnostic information acquisition unit 70 acquires diagnostic information of the subject from the external system 82 . The diagnostic information of the subject output from the diagnostic information acquisition unit 70 is input to the display control unit 76 .

The external system 82 provides diagnostic information, etc. of the subject to the information display system and is used by the doctor to create a diagnostic report. Technology (information technology) system, etc. can be used.
The diagnostic information of the subject includes patient information of the subject (information such as name, address, and contact information), and a diagnostic report (information such as medical conditions, contraindication information, doctor's observations, progress information, and the like).

The region-of-interest estimation unit 72 estimates the presence or absence of the region of interest, the position of the region of interest, and the like from the current medical image of the subject, for example, a normal image (endoscopic image). The estimation result of the attention area estimation unit 72 is input to the abnormal part discrimination unit 74 and the display control unit 76 .

The region of interest is a part of the subject that may not be normal, that is, a part of the subject that may be abnormal.
The region-of-interest estimating unit 72 calculates the relationship between, for example, a learning endoscopic image, the presence/absence of a region of interest displayed in this endoscopic image for learning, and the position of the region of interest, using a plurality of endoscopic images for learning. Presence/absence of a region of interest and the position of the region of interest displayed in the diagnostic endoscopic image can be estimated from the diagnostic endoscopic image of the subject based on the learned model learned for .
Note that the method by which the region-of-interest estimating unit 72 estimates the presence or absence of the region of interest, the position of the region of interest, and the like from the current medical image of the subject is not particularly limited.

The abnormal area discriminating unit 74 determines whether or not the area estimated to be the attention area is estimated to be an abnormal area from the current medical image of the subject, for example, the first special image, the position of the area estimated to be the abnormal area, and the Discriminate the probability of being estimated as an abnormal part. The discrimination result of the abnormal part discrimination section 74 is input to the display control section 76 .

The abnormal part discriminating unit 74 determines whether or not, for example, a learning endoscopic image and a part estimated to be a region of interest displayed in this learning endoscopic image is estimated to be an abnormal part. This diagnostic It is possible to discriminate whether or not the site estimated to be the region of interest displayed in the endoscopic image is estimated to be an abnormal site, the position of the site estimated to be an abnormal site, and the probability of being estimated to be an abnormal site. .
In addition, the abnormal area discriminating unit 74 determines whether or not the area estimated to be the attention area is estimated to be an abnormal area from the medical image such as the endoscopic image, the position of the area estimated to be the abnormal area, and the abnormal area. A method for discriminating the estimated probability or the like is not particularly limited.

The display control unit 76 controls the display of the medical image of the subject, the diagnostic information of the subject, the estimation information of the attention area, the discrimination information of the abnormal part, and the like on the monitor 18 . The medical image of the subject, the diagnostic information of the subject, the estimation information of the attention area, the discrimination information of the abnormal part, and the like output from the display control unit 76 are input to the monitor 18 .

The display control unit 76 displays the medical image of the subject on the monitor 18 without dividing the display screen of the monitor 18, or divides the medical image of the subject into two or more by dividing the display screen of the monitor 18. It can be displayed in at least one of the areas. In this case, the display control unit 76 can sequentially change the content of the medical image of the subject to be displayed on the display screen or divided areas of the monitor 18 according to the progress of the examination of the subject.
The number, area, aspect ratio, arrangement, etc. of the divided regions are not particularly limited.

In addition, the display control unit 76 causes the monitor 18 to display the medical image or diagnostic information of the subject without dividing the display screen of the monitor 18, or displays at least one of the medical image and diagnostic information of the subject. , at least one of the two or more divided areas. In this case, the display control unit 76 can sequentially change the content of the subject's medical image and diagnostic information to be displayed on the display screen or divided areas of the monitor 18 according to the progress of the subject's examination.

The examination information feedback unit 78 feeds back the examination information including the diagnosis result of the subject by the doctor, the examination time, the used equipment, etc. to the external system 82 at the stage of creating the diagnosis report.

The examination information is not particularly limited as long as it is information related to the examination of the subject, and may include various types of information other than the above, such as medical images of the subject.
A diagnosis report is a medical examination record of a subject created by a doctor (user) based on examination information and medical images obtained by various modalities 80, and includes the subject's medical condition, contraindication information, the doctor's findings on the medical condition, the subject's Treatment for the patient, progress information of the subject, etc. are recorded.

The monitor 18 is, for example, a display device such as a liquid crystal display, and displays a medical image of the subject, diagnostic information of the subject, estimation information of the attention area, discrimination information of the abnormal part, and the like under the control of the display control unit 76 .

The console 19 is an input device for inputting instructions from a doctor, such as a keyboard and a mouse, and functions as a UI (User Interface) for receiving input operations such as function settings.

Next, the operation of the endoscope diagnostic apparatus 10 will be described.

First, the normal observation mode is set by the mode switching switch 13a, and the insertion portion 12a of the endoscope 12 is inserted into the specimen.

In the normal observation mode, the light source controller 21 turns on all of the V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d, for example. The four colors of light emitted from the LEDs 20a to 20d are coupled by the optical path coupling section 23, and the four colors of violet light V, blue light B, green light G, and red light R are mixed to produce a substantially white light. It becomes light, and is irradiated to an observation target inside the subject through the light guide 41 and the illumination lens 45 .
Reflected light from the observation target enters the imaging sensor 48 via the objective lens 46 , and a reflected image of the observation target is formed on the imaging sensor 48 . An image signal output from the imaging sensor 48 passes through a CDS/AGC circuit 50 and is converted into a digital image signal by an A/D converter 52 . The A/D-converted digital image signal is input to the processor unit 16 and received by the receiving unit 53 .
The digital image signal received by the receiving unit 53 is subjected to various signal processing by the DSP 56 to obtain RGB image signals of a normal image, and is subjected to noise removal processing by the noise removing unit 58 to obtain RGB image signals of the normal image. noise is removed from The noise-removed RGB image signal of the normal image is input to the normal image processing unit 62 by the image processing switching unit 60, and the normal image processing unit 62 performs image processing for the normal image.
The image-processed RGB image signal of the normal image is converted by the video signal generator 66 into a video signal to be displayed as the normal image.

When the mode switch SW 13a switches to the first special observation mode, the light source controller 21 turns on all of the V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d, for example. At that time, the emission amount of the violet light V, which has a high absorption rate in the superficial blood vessels, is set to be relatively high, and the emission amount of the green light G, which has a high absorption rate in the middle blood vessels, is set to be relatively low. , violet light V, blue light B, green light G, and red light R are irradiated so that the light amount ratio of four colors of light becomes a predetermined ratio.
After that, the operation until noise removal processing is performed by the noise removal unit 58 is the same as in the case of the normal observation mode, and the noise removal unit 58 outputs the RGB image signals of the first special image from which the noise has been removed. output. The noise-removed RGB image signal of the first special image is input to the first special image processing unit 64a by the image processing switching unit 60, and the image processing for the first special image is performed by the first special image processing unit 64a. is applied.
The image-processed RGB image signal of the first special image is converted by the video signal generator 66 into a video signal to be displayed as the first special image.

When the mode switching SW 13a is used to switch to the second special observation mode, the operation until image processing for the first special image is performed by the first special image processing section 64a is the same as in the first special observation mode. The first special image processing unit 64a outputs RGB image signals of the first special image that has undergone image processing.
The RGB image signal of the first special image subjected to the image processing is subjected to image processing for the second special image by the second special image processing unit 64b, and the RGB image signal of the second special image is obtained.
The RGB image signal of the second special image is converted by the video signal generator 66 into a video signal to be displayed as the second special image.

When the mode switching SW 13a switches to the third special observation mode, the light source controller 21 turns on the B-LED 20b, the G-LED 20c, and the R-LED 20d, for example. At this time, the irradiation of the first blue light B1, the green light G1 and the red light R1 and the irradiation of the second blue light B2, the green light G2 and the red light R2 are alternately repeated at regular intervals.
After that, the operation until noise removal processing is performed by the noise removal unit 58 is the same as in the case of the normal observation mode. An image signal is output. The noise-removed RGB image signals of the two types of endoscopic images are input to the third special image processing unit 64c by the image processing switching unit 60, and processed by the third special image processing unit 64c for the third special image. image processing is applied.
The image-processed RGB image signal of the third special image is converted by the video signal generator 66 into a video signal to be displayed as the third special image.

Next, the display screen of the monitor 18 controlled by the display control unit 76 will be described.

The display control unit 76 can divide, for example, the display screen of the monitor 18 into a first divided area and one or more second divided areas having a smaller area than the first divided area. In this case, the display control unit 76 displays, for example, the medical image of the subject acquired from one modality out of the one or more modalities 80 in the first divided area, and displays the subject image acquired from the one or more modalities 80. and at least one of the diagnostic information of the subject acquired from the external system 82 can be displayed in at least one of the one or more second divided areas.

FIG. 4 is a conceptual diagram of one embodiment representing a display screen of a monitor. As shown in FIG. 4, the display control unit 76 divides the display screen of the monitor 18 into, for example, a first divided area (Area 1) and two second divided areas (Area 2, Area 3) smaller than the first divided area. ) can be divided into three regions. In this case, the display control unit 76 causes the medical image of the subject acquired from one modality out of the one or more modalities 80 to be displayed in the first divided area, and the image of the subject acquired from the one or more modalities 80 is displayed. At least one of the medical image and diagnostic information of the subject obtained from the external system 82 can be displayed in at least one of the two second divided areas.

Further, as shown in FIG. 4, the display control unit 76 displays information that can be displayed as text information among the diagnostic information of the subject, such as patient information of the subject, at the bottom of the display screen of the monitor 18. It can be displayed in the third divided area (Area 4) consisting of one or more lines extending in the horizontal direction. In this case, the display screen of the monitor 18 is divided into four areas consisting of a first divided area, two second divided areas and a third divided area.

When the information display system of the present invention is incorporated in the endoscopic diagnostic apparatus 10 as in this embodiment, the display control unit 76 displays the current normal of the subject acquired from the endoscopic diagnostic apparatus 10, for example. It is desirable to display the image in the first segmented area. As a result, the doctor can observe the current normal image of the subject in a relatively large size on the monitor 18 of the endoscopic diagnostic apparatus 10 that is being used for examination of the subject, while at the same time using various modalities 80 It is possible to view medical images, diagnostic information of the subject, and the like.

What is displayed in each divided area is not particularly limited. For example, medical images obtained by various modalities 80 other than endoscopic images or diagnostic information of the subject may be displayed in the first divided area or may be displayed in the second divided area. Also, an endoscopic image other than the normal image, for example, at least one of the first special image, the second special image and the third special image may be displayed in the first divided area, or may be displayed in the second divided area. may

When the aspect ratio of the display screen of the monitor 18 is 16 (horizontal):9 (vertical), the aspect ratios of the first divided area and the two second divided areas are 5 (horizontal):4 (vertical). is desirable. In this case, as shown in FIG. 4, for example, the first divided area is arranged from the left part to the central part of the display screen of the monitor 18, and the two second divided areas are arranged in the upper right part of the display screen of the monitor 18, respectively. and lower right. Alternatively, the first divided area may be arranged from the center to the right part of the display screen of the monitor 18, and the two second divided areas may be arranged respectively at the upper left part and the lower left part of the display screen of the monitor 18. .

In recent years, the aspect ratio of the display screen of the monitor 18 is mainly 16:9. On the other hand, the aspect ratio of the light receiving surface of the imaging sensor of the modality 80 that captures medical images, in other words, the aspect ratio of medical images is often 5:4. Therefore, as described above, by using the monitor 18 with an aspect ratio of 16:9, the display screen of the monitor 18 with an aspect ratio of 16:9 can be displayed with an aspect ratio of 5:4 without trimming. can be preferably displayed in each of the three divided areas shown in FIG.

Next, a procedure for examining a subject by the endoscopic diagnosis apparatus 10 will be described.

In the endoscopic diagnostic apparatus 10, current endoscopic images of the subject are sequentially captured. As shown in FIG. 5, an examination of a subject by the endoscope diagnostic apparatus 10 includes a preparation stage, an endoscope insertion stage, an attention area pick-up stage, an attention area differentiation/treatment stage, and a diagnosis report preparation stage. Including the creation stage.

The preparation stage is a stage in which the doctor makes preparations before observing the endoscopic image of the subject using the endoscopic diagnostic apparatus 10 .

In the preparation stage, the display control unit 76 causes the monitor 18 to display at least one of the subject's diagnostic information, for example, the subject's patient information and the subject's past diagnostic reports, as shown in FIG. The display control unit 76 can also cause the monitor 18 to display past medical images of the subject, similar cases of the subject's medical condition, and the like.
A past medical image of the subject is displayed on the display screen of the monitor 18 shown in FIG.
As a result, the doctor can grasp the disease condition of the subject and confirm in advance the part of the subject to be observed before actually observing the endoscopic image of the subject. .

The endoscope insertion step is a step in which the doctor inserts the endoscope 12 into the body cavity of the subject and observes the endoscopic image of the subject.

At the stage of inserting the endoscope, the display control unit 76 displays, for example, the current endoscopic image of the subject in one of the two or more divided areas, as shown in FIG. at least one of past endoscopic images, biological information of the subject, insertion shape of the endoscope 12 in the body cavity of the subject, and support information for moving the endoscope 12 in the body cavity of the subject One is displayed in at least one of the other divided areas of the two or more divided areas.
In the display screen of the monitor 18 shown in FIG. 7, the normal image is displayed as the current endoscopic image of the subject in the first divided area. In addition, four past endoscopic images (thumbnail images) of the subject are displayed in the upper right second divided area, and an endoscopic image in the body cavity of the subject is displayed in the lower right second divided area. An image representing 12 insertion shapes is displayed.
As a result, while observing the current endoscopic image of the subject, the doctor can simultaneously view the past endoscopic images of the subject, the biological information of the subject, and the insertion shape of the endoscope 12 in the body cavity of the subject. , and support information for moving the endoscope 12 within the body cavity of the subject, while confirming at least one of them, the endoscope 12 is safely and reliably moved within the body cavity of the subject, and observed. Subject can be observed.

The display control unit 76 causes the endoscope 12, which outputs an image signal obtained by imaging the subject in response to an instruction to start an examination from a doctor or using illumination light, to image-process the image signal to display the image of the subject. When connected to at least one of the processor device 16 that outputs the current endoscopic image and the light source device 14 that outputs the illumination light, the content to be displayed in the preparation stage to the content to be displayed in the insertion stage of the endoscope switch to

The step of picking up a region of interest is a step in which the region-of-interest estimating section 72 picks up a site estimated to be the region of interest in the current endoscopic image of the subject. For example, according to an instruction from a doctor, the endoscope insertion stage is shifted to the attention area pick-up stage.

In the stage of picking up the region of interest, the display control unit 76 displays, for example, the current endoscopic image of the subject in one of the two or more divided regions, as shown in FIG. At least one of the image representing the position of the region estimated to be the region of interest in the current endoscopic image, and at least one of the color-enhanced image in at least one of the other divided regions of the two or more divided regions display.
In the display screen of the monitor 18 shown in FIG. 8, the normal image is displayed as the current endoscopic image of the subject in the first divided area. In addition, in the second divided area in the upper right part, an image representing the position of the part estimated to be the attention area in the current endoscopic image of the subject is displayed, and the position of the part estimated to be the attention area is indicated by the frame line. , and a color-enhanced image including the position of the part estimated to be the attention area is displayed in the second divided area at the lower right.
As a result, while observing the current endoscopic image of the subject, the doctor can, at the same time, obtain an image representing the position of the estimated region of interest in the current endoscopic image of the subject and a color-enhanced image. At least one of the images can be identified.

In the discrimination/treatment stage of the region of interest, the abnormal region discriminating unit 74 discriminates whether or not the region estimated to be the region of interest is estimated to be an abnormal region. It is a step of diagnosing whether or not the site that is presumed to be a lesion is a lesion, and treating the site that is diagnosed as a lesion.

The display control unit 76 displays, for example, the current endoscopic image of the subject in one of the two or more divided regions, as shown in FIG. An image showing the position of the part that was identified as an abnormal part, the probability that the part that was identified as an abnormal part is estimated to be an abnormal part, and the number of parts that were identified as an abnormal part. At least one of the length measurement results representing the length is displayed in at least one of the other divided areas of the two or more divided areas.
In the display screen of the monitor 18 shown in FIG. 9, the normal image is displayed as the current endoscopic image of the subject in the first divided area. In addition, in the second divided area in the upper right part, an image representing the position of the site that is estimated to be an abnormal part is displayed in a predetermined color, such as green, and the second divided area in the lower right part. indicates that the probability that the part that is discriminated as being an abnormal part is estimated to be an abnormal part is 90%.
As a result, while observing the current endoscopic image of the subject, the doctor can simultaneously view an image showing the position of the site that is estimated to be an abnormal site, and the site that is estimated to be an abnormal site. While confirming at least one of the probability of being estimated as an abnormal part and the length measurement result representing the length of the part determined as being estimated as an abnormal part, the part determined as being estimated as being abnormal is determined. Whether or not it is a lesion can be diagnosed.

Further, the display control unit 76 converts the current endoscopic image of the subject to the current endoscopic image of the subject as shown in FIG. It is possible to switch to an image representing the oxygen saturation in the mirror image and display it in one segmented region, for example, the first segmented region. Alternatively, the display control unit 76 may display the current endoscopic image of the subject in one divided area and the oxygen saturation image in at least one of the other divided areas.
In the display screen of the monitor 18 shown in FIG. 10, an image representing the oxygen saturation is displayed as the current endoscopic image of the subject in the first divided area. Four past endoscopic images (thumbnail images) of the subject are displayed in the second divided area in the upper right part, and a color-enhanced image is displayed in the second divided area in the lower right part.
As a result, while observing the image representing the oxygen saturation, the doctor can more accurately diagnose whether or not the site that is presumed to be abnormal is a lesion.

In the region-of-interest pick-up stage, the region-of-interest estimating unit 72 picks up a region that is estimated to be the region of interest in the current endoscopic image of the subject, and in the region-of-interest discrimination/treatment stage, Any of the processes of discriminating whether or not the site estimated to be the region of interest is estimated to be an abnormal part by the abnormal part discriminating part 74 can be automatically performed. Therefore, the content of the medical image of the subject to be displayed on the monitor 18 can be automatically changed according to the progress of the examination of the subject. Also, these processes are repeatedly performed for each of the plurality of attention areas.

The diagnostic report creation stage is a stage in which the doctor creates a diagnostic report for the subject in the external system 82 based on the examination information and the like fed back from the examination information feedback section 78 .

The examination information feedback unit 78 feeds back examination information including the diagnosis result of the subject, the examination time, and the equipment used to the external system 82 at the stage of creating the diagnostic report.
In the external system 82, the doctor can easily create a diagnostic report of the subject as shown in FIG.
The examination information feedback unit 78 may feed back to the external system 82 as examination information including medical images of the subject. In this case, the doctor can create a diagnostic report based on the subject's medical image and examination information, and can attach, for example, the subject's medical image and the like to the subject's diagnostic report.

As described above, in the endoscopic diagnostic apparatus 10, medical images and diagnostic information of the subject by various modalities 80 can be aggregated and observed on the monitor 18 of the endoscopic diagnostic apparatus 10. FIG. In addition, before observing the endoscopic image of the subject, the diagnostic information of the subject can be confirmed on the monitor 18 in advance, and the endoscopic diagnostic apparatus 10 The content displayed on the monitor 18 can be changed during inspection.

By the way, in the stage of inserting the endoscope, the stage of picking up the region of interest, and the stage of discrimination and treatment of the region of interest, for example, a normal image is displayed in the first segmented region as the current endoscopic image of the subject. In order to operate the endoscope 12, the doctor tends to concentrate on observing the normal image and hardly see the medical image and diagnostic information of the subject displayed in the second divided area. . Therefore, it would be desirable to automatically notify the physician of specific situations that should be notified to the physician to bring them to the attention of the physician.

For example, as shown in FIG. 12, the display control unit 76 causes the biological information of the subject including the heartbeat value and the blood pressure value to be displayed in one of the other divided regions, for example, the second divided region in the upper right part, and During the examination, when at least one of the heart rate value and the blood pressure value exceeds the upper limit value or falls below the lower limit value, the arrow pointing to the biological information of the subject is displayed in a predetermined color, such as red, with two or more Of the divided areas, the biological information of the subject may be displayed in a divided area other than one of the other divided areas, for example, in the first divided area as shown in FIG. 12, or in the lower right part may be displayed in the second divided area of . In this case, the display control unit 76 may blink the arrow at regular time intervals.
This allows the doctor to immediately recognize that at least one of the heart rate value and blood pressure value of the subject is abnormal.

In addition, as shown in FIG. 13, the display control unit 76, when it is determined to be an abnormal part, or in response to an instruction from the user, displays the part that is estimated to be an abnormal part. An image representing the position is overlaid on the current endoscopic image of the subject and displayed, and after a certain period of time has elapsed or in response to an instruction from the user, the site that is estimated to be an abnormal site is displayed. The positional image overlay may be turned off to display the current endoscopic image of the subject.
As a result, the doctor can immediately recognize that there is a part that has been identified as being an abnormal part, and can easily confirm its position.

Alternatively, the display control unit 76 displays an image representing the position of the part that is estimated to be abnormal as shown in the second divided area in the upper right part of FIG. It may be displayed by enclosing it with a line. In this case, the display control unit 76 may blink the frame line at regular time intervals. In addition, the display control unit 76 displays an arrow pointing to the image representing the position of the part that is estimated to be an abnormal part in a predetermined color, such as red, and estimates that it is an abnormal part out of the two or more divided areas. Then, it may be displayed in a divided area other than one of the other divided areas in which an image representing the position of the identified part is displayed. In this case, the display control unit 76 may blink the arrow at regular time intervals.

As in each of the above examples, even when a large amount of information is collectively displayed in the endoscope diagnostic apparatus 10, the important information can be displayed so that the doctor can easily recognize it.

Although an example in which the information display system of the present invention is incorporated in the endoscopic diagnostic apparatus 10 has been described, the information display system of the present invention can be similarly incorporated in various modalities 80 other than the endoscopic diagnostic apparatus. Is possible. Alternatively, the information display system of the present invention may be provided separately from the modality 80 without incorporating it into the modality 80, and various modalities 80, external systems 82, etc. may be connected to the information display system of the present invention.

In the apparatus of the present invention, for example, the light source control unit 21, the receiving unit 53, the DSP 56, the noise removing unit 58, the image processing switching unit 60, the normal image processing unit 62, the special image processing unit 64 (first special image processing unit 64a, 2 special image processing unit 64b, third special image processing unit 64c), video signal generation unit 66, medical image acquisition unit 68, diagnostic information acquisition unit 70, attention area estimation unit 72, abnormal part discrimination unit 74, display control unit 76 The hardware configuration of the processing unit (Processing Unit) that executes various processes such as the test information feedback unit 78 may be dedicated hardware, or may be various processors or computers that execute programs. may

For various processors, the circuit configuration can be changed after manufacturing such as CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), etc. which is a general-purpose processor that executes software (program) and functions as various processing units. Programmable Logic Device (PLD), which is a processor, ASIC (Application Specific Integrated Circuit), etc. .

One processing unit may be composed of one of these various processors, or a combination of two or more processors of the same or different type, such as a combination of multiple FPGAs, or a combination of FPGAs and CPUs. and so on. Also, the plurality of processing units may be configured by one of various processors, or two or more of the plurality of processing units may be combined into one processor.

For example, as typified by computers such as servers and clients, there is a form in which one or more CPUs and software are combined to constitute one processor, and this processor functions as a plurality of processing units. In addition, as typified by System on Chip (SoC), etc., there is a form of using a processor that implements the function of the entire system including a plurality of processing units with a single IC (Integrated Circuit) chip.

Further, the hardware configuration of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

Also, the method of the present invention can be implemented, for example, by a program for causing a computer to execute each step. It is also possible to provide a computer-readable recording medium on which this program is recorded.

Although the present invention has been described in detail above, it is needless to say that the present invention is not limited to the above-described embodiments, and various improvements and modifications may be made without departing from the gist of the present invention.

REFERENCE SIGNS LIST 10 endoscope diagnostic device 12 endoscope 12a insertion section 12b operation section 12c bending section 12d tip section 12e angle knob 13a mode switching SW
14 light source device 16 processor device 18 monitor 19 console 20a V-LED
20b B-LEDs
20c G-LED
20d R-LED
21 light source control section 23 optical path coupling section 30a illumination optical system 30b imaging optical system 41 light guide 45 illumination lens 46 objective lens 48 imaging sensor 50 CDS/AGC circuit 52 A/D converter 53 receiving section 56 DSP
58 noise removal unit 60 image processing switching unit 62 normal image processing unit 64 special image processing unit 64a first special image processing unit 64b second special image processing unit 64c third special image processing unit 66 video signal generation unit 68 medical image acquisition unit 70 diagnostic information acquisition unit 72 attention area estimation unit 74 abnormal part discrimination unit 76 display control unit 78 examination information feedback unit 80 modality 82 external system

Claims (22)

a medical image acquisition unit that acquires medical images of a subject from one or more modalities;
a monitor;
a display control unit that displays the medical image of the subject in at least one of two or more divided areas obtained by dividing the display screen of the monitor;
The display control unit changes the content of the medical image of the subject to be displayed on the monitor according to the progress of the examination of the subject, and when a specific situation occurs, at least one of the biological information of the subject that is the specific situation and an image representing the position of the part of the subject that is the specific situation is displayed in one of the two or more divided areas; an information display system for displaying an arrow pointing to a specific situation in a divided area other than the divided area where the specific situation is displayed, among the two or more divided areas. further comprising a diagnostic information acquiring unit for acquiring diagnostic information of the subject from an external system,
The display control unit displays the diagnostic information of the subject on the monitor, or displays at least one of the medical image of the subject and the diagnostic information of the subject in at least one of the two or more divided areas. 2. The information display system according to claim 1, wherein the contents of the medical image of the subject and the diagnostic information of the subject displayed on the monitor are changed according to the progress of the examination of the subject. the medical image of the subject includes a current endoscopic image of the subject;
3. The information according to claim 2, wherein the examination of the subject includes a preparation stage, an endoscope insertion stage, a region of interest pick-up stage, a differentiation and treatment stage of the region of interest, and a diagnostic report creation stage. display system. 4. The information display system according to claim 3, wherein said display control unit causes said monitor to display at least one of patient information of said subject and past diagnostic reports of said subject in said preparation stage. The display control unit displays a current endoscopic image of the subject in one of the two or more divided areas, and displays a past endoscope image of the subject in a stage of inserting the endoscope. At least one of a scopic image, biological information of the subject, an insertion shape of the endoscope in the body cavity of the subject, and support information for moving the endoscope in the body cavity of the subject. 5. The information display system according to claim 3, wherein is displayed in at least one of the other divided areas of the two or more divided areas. The display control unit causes the biological information of the subject including the heartbeat value and the blood pressure value to be displayed in one of the other divided regions as the specific situation, and displays the heartbeat value during the examination of the subject. and when at least one of the blood pressure values exceeds the upper limit value or falls below the lower limit value, the arrow pointing to the biological information of the subject is moved to the biological information of the subject among the two or more divided regions. 6. The information display system according to claim 5, wherein information is displayed in a divided area other than one of said other divided areas where information is displayed. further comprising a region-of-interest estimating unit for estimating the presence or absence of the region of interest and the position of the region of interest from the current endoscopic image of the subject;
The display control unit causes the current endoscopic image of the subject to be displayed in one of the two or more divided areas in the step of picking up the region of interest, An image representing the position of the region estimated to be the region of interest in the endoscopic image, and the current endoscopic image of the subject generated by performing color enhancement processing on the current endoscopic image of the subject. displaying at least one color-enhanced image in which a color difference between a normal portion and an abnormal portion in an endoscopic image is emphasized in at least one of the other divided regions of the two or more divided regions; 7. The information display system according to any one of 3 to 6. Further, from a blue narrow-band light image in which the vascular structure of the mucosal surface layer in the current endoscopic image of the subject is captured by irradiating the subject with white light and blue narrow-band light, the an abnormal part discriminating unit that discriminates whether or not the part estimated to be the attention area is estimated to be an abnormal part, the position of the part estimated to be the abnormal part, and the probability that the part is estimated to be the abnormal part;
The display control unit causes the current endoscopic image of the subject to be displayed in one of the two or more divided regions in a stage of discrimination and treatment of the region of interest, so that the abnormal region is estimated to be the abnormal region. An image representing the position of the part identified as being the abnormal part, the probability that the part identified as being the abnormal part is estimated as the abnormal part, and the length of the part identified as being the abnormal part 8. The information display system according to any one of claims 3 to 7, wherein at least one of the length measurement results representing height is displayed in at least one of the other divided areas of the two or more divided areas. The display control unit switches the current endoscopic image of the subject to an image representing the oxygen saturation in the current endoscopic image of the subject in response to an instruction from the user, and divides the image into one. 9. The information display system according to claim 8, wherein the information is displayed in an area. The display control unit displays an image representing the position of the site that is estimated to be the abnormal part when it is determined to be the abnormal part or in response to an instruction from the user. 10. The information display system according to claim 8 or 9, wherein the current endoscopic image of the specimen is overlaid and displayed. After a certain period of time has elapsed or in response to an instruction from a user, the display control unit stops overlaying the image representing the position of the site that is estimated to be the abnormal site, and displays the image of the subject. 11. The information display system of claim 10, for displaying a current endoscopic image. 12. The information display system according to any one of claims 8 to 11, wherein the display control unit displays an image representing the position of the part that is estimated to be the abnormal part by surrounding it with a frame line. 13. The information display system according to claim 12, wherein said display control unit blinks said frame line at regular time intervals. The display control unit causes the arrow pointing to the image representing the position of the part identified as being the abnormal part to be identified as being the abnormal part out of the two or more divided regions. 14. The information display system according to any one of claims 8 to 13, wherein an image representing the position of the part that has been pulled is displayed in a divided area other than one of the other divided areas in which the image is displayed. 15. The information display system according to claim 6, wherein said display control unit blinks said arrow at regular time intervals. The display control unit performs image processing of the image signal by an endoscope that outputs an image signal obtained by capturing an image of the subject using illumination light in response to an instruction to start an examination from a user. When connected to at least one of a processor device that outputs the current endoscopic image and a light source device that outputs the illumination light, display at the insertion stage of the endoscope from the content to be displayed at the preparation stage 16. The information display system according to any one of claims 3 to 15, which automatically switches to the content to be displayed. 17. The method according to any one of claims 3 to 16, further comprising an examination information feedback unit that feeds back examination information including the diagnosis result of the subject, examination time, and used equipment to the external system at the stage of creating the diagnosis report. The information display system according to item 1. The display control unit divides the display screen of the monitor into a first divided area and one or more second divided areas having an area smaller than that of the first divided area, and selects one of the one or more modalities. displaying the medical image of the subject acquired from the first divided area, the medical image of the subject acquired from the one or more modalities, and the diagnosis of the subject acquired from the external system 18. The information display system according to any one of claims 2 to 17, wherein at least one piece of information is displayed in at least one of said one or more second divided areas. The display control unit divides the display screen of the monitor into the first divided area and two of the second divided areas, and divides the medical image of the subject acquired from the one modality into the first divided area. to display at least one of the medical image of the subject obtained from the one or more modalities and the diagnostic information of the subject obtained from the external system at least one of the two second divided areas 19. The information display system according to claim 18, wherein the information display system displays two. The aspect ratio of the display screen of the monitor is 16:9, the aspect ratio of the first divided area and the two second divided areas is 5:4,
The first divided area is arranged from the left part to the center part of the display screen of the monitor, and the two second divided areas are arranged in the upper right part and the lower right part of the display screen of the monitor, respectively. Item 20. The information display system according to Item 19. An information display method for an information display system that displays a medical image of a subject, comprising:
a step in which a medical image acquisition unit included in the information display system acquires a medical image of the subject from one or more modalities;
causing the display control unit included in the information display system to display the medical image of the subject in at least one of two or more divided areas obtained by dividing the display screen of the monitor included in the information display system ;
The display control unit changes the content of the medical image of the subject to be displayed on the monitor according to the progress of the examination of the subject, and when a specific situation occurs, the specific situation at least one of the biological information of the subject that is the specific situation and an image representing the position of the part of the subject that is the specific situation is displayed in one of the two or more divided areas; an information display method, wherein an arrow pointing to a specific situation is displayed in a divided area other than the divided area displaying the specific situation among the two or more divided areas. Furthermore, a diagnostic information acquisition unit provided in the information display system acquires diagnostic information of the subject from an external system,
The display control unit displays the diagnostic information of the subject on the monitor, or displays at least one of the medical image of the subject and the diagnostic information of the subject in at least one of the two or more divided areas. 22. The information display method according to claim 21, wherein the contents of the medical image of the subject and the diagnostic information of the subject displayed on the monitor are changed according to the progress of the examination of the subject.

Images