JP2021090782A - Information processing system, information processing method and program - Google Patents

Abstract

To recommend a more proper biological observation image to a user during work with a medical observation apparatus.SOLUTION: An information processor includes a recommendation unit that determines, based on a plurality of biological images generated by imaging a living body in a plurality of wavelength bands with a medical observation apparatus for observing the inside of the living body and on status information on a status during observation of the inside of the living body, at least one of the plurality of biological images or at least some of images generated from the plurality of biological images as a recommended image suitable for the situation.SELECTED DRAWING: Figure 4

Description

The present disclosure relates to information processing devices, information processing methods, programs and medical observation systems.

In recent years, various medical observation devices such as medical endoscopes and medical microscopes have come to be used in the medical field. Further, particularly in a medical endoscope, as disclosed in Patent Document 1 below, in addition to the observation image in the normal visible light band, the observation site is irradiated with light in a specific wavelength band. Attempts have been made to generate a spectroscopic image in a specific wavelength band by imaging and display the spectroscopic image on a display screen.

Japanese Unexamined Patent Publication No. 2012-81087

However, in a medical endoscope as disclosed in Patent Document 1, it is more appropriate because the selection of the image to be displayed on the display screen is left to the judgment of the operator of the medical endoscope. If the operator does not notice the existence of the image even though it exists, it may not be possible to improve the efficiency of the work under endoscopic observation.

In addition, the above-mentioned circumstances can occur not only when the work is performed under the observation with a medical endoscope but also when the work is performed under the observation with a medical microscope.

Therefore, in this disclosure, in view of the above circumstances, an information processing device, an information processing method program, and a medical use that can recommend a more appropriate biological observation image to the user when working with a medical observation device. We propose an observation system.

According to the present disclosure, a plurality of biological images generated by a medical observation device for observing the inside of a living body by imaging the living body in a plurality of wavelength bands, and a situation relating to the situation when observing the inside of the living body. A recommendation unit for determining at least one of the plurality of biological images or at least one of the images generated from the plurality of biological images as a recommended image according to the situation based on the information is provided. An information processing device is provided.

Further, according to the present disclosure, a plurality of biological images generated by imaging the living body in a plurality of wavelength bands by a medical observation device for observing the inside of the living body, and a situation when observing the inside of the living body. Including determining at least one of the plurality of biological images or at least one of the images generated from the plurality of biological images as a recommended image according to the situation based on the situation information regarding the above. , Information processing methods are provided.

Further, according to the present disclosure, a computer capable of intercommunication with a medical observation device that generates a plurality of biological images generated by observing the inside of a living body and imaging the living body in a plurality of wavelength bands. At least one of the plurality of biological images, or at least an image generated from the plurality of biological images, based on the plurality of biological images and situation information regarding the situation when observing the inside of the living body. A program for realizing a recommended function of determining any of them as a recommended image according to the situation is provided.

Further, according to the present disclosure, a medical observation device that observes the inside of a living body and images the living body in a plurality of wavelength bands to generate a plurality of biological images, and the medical observation device generated by the medical observation device. At least one of the plurality of biological images, or at least an image generated from the plurality of biological images, based on the plurality of biological images and situation information regarding the situation when observing the inside of the biological image. A medical observation system is provided, comprising an information processing apparatus having a recommended unit for determining any of them as a recommended image according to the situation.

According to the present disclosure, a plurality of biological images generated by a medical observation device for observing the inside of a living body by photographing the living body in a plurality of wavelength bands, and situation information regarding a situation when observing the inside of the living body. , At least one of the plurality of biological images, or at least one of the images generated from the plurality of biological images is determined as a recommended image according to the situation.

As described above, according to the present disclosure, it is possible to recommend a more appropriate biological observation image to the user when working with a medical observation device.

It should be noted that the above effects are not necessarily limited, and any of the effects shown in the present specification, or other effects that can be grasped from the present specification, together with the above effects or in place of the above effects. The effect of may be achieved.

It is explanatory drawing which showed typically the whole structure of the medical observation system which concerns on embodiment of this disclosure. It is explanatory drawing which shows typically an example of the image pickup unit which the medical observation apparatus which concerns on this embodiment has. It is explanatory drawing which shows typically an example of the image pickup unit which the medical observation apparatus which concerns on this embodiment has. It is explanatory drawing which shows typically an example of the image pickup unit which the medical observation apparatus which concerns on this embodiment has. It is a block diagram which shows typically an example of the structure of the information processing apparatus which concerns on this embodiment. It is a block diagram which shows typically an example of the structure of the image recommendation processing part which the information processing apparatus which concerns on this embodiment has. It is explanatory drawing for demonstrating the situation identification database which the information processing apparatus which concerns on this embodiment has. It is explanatory drawing which showed an example of the absorption spectrum of the main substance in a living tissue. It is explanatory drawing for demonstrating the recommended image database which the information processing apparatus which concerns on this embodiment has. It is explanatory drawing which shows typically an example of the display screen in the information processing apparatus which concerns on this embodiment. It is explanatory drawing which shows typically an example of the display screen in the information processing apparatus which concerns on this embodiment. It is a flow chart which showed an example of the flow of the information processing method which concerns on the same embodiment. It is a block diagram which showed an example of the hardware composition of the information processing apparatus which concerns on this embodiment.

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, so that duplicate description will be omitted.

The explanations will be given in the following order.
1. 1. Embodiment 1.1. Overall configuration of medical observation system 1.2. Configuration of medical observation equipment 1.3. Configuration of information processing device 1.4. Information processing method 1.5. About hardware configuration

(Embodiment)
<Overall configuration of medical observation system>
First, with reference to FIG. 1, the overall configuration of the medical observation system 1 according to the embodiment of the present disclosure will be briefly described. FIG. 1 is an explanatory diagram schematically showing the overall configuration of the medical observation system 1 according to the present embodiment.

As shown in FIG. 1, the medical observation system 1 according to the present embodiment includes a medical observation device 10 for observing the inside of a living body and an information processing device 20.

The medical observation device 10 is a medical device for observing the inside of the living body S by inserting a part thereof into the inside of the living body S or arranging it in the vicinity of the internal organs of the living body S. .. Examples of such a medical observation device 10 include various endoscope devices such as a rigid endoscope, a flexible endoscope, and an arthroscope, and various microscope devices.

The medical observation device 10 according to the present embodiment irradiates a predetermined observation light to a portion of the living body S to be observed, and images the portion of the living body S irradiated with the observation light to obtain visible light. A plurality of biological images including a captured image of a biological body in a band (a band corresponding to a wavelength in the range of about 400 nm to 800 nm) are generated.

In addition to the above-mentioned image of the living body in the visible light band, such a biological image includes, for example, an image of the living body in the near-infrared band (a band corresponding to a wavelength of about 800 nm to a wavelength of about 2.5 μm). , An image of a living body in various wavelength bands such as an image of a living body in an ultraviolet band may be included. In addition, the biological image includes, for example, an image obtained when the living body is observed using ultrasonic waves, an image obtained when the living body is observed by the optical coherence tomography (OCT) method, and the like. Images may be included.

A plurality of biological images including images of the living body in the visible light band generated by the medical observation device 10 are displayed at any time on a display device such as a display connected to the medical observation device 10 by wire or wirelessly. In addition, it is output to the information processing device 20 according to the present embodiment, which will be described in detail below.

The detailed configuration of the medical observation device 10 will be described again below.

The information processing device 20 according to the present embodiment analyzes the captured image generated by the medical observation device 10, determines a more appropriate image as a recommended image during the work, and then determines the recommended image as the recommended image. Recommended for 10 users (eg, doctors performing surgery, etc.). As a result, in the information processing device 20 according to the present embodiment, the user grasps the situation when observing the inside of the living body, and the observation image according to the situation is used as a recommended image for the user of the medical observation device 10. Can be recommended. As a result, the user of the medical observation device 10 can be aware that there is something that can be a more appropriate observation image in continuing the work being carried out, and under the medical observation device. It is possible to improve the efficiency of the work.

The detailed configuration of the information processing device 20 will be described again below.

Note that FIG. 1 describes a case where the medical observation device 10 and the information processing device 20 according to the present embodiment exist as different devices, but the function of the information processing device 20 described in detail below is medical treatment. Needless to say, it may be realized as one function of the observation device 10.

<About the configuration of the medical observation device 10>
Subsequently, the detailed configuration of the medical observation device 10 according to the present embodiment will be briefly described with reference to FIGS. 1 to 2C. 2A to 2C are explanatory views schematically showing an example of an imaging unit included in the medical observation device according to the present embodiment.

As mentioned earlier, the medical observation device 10 according to the present embodiment is partially inserted into the living body S or arranged in the vicinity of the internal organs of the living body S. It is a medical device that observes the inside of the body. As schematically shown in FIG. 1, the medical observation device 10 mainly includes a control unit 101, an observation unit 103, a light source unit 105, an imaging unit 107, and a display unit 109. There is.

The control unit 101 is realized by, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), or the like. The control unit 101 is a unit that controls the overall functions of the medical observation device 10, and constitutes the medical observation device 10, the observation unit 103, the light source unit 105, the image pickup unit 107, and the display unit 109. Control the operating status of all at once. The control unit 101 is, for example, a unit corresponding to a camera control unit (CCU) of a medical endoscope device or a medical microscope device. Further, the control unit 101 has various information such as information on setting conditions set in each unit constituting the medical observation device 10, information on the operating state of each unit constituting the medical observation device 10, and the like. Can be output to the information processing apparatus 20 according to the present embodiment at an arbitrary timing.

Under the control of the control unit 101, the observation unit 103 is a unit in which a part of the observation unit 103 is inserted into the living body S or arranged in the vicinity of the internal organs of the living body S. The observation unit 103 forms an image of an observation image obtained by observing the inside of the living body S using the observation light emitted from the light source unit 105 on the image pickup unit 107 in the subsequent stage. The observation unit 103 corresponds to, for example, an endoscope unit in a medical endoscope device or a microscope unit in a medical microscope device. The observation unit 103 according to the present embodiment is not particularly limited, and a known endoscope unit, microscope unit, or the like can be applied.

The light source unit 105 is a unit that emits observation light for observing the inside of a living body under the control of the control unit 101, and is optically connected to the observation unit 103. The light source unit 105 has at least a visible light source (for example, a white light source) for obtaining an image of a living body in the visible light band. In addition to the visible light source, the light source unit 105 also includes a near-infrared light source for obtaining an image of a living body in the near-infrared band, an ultraviolet light source, and various laser light sources that emit specific visible light. It may have various known light sources such as. Under the control of the control unit 101, the light source unit 105 can switch the type of observation light emitted from these various light sources at an arbitrary timing. Further, the light source unit 105 may further have a light source for realizing a specific function, such as an OCT light source or a distance measuring light source, in addition to the above-mentioned light source.

Under the control of the control unit 101, the imaging unit 107 captures an observation image of the inside of the living body S by the observation light from the light source unit 105, and a plurality of biological images including the captured image of the living body in the visible light band. It is a unit that generates image data. The imaging unit 107 is optically connected to the observation unit 103. The imaging unit 107 according to the present embodiment captures images of various types of observation light (for example, light in various wavelength bands such as visible light, near infrared rays, ultraviolet rays, etc.) while distinguishing them from each other to capture a plurality of biological images. Generate image data.

In a general medical observation system, by using an image sensor that is sensitive to wavelengths in the visible light band, an image close to the situation directly observed by the human eye is captured, and the image is appropriately developed. , The observation result is displayed on the display device as a normal observation image. Further, in a general medical observation system capable of special light observation, in addition to the normal observation mode for displaying an image captured in the visible light band, an imaging element sensitive to wavelengths in the near infrared band is used. A fluorescence observation mode for observing fluorescence generated in a living body, and a narrow wavelength image (NBI) observation mode that makes it easy to identify blood vessels having different depths from the skin surface by combining a plurality of specific narrow wavelengths. , Various functions are often realized.

However, the special light observation as described above is realized by combining an illumination device that selectively irradiates a specific wavelength and an optical filter that transmits or reflects a specific wavelength, so that it is a visible normal observation mode. It becomes difficult to observe at the same time.

Therefore, in the medical observation device 10 according to the present embodiment, by providing the imaging unit 107 as shown in FIGS. 2A to 2C, an image of a living body in the visible light band and an image of another living body can be obtained. , Can be acquired at the same time.

For example, as shown in FIG. 2A, the image sensor 107 includes various beam splitters such as a normal beam splitter or a polarizing beam splitter, and two types of image sensors A and B. It may have. In the case of the image pickup unit 107 as shown in FIG. 2A, the observation image from the observation unit 103 is split into two optical paths by a beam splitter, and images are formed on the image pickup element A and the image pickup element B, respectively. Here, for example, by using a visible light image sensor having sensitivity to light in the visible light band as the image sensor A and using a near infrared image sensor having sensitivity to light in the near infrared band as image sensor B. , An image of a living body in the visible light band and an image of a living body in the near infrared band can be generated. Further, by using another known image pickup element as the image pickup element B, it is possible to generate an image pickup image of various living organisms in addition to the image pickup image of the living body in the visible light band.

Although FIG. 2A shows a case where one beam splitter is used, the observation image can be branched into more optical paths by using two or more beam splitters in combination. It is possible to simultaneously capture more types of images of living organisms.

Further, as shown in FIG. 2B, for example, the image pickup unit 107 includes a spectroscopic prism in which a plurality of (three in FIG. 2B) optical prisms are joined to each other, and a plurality of types of image pickup elements (in FIG. It may have three types of image pickup elements C). In the case of the imaging unit 107 as shown in FIG. 2B, the observation image from the observation unit 103 can be obtained by making the junction surface of the adjacent optical prisms function as any of a beam splitter, a polarizing beam splitter, and a wavelength selection filter. It is possible to branch into a plurality of optical paths (three optical paths in the case of FIG. 2B). The observation image from the observation unit 103 is branched into a plurality of optical paths by the spectroscopic prism, and an image is formed on each image sensor provided at the end of each optical path. Here, for example, by using a visible light image sensor having sensitivity to light in the visible light band as the image sensor A and using a near infrared image sensor having sensitivity to light in the near infrared band as image sensor B. , An image of a living body in the visible light band and an image of a living body in the near infrared band can be generated. Further, by disposing another known image sensor at the position of the image sensor C, if necessary, in addition to the image of the living body in the visible light band and the image of the living body in the near infrared band. In addition, it is possible to generate captured images of various living bodies.

The spectroscopic prism as shown in FIG. 2B can branch the observation image into more optical paths by increasing the number of optical prisms used, and can simultaneously capture images of more types of living organisms. It becomes possible to take an image.

Further, the image pickup unit 107 may have a so-called multispectral sensor as shown in FIG. 2C. As schematically shown in FIG. 2C, the multispectral sensor is a two-dimensional image sensor including at least pixels that distinguish light intensities in a plurality of wavelength bands for each wavelength band and detect them at one time. By using such a multi-spectral sensor, it is possible to acquire multi-spectral data which is the intensity of light in a plurality of narrow wavelength bands for each pixel of an image.

It is preferable that all the pixels constituting the multispectral sensor as schematically shown in FIG. 2C are pixels that detect the intensity of light in a plurality of wavelength bands at one time by distinguishing each wavelength band. However, all the pixels do not have to be the pixels that detect the intensity of light in a plurality of wavelength bands as described above by distinguishing each wavelength band at one time. By performing various known interpolation processes using the output from the pixels that distinguish the light intensities in a plurality of wavelength bands for each wavelength band and detect them at once as described above, as described above. This is because it is possible to calculate the intensity of light in a plurality of wavelength bands for each wavelength band even at the positions of pixels that do not have a function.

By using the multispectral sensor as shown in FIG. 2C, the imaging unit 107 can be made smaller than the imaging unit 107 as shown in FIGS. 2A and 2B, and the medical observation device 10 can be used. It is possible to further reduce the size of the.

A beam splitter or spectroscopic prism as shown in FIG. 2A or 2B and a multispectral sensor as shown in FIG. 2C are combined with each other to generate more kinds of biological images. Needless to say, 107 may be configured.

For example, various biological images generated by the imaging unit 107 as shown in FIGS. 2A to 2C are appropriate when the control unit 101 appropriately controls according to the user operation of the user of the medical observation device 10. Image is selected and displayed on the display screen provided in the display unit 109 in the subsequent stage. Usually, the user of the medical observation device 10 often pays attention to the captured image of the living body in the visible light band as the observation image of the inside of the living body, so that the display screen of the display unit 109 in the subsequent stage is mainly used. Images of living organisms in the visible light band are often displayed.

Further, the imaging unit 107 outputs a plurality of biological images generated as described above to the information processing device 20 according to the present embodiment at any time under the control of the control unit 101.

In the image pickup unit 107 according to the present embodiment, there is only one image pickup element in the image pickup unit 107, for example, there is only one image pickup element having sensitivity in the visible light band to the near infrared band. It is possible that it is not done. Even in such a case, the light source unit 105 and the image pickup unit 107 cooperate with each other under the control of the control unit 101 to perform imaging while switching the wavelength band of the light imaged on the image pickup element. It becomes possible to generate a biological image in a wavelength band.

The display unit 109 is a unit that displays at least one biological image among the plurality of biological images generated by the imaging unit 107 to the user of the medical observation device 10 under the control of the control unit 101. The number of display screens provided in the display unit 109 is not particularly limited, and the display unit 109 may have only one display screen or may have a plurality of display screens. May be good. The display unit 109 is not particularly limited, and a known display unit 109 can be used.

Further, on the display screen of the display unit 109 according to the present embodiment, information on recommended images recommended by the information processing apparatus 20 described later is appropriately displayed.

As described above, an example of the configuration of the medical observation device 10 according to the present embodiment has been briefly described with reference to FIGS. 1 to 2C. The medical observation device 10 according to the present embodiment has various other units such as an ultrasonic oscillator and an ultrasonic detection unit in addition to the various units shown in FIG. You may.

<About the configuration of the information processing device 20>
Next, the information processing apparatus 20 according to the present embodiment will be described in detail with reference to FIGS. 3 to 9.
FIG. 3 is a block diagram schematically showing an example of the configuration of the information processing apparatus according to the present embodiment. FIG. 4 is a block diagram schematically showing an example of the configuration of the image recommendation processing unit included in the information processing apparatus according to the present embodiment. FIG. 5 is an explanatory diagram for explaining a situation-specific database included in the information processing apparatus according to the present embodiment, and FIG. 6 is an explanatory diagram showing an example of an absorption spectrum of a main substance in a living tissue. .. FIG. 7 is an explanatory diagram for explaining a recommended image database included in the information processing apparatus according to the present embodiment. 8 and 9 are explanatory views schematically showing an example of a display screen in the information processing apparatus according to the present embodiment.

[Overall configuration of information processing device 20]
As mentioned earlier, the information processing apparatus 20 according to the present embodiment analyzes the captured image generated by the medical observation device 10 and determines a more appropriate image as a recommended image in the work, and then makes such a recommendation. This is a device that recommends images to users of the medical observation device 10. As schematically shown in FIG. 3, the information processing device 20 mainly includes an information acquisition unit 201, an image recommendation processing unit 203, an output processing unit 205, a display control unit 207, and a storage unit 209. Have in.

The information acquisition unit 201 is realized by, for example, a CPU, a ROM, a RAM, a communication device, or the like. The information acquisition unit 201 acquires various types of information from the medical observation device 10. For example, the information acquisition unit 201 constantly acquires image data of a plurality of biological images including images of the living body in the visible light band generated by the medical observation device 10. Further, the information acquisition unit 201 can acquire various information about the medical observation device 10 from the medical observation device 10 (more specifically, the control unit 101 of the medical observation device 10 or the like). Further, the information acquisition unit 201 acquires information on the recommended image usage status from the medical observation device 10, such as the user's selection result for the image recommended by the information processing apparatus 20 according to the present embodiment. Is also possible. Further, the information processing device 20 according to the present embodiment can also acquire information on various user operations performed by the user of the medical observation device 10 or the operator of the information processing device 20.

Here, various information regarding the medical observation device 10 includes, for example, setting information regarding various settings of the medical observation device 10, characteristic information regarding the characteristics of the medical observation device 10, medical observation device 10, and medical observation device. Examples include operating information regarding the operating status of various peripheral devices linked with 10. The setting information related to various settings of the medical observation device 10 includes, for example, the setting information of the light source unit 105 in the medical observation device 10 (for example, information on the wavelength of the observation light) and what kind of peripheral device is the medical observation device. Information such as whether or not it is linked with 10 can be mentioned. Further, as the characteristic information regarding the characteristics of the medical observation device 10, for example, setting information regarding the sensor spectral characteristics of the image pickup unit 107 in the medical observation device 10 can be mentioned. Further, as the operation information regarding the operating state of the medical observation device 10 and various peripheral devices linked with the medical observation device 10, for example, what kind of light source is operating as the light source unit 105, an energy device such as an electric knife, etc. The operating status of the smoke exhaust device for discharging mist and smoke existing in the body cavity to the outside of the body can be mentioned.

When the information acquisition unit 201 acquires image data related to a plurality of biological images from the medical observation device 10, the information acquisition unit 201 outputs the acquired image data to the image recommendation processing unit 203, which will be described later. Further, the information acquisition unit 201 may record the acquired image data in the storage unit 209 described later as history information after associating the acquired image data with time information related to the date and time when the image data was acquired. Good. Further, the information acquisition unit 201 may record the usage history of the recommended image in the storage unit 209 described later based on the information regarding the usage status of the recommended image acquired from the medical observation device 10.

Further, when the information acquisition unit 201 acquires various information related to the medical observation device 10 as described above from the medical observation device 10, the information acquisition unit 201 outputs the acquired various information to the image recommendation processing unit 203 described later. In addition, various information about the medical observation device 10 may be notified from the control unit 101 of the medical observation device 10 each time the image data of a plurality of biological images is output to the information processing device 20. Further, the control unit 101 of the medical observation device 10 does not notify various information about the medical observation device 10 every time the image data is output, but the timing when the content of any of the information is changed. You may notify with.

The image recommendation processing unit 203 is realized by, for example, a CPU, a ROM, a RAM, or the like. The image recommendation processing unit 203 uses the image data and various information acquired by the information acquisition unit 201 to determine a recommended image according to the situation at the time when the inside of the living body is observed, and the recommended image is obtained. Is a processing unit that performs image recommendation processing recommended for the user of the medical observation device 10. The image recommendation processing unit 203 is generated from at least one of the plurality of biological images or a plurality of biological images based on the plurality of biological images and the situation information regarding the situation when observing the inside of the living body. It has at least a recommended part for determining at least one of the images as a recommended image according to the above situation.

The detailed configuration of the image recommendation processing unit 203 will be described again below.

When the image recommendation processing unit 203 determines a recommended image according to the situation at the time when the inside of the living body is observed by using a plurality of biological images, the information regarding the determined recommended image is output to the output processing unit 205 in the subsequent stage. Output to. Further, the image recommendation processing unit 203 associates the processing result related to the determined recommended image with the time information related to the date and time when the processing result was obtained, and stores the processing result as history information in the subsequent stage. It may be stored in 209.

The output processing unit 205 is realized by, for example, a CPU, a ROM, a RAM, an output device, a communication device, or the like. The output processing unit 205 outputs the processing result regarding the determined recommended image output from the image recommendation processing unit 203 to the medical observation device 10. The control unit 101 of the medical observation device 10 that has acquired the processing result displays the acquired processing result on the display unit 109, so that the user of the medical observation device 10 is recommended to observe the living body according to the situation. It becomes possible to do. Further, the output processing unit 205 may directly output the processing result to the display unit 109 of the medical observation device 10 in cooperation with the display control unit 207 in the subsequent stage.

The output processing unit 205 can output the processing result related to the recommended image output from the image recommendation processing unit 203 as a printed matter or output as data to an external information processing device, a server, or the like. is there.

The display control unit 207 is realized by, for example, a CPU, a ROM, a RAM, an output device, a communication device, or the like. The display control unit 207 outputs various results output from the output processing unit 205 to an output device such as a display included in the information processing device 20 or an output device provided outside the arithmetic processing unit 200 (for example, medical observation). Display control is performed when displaying on the display unit 109) of the device 10. As a result, the user of the medical observation device 10 can grasp various results on the spot.

The storage unit 209 is an example of a storage device included in the information processing device 20, and is realized by a RAM, a storage device, or the like included in the information processing device 20. Various databases used by the information processing apparatus 20 according to the present embodiment when performing the image recommendation processing described in detail below are recorded in the storage unit 209. In addition, various historical information related to the image recommendation process described in detail below may be recorded in the storage unit 209. Further, in the storage unit 209, various parameters that need to be saved when the information processing apparatus 20 according to the present embodiment performs some processing, the progress of the processing, etc., or various databases, programs, etc. are stored in the storage unit 209. Recorded as appropriate. In this storage unit 209, the information acquisition unit 201, the image recommendation processing unit 203, the output processing unit 205, the display control unit 207, and the like can freely perform read / write processing.

[About the configuration of the image recommended processing unit 203]
Subsequently, the configuration of the image recommendation processing unit 203 included in the information processing apparatus 20 according to the present embodiment will be described in detail with reference to FIGS. 4 to 7.
As mentioned earlier, the image recommendation processing unit 203 according to the present embodiment uses the image data and various information acquired by the information acquisition unit 201 according to the situation at the time when the inside of the living body is observed. In addition to determining the recommended image, the image recommendation process for recommending the recommended image to the user of the medical observation device 10 is performed. As schematically shown in FIG. 4, the image recommendation processing unit 203 includes a situation specifying unit 211, a recommendation unit 213, and a recommended image generation unit 215.

The situation specifying unit 211 is realized by, for example, a CPU, a ROM, a RAM, or the like. The situation specifying unit 211 is a processing unit that identifies the situation when the inside of the living body is observed based on a plurality of biological images output from the medical observation device 10. More specifically, the situation specifying unit 211 acquires setting information regarding the setting of the medical observation device 10, characteristic information regarding the characteristics of the medical observation device 10, medical observation device 10, and medical treatment acquired by the information acquisition unit 201. The situation when the inside of the living body is observed based on at least one of the operating information, the biological image, or the secondary information generated based on the biological image regarding the operating state of the peripheral device linked with the observation device 10. Identify.

The secondary information generated based on the above biological image is not particularly limited, and may be information that is secondarily calculated by performing various image processing or information processing on the biological image. For example, any one can be used. Such secondary information includes, for example, information on the spectral reflectance or absorption coefficient of organs and biological substances existing inside the living body, which can be calculated by paying attention to an image of the living body in a certain wavelength band. , Information on the oxygen saturation of blood, information on the recognition result of performing image recognition processing on a biological image using image information on organs existing inside the living body and medical equipment used for medical practice, etc. Can be mentioned.

More specifically, the situation specifying unit 211 includes setting information regarding the setting of the medical observation device 10, characteristic information regarding the characteristics of the medical observation device 10, peripheral devices linked with the medical observation device 10 and the medical observation device 10. Refer to the situation-specific database in which at least one of the operation information, the biological image, or the secondary information generated based on the biological image and the situation when the biological image is generated are associated with each other in advance. Therefore, it is preferable to identify the situation when the inside of the living body is observed.

An example of such a situation-specific database is schematically shown in FIG.
As shown schematically in FIG. 5, the situation-specific database stored in the storage unit 209 or the like according to the present embodiment contains situation-association data in which various information and situations as described above are previously associated with each other. It is stored. The data format of such situation-related data is not particularly limited, but for example, the above-mentioned various information and the situation are recorded in a data format such as a look-up table. Can be done. By referring to the situation association data using the acquired various information as described above, the situation specifying unit 211 can specify the situation corresponding to the acquired various information.

For example, when there is information that an energy device such as an electric knife is operating, or information that a smoke exhaust device is operating, etc., with reference to various information as described above. The situation specifying unit 211 pays attention to the above information and refers to the situation association data, and can identify that the visibility of the visual field is deteriorated due to smoke or mist generated by the energy device. it can.

In addition, the situation-specific database may store information on the extinction coefficient or spectral reflectance of organs and biological substances existing inside the living body. For example, as shown in FIG. 6, the biological substance existing inside the living body has an extinction coefficient peculiar to each biological substance. Therefore, by paying attention to the captured image of the living body in a specific wavelength band included in the plurality of biological images, it is possible to detect the biological substance existing in the obtained biological image. In addition, the organs and biological substances existing inside the living body have their own spectral reflectances as well as the extinction coefficient. Here, by paying attention to the captured image of the living body in a specific wavelength band included in the plurality of biological images, the light source characteristic of the light source unit 105 of the medical observation device 10, the sensor characteristic of the imaging unit 107, and the like. The spectral reflectance of the obtained biological image can be calculated. Therefore, the situation specifying unit 211 can specify the situation when the inside of the living body is observed by comparing the obtained information with the information on the spectral reflectance and the extinction coefficient recorded in advance. it can.

For example, paying attention to the above information (particularly, the biological image itself or the secondary information obtained from the biological image), the spectral reflectance or the extinction coefficient corresponding to blood is specified from the value of the spectral reflectance or the extinction coefficient. If so, the situation identifying unit 211 can identify that blood vessels are present and that bleeding may have occurred. Further, when the above information (particularly, the biological image itself, the secondary information obtained from the biological image, etc.) is focused on and the data corresponding to water is detected, the situation specifying unit 211 uses the energy device. It is possible to estimate that mist is generated accordingly.

In addition, the situation-specific database may store image information about organs existing inside the living body, medical instruments used for medical practice, and the like. By recognizing a biological image obtained from the medical observation device 10 using these image information, various organs existing in the biological image, a blood flow blocking medical device, a blood vessel anastomosis medical device, and the like are used. Medical equipment, etc. can be detected. As a result, the situation specifying unit 211 can estimate the surgical site, the surgical situation, and the like. The image recognition process performed by the situation specifying unit 211 is not particularly limited, and various known image recognition processes such as pattern recognition and recognition by machine learning can be appropriately used.

Further, in the situation-specific database, in addition to the various data as described above, various situation estimation data for estimating the situation may be appropriately stored.

As described above, the situation specifying unit 211 identifies the situation when the inside of the living body is observed while appropriately using the situation specifying database, etc., and then provides the situation information related to the specified situation to the recommendation unit 213 in the latter stage. Output to.

The situation-specific database as shown in FIG. 5 can be appropriately updated via various networks and external memory devices. Further, the situation specifying unit 211 may automatically update the situation specifying database by a known machine learning process or the like.

The recommendation unit 213 is realized by, for example, a CPU, a ROM, a RAM, or the like. The recommendation unit 213 is a plurality of biological images generated by the medical observation device 10 and a plurality of biological images output from the situation identification unit 211 regarding the situation when the inside of the living body is being observed. At least one of the biological images of the above, or at least one of the images generated from the plurality of biological images is determined as a recommended image according to the situation. In addition, the recommendation unit 213 recommends the determined recommended image to the user of the medical observation device 10.

More specifically, the recommendation unit 213 refers to a recommended image database in which the situation when the inside of the living body is observed and the type of recommended image that can be presented in each situation are associated with each other in advance. It is preferable to recommend the recommended image according to the situation to the user.

An example of such a recommended image database is schematically shown in FIG.
In the recommended image database stored in the storage unit 209 or the like according to the present embodiment, as schematically shown in FIG. 7, the situation when the inside of the living body is observed and the situation when each situation is presented are presented. Possible recommended image types and are recorded in association with each other. When the situation information output from the situation specifying unit 211 describes that a certain situation (situation A) has occurred, the recommendation unit 213 refers to the item related to the situation A in the recommended image database. As a recommended image, it can be determined that, for example, a near-infrared image is recommended.

Further, in the recommended image database, when a plurality of recommended images are associated with a certain situation, the recommendation unit 213 does not recommend one optimum image from the plurality of recommended images. It is preferable to recommend all the associated recommended images. Further, when a plurality of situations that may occur are listed in the situation information output from the situation specifying unit 211, the recommendation unit 213 is a recommended image for one of the plurality of situations. It is preferable to recommend the recommended images corresponding to all the situations described in the situation information, instead of recommending. This is because it is preferable to have the user of the medical observation device 10 (that is, a doctor or the like performing the operation) perform the determination as to which recommended image to use.

When the recommendation unit 213 determines the recommended image recommended for the user of the medical observation device 10, the recommendation unit 213 outputs the information regarding the determined recommended image to the output processing unit 205. The information regarding such a recommended image is not particularly limited, and for example, at least one of text information indicating the type of the recommended image and a thumbnail image of the recommended image can be mentioned. The output processing unit 205 causes the user of the medical observation device 10 or the operator of the information processing device 20 to output the information regarding the recommended image. As a result, the user of the medical observation device 10 or the operator of the information processing device 20 can determine on the spot whether or not to use the presented recommended image. Further, the output processing unit 205 presents information on the recommended image to the user of the medical observation device 10 or the operator of the information processing device 20 by using means other than characters and images such as voice and vibration. You may.

Further, when the recommendation unit 213 determines the type of the recommended image recommended for the user of the medical observation device 10, it requests the recommended image generation unit 215 in the subsequent stage to generate the determined recommended image, if necessary.

When the user of the medical observation device 10 chooses to refer to the recommended image, the recommendation unit 213 appropriately transmits the selected recommended image to the user of the medical observation device 10 via the output processing unit 205. Have them present. In addition, the recommendation unit 213 appropriately records the history of selection results by the user or the like of the medical observation device 10 in the storage unit 209 or the like, and displays the history information together with the next recommendation for medical use. The selection of the user or the like of the observation device 10 may be assisted.

The recommended image database as shown in FIG. 7 can be appropriately updated via various networks and external memory devices. Further, the recommendation unit 213 may automatically update the recommended image database by a known machine learning process or the like.

The recommended image generation unit 215 is realized by, for example, a CPU, a ROM, a RAM, or the like. The recommended image generation unit 215 captures at least one of at least a part of a plurality of biological images or an image that visualizes secondary information generated by analyzing at least one of the plurality of biological images. It is used to generate a recommended image recommended by the recommendation unit 213.

For example, when the recommendation unit 213 recommends an observation image in a specific wavelength band (for example, the near infrared band) as a recommended image, the recommended image generation unit 215 uses the corresponding wavelength band among the plurality of biological images. It is possible to generate an observation image of a corresponding wavelength band, a thumbnail image thereof, and the like as a recommended image by using the captured image of the living body of the above. Further, when the oxygen saturation visualization information that visualizes the distribution of oxygen saturation that can be specified as the secondary information of the biological image as shown in the database of FIG. 7 becomes the recommended image, such secondary information is used. Therefore, the corresponding recommended image, its thumbnail image, and the like can be generated. In addition to these images, various known images such as a blood vessel-enhanced image in which the presence of blood vessels are emphasized and an image showing the state of blood flow are used by using a known method such as the principle of NBI. The thumbnail image or the like can be appropriately generated.

Further, when a plurality of biological images include an image of the living body in the visible light band and an image of the living body in the near infrared band (for example, a fluorescent image in the near infrared band). Pay attention to. Here, in the fluorescence image in the near-infrared band, the portion corresponding to fluorescence is displayed in a predetermined color tone, and the portion not corresponding to fluorescence (the portion whose brightness value is zero or close to zero) is black. Often displayed. Even if such a fluorescence image is presented as a recommended image, there is a high possibility that the positional relationship of organs and the like will be difficult to understand. Therefore, the recommended image generation unit 215 first sets the color tone of the region (for example, the portion corresponding to fluorescence) in which the light in the near infrared band is detected in the captured image of the living body in the near infrared band to a predetermined color tone (). For example, it is changed to green, which is a color tone that does not exist in the living body, and the brightness value of the region where light in the near infrared band is not detected is changed to zero. Then, the recommended image generation unit 215 generates a superposed image in which the image of the living body in the near-infrared band after the change is superimposed on the image of the living body in the visible light band, and serves as the recommended image. May be good.

The above is an example of the function of the information processing apparatus 20 according to the present embodiment. Each of the above components may be configured by using general-purpose members or circuits, or may be configured by hardware specialized for the function of each component. Further, the CPU or the like may perform all the functions of each component. Therefore, it is possible to appropriately change the configuration to be used according to the technical level at each time when the present embodiment is implemented.

It is possible to create a computer program for realizing each function of the information processing apparatus according to the present embodiment as described above and implement it on a personal computer or the like. It is also possible to provide a computer-readable recording medium in which such a computer program is stored. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above computer program may be distributed via, for example, a network without using a recording medium.

[Example of recommended image presentation]
Next, the method of presenting the recommended image determined by the recommendation unit 213 will be briefly described with reference to FIGS. 8 and 9.
FIG. 8 is viewed by the user of the medical observation device 10 or the operator of the information processing device 20, such as the display unit 109 of the medical observation device 10 and the display device (not shown) of the information processing device 20. An example of the display screen is schematically shown.

Normally, when various medical actions such as surgery are performed, the control unit 101 of the medical observation device 10 or the display control unit 207 of the information processing device 20 schematically shows the figure in the upper left of FIG. As described above, it is considered that a captured image (visible light image) of a living body in the visible light band is displayed on the display screen as a normal display. Here, when the recommendation unit 213 proposes a recommended image according to the situation, the recommendation unit 213 recommends the recommended image via the output processing unit 205, for example, as schematically shown in the upper right figure of FIG. Text information indicating the type of the image may be superimposed on the visible light image and displayed. Further, the recommendation unit 213 displays not only the text information but also a text image indicating the type of the recommended image and a thumbnail image of the recommended image, for example, as schematically shown in the lower left figure of FIG. Alternatively, the thumbnail image of the recommended image may be displayed independently in the so-called Picture-in-picture display format. Further, the recommendation unit 213 is provided with a display area for displaying information about the recommended image around the visible light image, as schematically shown in the lower right figure of FIG. Text information indicating the type of the recommended image, a thumbnail image of the recommended image, text information explaining the recommended image, and the like may be displayed together.

By generating the display screen as shown in FIG. 8, it is possible for the user of the medical observation device 10 to select whether or not to display the recommended image on the display screen.

Further, it is not in one display screen as shown in FIG. 8, but as schematically shown in FIG. 9, it is not a main monitor on which a visible light image is displayed, but is provided separately from the main monitor. The sub monitor may display the recommended image as described above or information related to the recommended image.

The display of information on the recommended image as shown in FIGS. 8 and 9 may be deleted each time when the user of the medical observation device 10 does not select the use of the recommended image. .. Further, the display of the information regarding the recommended image as shown in FIGS. 8 and 9 may be automatically deleted after a certain period of time has elapsed. In addition, the display of information on the recommended image as shown in FIGS. 8 and 9 may be recorded by various methods so that the reproduction can be confirmed during or after the operation.

On the other hand, when the user of the medical observation device 10 selects to use the recommended image as a result of displaying the information regarding the recommended image as shown in FIGS. 8 and 9, the selected recommended image is displayed on the display screen. It will be displayed. Further, the timing of re-switching from the recommended image to the normal visible light image is not particularly limited, and is appropriately performed according to the user operation of the user of the medical observation device 10.

The method of presenting the recommended image determined by the recommendation unit 213 has been briefly described above.

[Specific examples of recommended images]
In the following, specific examples of recommended images will be briefly described with some examples.

○ Case 1
For example, when bleeding occurs during an operation under the medical observation device 10, it is important for the user of the medical observation device 10 to detect the bleeding point as soon as possible and perform hemostasis treatment. However, when there is a large amount of bleeding, the surrounding area may become covered with blood, making it difficult to find a bleeding point.

At this time, it is assumed that the blood is recognized by the situation specifying process (for example, the analysis process of the captured image in a specific wavelength band) by the situation specifying unit 211, and the situation of bleeding is specified. On the other hand, in the near-infrared band, the absorption coefficient of blood is low as shown in FIG. 6, and light in the near-infrared band passes through the blood. Therefore, the recommendation unit 213 selects an observation image in a wavelength band having a relatively low blood absorption coefficient as a recommended image by referring to a recommended image database or the like. By using such a recommended image, the user of the medical observation device 10 can easily see the surface of the living body which is buried in blood and is difficult to see, and can easily find the bleeding point.

○ Case 2
For example, when an energy device such as an electric knife is used, the inside of the body cavity may be covered with surgical smoke or mist, resulting in poor visibility.

At this time, it is said that the situation that surgical smoke or mist is being generated has been identified by the situation identification process (for example, the analysis process of the captured image in a specific wavelength band and the information on the operating status of various devices) by the situation identification unit 211. To do. In this case, the recommendation unit 213 refers to a recommended image database or the like and selects an observation image in the near infrared band having a relatively low absorption coefficient of surgical smoke or mist as the recommended image. By using such a recommended image, the user of the medical observation device 10 can continue the medical practice without interrupting the work by surgical smoke or mist.

In addition, if it is determined that the surgical smoke, mist, etc. have disappeared due to the normal operation of the smoke exhaust device, etc. after the above situation has passed for a while, the recommendation section 213 is based on the observation image in the near infrared band. , The display may be switched to the observation image in the normal visible light band. As a result, the user of the medical observation device 10 can continue the medical practice by using an image that is usually familiar to the user.

○ Case 3
For example, when safely excising a blood vessel, a doctor puts a blood flow blocking medical device such as a clip on the blood vessel to completely block the blood, and then excises the blood vessel with a predetermined surgical tool. At this time, the blood flow may not be completely blocked due to insufficient application of a blood flow blocking medical device such as a clip. Resection of blood vessels in situations where blood flow is not completely blocked can result in unintended bleeding. Similarly, it is necessary to appropriately anastomose blood vessels and tissues with each other using a vascular anastomosis medical device such as a stapler to confirm whether blood flow is resumed.

At this time, it is recommended when the situation information indicating that the blood flow blocking medical device or the blood vessel anastomosis medical device is detected is generated by the situation specifying process (for example, image recognition processing of a clip or stapler) by the situation specifying unit 211. As secondary information based on the biological image, the part 213 can calculate the oxygen saturation by a known method and visualize the distribution of the oxygen saturation as a recommended image.

It is also possible to use an image of a living body in the near-infrared band corresponding to the distribution of blood as a recommended image instead of an image that visualizes the distribution of oxygen saturation. For example, a fluorescent reagent called ICG (indocyanine green) has a property of emitting fluorescence of 830 nm by excitation light of 760 nm by binding to plasma protein in blood. Therefore, when the above situation is grasped, the recommendation unit 213 first recommends the use of the fluorescent reagent to the doctor who is the user of the medical observation device 10, and is generated due to the fluorescent reagent. The near-infrared image having a wavelength near 830 nm may be recommended as an observation image in the near-infrared band corresponding to the distribution of blood.

○ Case 4
For example, during an operation under the medical observation device 10, the user of the medical observation device 10 may want to know information on the running of blood vessels and the depth of blood vessels, but imaging in a general visible light band may occur. In the image, such information may not be sufficiently obtained.

At this time, when it is specified that a blood vessel exists in the biological image by the situation identification process (for example, the analysis process of the captured image in a specific wavelength band) by the situation identification unit 211, the recommendation unit 213 determines. By means similar to the NBI principle, an image in a wavelength band suitable for visually recognizing blood vessels can be recommended as a blood vessel-enhanced image.

The specific examples of the recommended images have been briefly described above with some examples, but the recommended images in the information processing apparatus 20 according to the present embodiment are not limited to the above examples, and various known images are available. The image can be used as a recommended image.

<Information processing method>
Next, the flow of the information processing method according to the present embodiment will be briefly described with reference to FIG. FIG. 10 is a flow chart showing an example of the flow of the information processing method according to the present embodiment.

In the information processing method according to the present embodiment, first, a biological image is generated by the medical observation device 10 and output to the information processing device 20. When the information acquisition unit 201 of the information processing device 20 acquires the biological image data output from the medical observation device 10 (step S101), the information acquisition unit 201 outputs the acquired data to the image recommendation processing unit 203.

The situation specifying unit 211 of the image recommendation processing unit 203 identifies the situation when the inside of the living body is observed by the method as described above (step S103), and recommends the situation information regarding the specified situation. Output to 213.

The recommendation unit 213 determines whether or not there is an image that can be recommended to the user or the like of the medical observation device 10 by using the situation information or the like output from the situation identification unit 211 (step S105). If there is no recommended image (step S105-NO), step S115, which will be described later, is performed. On the other hand, when there is a recommended image (step S105-YES), the recommendation unit 213 notifies the user of the medical observation device 10 of the information regarding the recommended image by the method described above (step S105-YES). Step S107).

Here, when the user or the like of the medical observation device 10 does not select the recommended image (step S109-NO), step S115 described later is carried out. On the other hand, when the user or the like of the medical observation device 10 selects the recommended image (step S109-YES), the recommended image generation unit 215 generates the recommended image as necessary (step S111), and the output process is performed. The recommended image is displayed on the screen when the unit 205, the control unit 101 of the medical observation device 10, and the like function in cooperation with each other (step S113). After that, the process returns to step S101, and the information processing method according to the present embodiment is continued.

On the other hand, if it is determined in step S105 that there is no recommended image, or if the recommended image is not selected in step S109, an image of the living body in the visible light band is generated (step S115). The visible light image is displayed on the screen by the function of the output processing unit 205, the control unit 101 of the medical observation device 10, and the like in cooperation with each other (step S117). After that, the process returns to step S101, and the information processing method according to the present embodiment is continued.

As described above, the flow of the information processing method according to the present embodiment has been briefly described with reference to FIG.

<About hardware configuration>
Next, the hardware configuration of the information processing apparatus 20 according to the embodiment of the present disclosure will be described in detail with reference to FIG. FIG. 11 is a block diagram for explaining the hardware configuration of the information processing apparatus 20 according to the embodiment of the present disclosure.

The information processing device 20 mainly includes a CPU 901, a ROM 903, and a RAM 905. Further, the information processing device 20 further includes a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, and a connection port 923. And a communication device 925.

The CPU 901 functions as an arithmetic processing device and a control device, and controls all or a part of the operation in the information processing device 20 according to various programs recorded in the ROM 903, the RAM 905, the storage device 919, or the removable recording medium 927. The ROM 903 stores programs, calculation parameters, and the like used by the CPU 901. The RAM 905 primarily stores the program used by the CPU 901, parameters that change as appropriate in the execution of the program, and the like. These are connected to each other by a host bus 907 composed of an internal bus such as a CPU bus.

The host bus 907 is connected to an external bus 911 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 909.

The input device 915 is an operating means operated by the user, such as a mouse, a keyboard, a touch panel, buttons, switches, and levers. Further, the input device 915 may be, for example, a remote control means (so-called remote controller) using infrared rays or other radio waves, or an externally connected device such as a mobile phone or PDA that supports the operation of the information processing device 20. It may be 929. Further, the input device 915 is composed of, for example, an input control circuit that generates an input signal based on the information input by the user using the above-mentioned operating means and outputs the input signal to the CPU 901. By operating the input device 915, the user can input various data to the information processing device 20 and instruct the processing operation.

The output device 917 is composed of a device capable of visually or audibly notifying the user of the acquired information. Such devices include display devices such as CRT display devices, liquid crystal display devices, plasma display devices, EL display devices and lamps, audio output devices such as speakers and headphones, printer devices, mobile phones, facsimiles and the like. The output device 917 outputs, for example, the results obtained by various processes performed by the information processing device 20. Specifically, the display device displays the results obtained by various processes performed by the information processing device 20 as text or an image. On the other hand, the audio output device converts an audio signal composed of reproduced audio data, acoustic data, and the like into an analog signal and outputs the signal.

The storage device 919 is a data storage device configured as an example of the storage unit of the information processing device 20. The storage device 919 is composed of, for example, a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, an optical magnetic storage device, or the like. The storage device 919 stores programs executed by the CPU 901, various data, various data acquired from the outside, and the like.

The drive 921 is a reader / writer for a recording medium, and is built in or externally attached to the information processing device 20. The drive 921 reads the information recorded on the removable recording medium 927 such as the mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 905. The drive 921 can also write a record to a removable recording medium 927 such as a mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory. The removable recording medium 927 is, for example, a DVD media, an HD-DVD media, a Blu-ray (registered trademark) media, or the like. Further, the removable recording medium 927 may be a compact flash (registered trademark) (CompactFlash: CF), a flash memory, an SD memory card (Secure Digital memory card), or the like. Further, the removable recording medium 927 may be, for example, an IC card (Integrated Circuit card) or an electronic device on which a non-contact type IC chip is mounted.

The connection port 923 is a port for directly connecting the device to the information processing device 20. As an example of the connection port 923, there are a USB (Universal Serial Bus) port, an IEEE1394 port, a SCSI (Small Computer System Interface) port, and the like. Another example of the connection port 923 is an RS-232C port, an optical audio terminal, an HDMI (High-Definition Multimedia Interface) port, and the like. By connecting the externally connected device 929 to the connection port 923, the information processing device 20 acquires various data directly from the externally connected device 929 and provides various data to the externally connected device 929.

The communication device 925 is, for example, a communication interface composed of a communication device or the like for connecting to the communication network 931. The communication device 925 is, for example, a communication card for a wired or wireless LAN (Local Area Network), Bluetooth (registered trademark), WUSB (Wireless USB), or the like. Further, the communication device 925 may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), a modem for various communications, or the like. The communication device 925 can transmit and receive signals and the like to and from the Internet and other communication devices in accordance with a predetermined protocol such as TCP / IP. Further, the communication network 931 connected to the communication device 925 is configured by a network or the like connected by wire or wirelessly, and may be, for example, the Internet, a home LAN, infrared communication, radio wave communication, satellite communication, or the like. ..

The above is an example of a hardware configuration capable of realizing the functions of the information processing apparatus 20 according to the embodiment of the present disclosure. Each of the above-mentioned components may be configured by using general-purpose members, or may be configured by hardware specialized for the function of each component. Therefore, it is possible to appropriately change the hardware configuration to be used according to the technical level at each time when the present embodiment is implemented.

Although the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field of the present disclosure can come up with various modifications or modifications within the scope of the technical ideas described in the claims. Of course, it is understood that the above also belongs to the technical scope of the present disclosure.

In addition, the effects described herein are merely explanatory or exemplary and are not limited. That is, the technique according to the present disclosure may exhibit other effects apparent to those skilled in the art from the description of the present specification, in addition to or in place of the above effects.

The following configurations also belong to the technical scope of the present disclosure.
(1)
Based on a plurality of biological images generated by imaging the living body in a plurality of wavelength bands by a medical observation device for observing the inside of the living body, and situation information regarding the situation when observing the inside of the living body. An information processing apparatus comprising a recommendation unit that determines at least one of the plurality of biological images or at least one of images generated from the plurality of biological images as a recommended image according to the situation.
(2)
The plurality of biological images include at least an image of the living body in the visible light band.
A display control unit that controls the display of the display screen on which the biological image is displayed is further provided.
The display control unit displays an image of a living body in the visible light band on the display screen, and when the user selects the display of the recommended image, the display content of the display screen is displayed as the recommended image. The information processing device according to (1), which is switched to.
(3)
The recommendation unit refers to a recommended image database in which the situation when the inside of the living body is observed and the type of the recommended image that can be presented in each of the situations are associated with each other in advance, and the situation is changed to the above situation. The information processing apparatus according to (1) or (2), which determines a recommended image according to the situation.
(4)
The recommended image using at least a part of the plurality of biological images or an image that visualizes secondary information generated by analyzing at least one of the plurality of biological images. The information processing apparatus according to any one of (1) to (3), further comprising a recommended image generation unit.
(5)
The plurality of biological images include at least an image of the living body in the visible light band and an image of the living body in the near infrared band.
The recommended image generator is
Of the image of the living body in the near-infrared band, the color tone of the region where the light in the near-infrared band is detected is changed to a predetermined color tone, and the region in which the light in the near-infrared band is not detected. After changing the brightness value to zero, the superimposed image obtained by superimposing the changed image of the living body in the near-infrared band on the image of the living body in the visible light band is used as the recommended image. The information processing apparatus according to (4), which is generated.
(6)
The plurality of biological images include at least an image of the living body in the visible light band.
The recommendation unit displays information about the recommended image on a display screen on which an image of a living body captured in the visible light band is displayed, and allows the user to select whether or not to display the recommended image. The information processing apparatus according to any one of 1) to (5).
(7)
The information processing apparatus according to (6), wherein the information regarding the recommended image is at least one of text information indicating the type of the recommended image and thumbnail image of the recommended image.
(8)
The recommended unit is any one of (1) to (7) that displays the recommended image on a display screen different from the display screen on which the captured image of the living body in the visible light band is displayed. The information processing device described in.
(9)
Further provided with a situation specifying part for specifying the situation when the inside of the living body is observed,
The situation specifying unit is set information regarding the setting of the medical observation device, characteristic information regarding the characteristics of the medical observation device, and operation information regarding the operating state of the medical observation device and peripheral devices linked with the medical observation device. , (1) to (8), which specifies the situation when the inside of the living body is observed based on at least one of the biological image or secondary information generated based on the biological image. The information processing device according to one.
(10)
The situation specifying unit is set information regarding the setting of the medical observation device, characteristic information regarding the characteristics of the medical observation device, and operation information regarding the operating state of the medical observation device and peripheral devices linked with the medical observation device. , The biological image, or at least one of the secondary information generated based on the biological image, and the situation when the biological image is generated, refer to a situation-specific database in which the biological image is associated with each other in advance, and the living body. The information processing apparatus according to (9), which identifies the situation when the inside of the computer is being observed.
(11)
The situation-specific database further records information on the spectral reflectance or extinction coefficient of the organs and biological substances existing inside the living body.
The situation specifying unit generates information on spectral reflectance as secondary information generated based on the biological image, or specifies the extinction coefficient in a specific wavelength band with reference to the biological image, and the above-mentioned The information processing apparatus according to (10), which identifies the situation when the inside of the living body is observed by comparing with the information on the spectral reflectance or the extinction coefficient of the organs existing inside the living body and the biological substance. ..
(12)
The situation-specific database further records image information about organs existing inside the living body and medical instruments used for medical practice.
The situation specifying unit generates, as secondary information generated based on the biological image, information on the recognition result of performing image recognition processing on the biological image using the image information.
The information processing apparatus according to (10) or (11), which identifies a situation when the inside of the living body is observed by using the information regarding the recognition result.
(13)
The biological image includes at least an image of the living body in the visible light band and an image of the living body in the near infrared band.
The recommended unit determines an image of a living body in the near-infrared band as the recommended image when it acquires information that blood has been detected as the situation information, according to (1) to (12). The information processing device according to any one.
(14)
The biological image includes at least an image of the living body in the visible light band and an image of the living body in the near infrared band.
When the recommended part acquires at least one of the information that smoke or mist is detected or the information about the use of the medical energy device as the situation information, the living body in the near infrared band The information processing apparatus according to any one of (1) to (13), wherein the captured image is determined as the recommended image.
(15)
The recommendation unit is displayed on the display screen when at least one of the information that smoke or mist is no longer detected or the information that the use of the medical energy device is finished is acquired as the status information. The information processing apparatus according to (14), wherein the image to be processed is switched from the recommended image to an image of a living body in the visible light band.
(16)
The recommended part distributes the oxygen saturation generated as secondary information based on the biological image when the information that the blood flow blocking medical device or the vascular anastomosis medical device is detected is acquired as the situation information. The information processing according to any one of (1) to (15), wherein the image obtained by visualizing the above image or the image of the living body in the near infrared band corresponding to the blood distribution is determined as the recommended image. apparatus.
(17)
When the recommendation unit acquires information indicating that the presence of a blood vessel has been detected as the situation information, the recommendation unit determines an image of the living body in a wavelength band suitable for visually recognizing the blood vessel as the recommended image. 1) The information processing apparatus according to any one of (16).
(18)
The medical observation device has a multi-spectral sensor including at least pixels that distinguish light intensities in a plurality of wavelength bands for each wavelength band and detect them at a time, and from the output results of the multi-spectral sensor, a plurality of. The information processing apparatus according to any one of (1) to (17), wherein the biological image is generated in the wavelength band of (1) to (17).
(19)
The medical observation device has a plurality of image pickup elements corresponding to a specific wavelength band, and the biological image in a plurality of wavelength bands is generated from the output results of the respective image pickup devices (1). The information processing apparatus according to any one of (17).
(20)
The medical observation device generates the biological image in a plurality of wavelength bands by performing imaging while switching the wavelength band of the light imaged on the image sensor included in the medical observation device (1) to The information processing apparatus according to any one of (17).
(21)
The information processing device according to any one of (1) to (20), wherein the medical observation device is a medical endoscope or a medical microscope.
(22)
Based on a plurality of biological images generated by imaging the living body in a plurality of wavelength bands by a medical observation device for observing the inside of the living body, and situation information regarding the situation when observing the inside of the living body. An information processing method comprising determining at least one of the plurality of biological images, or at least one of images generated from the plurality of biological images, as a recommended image according to the situation.
(23)
A computer capable of intercommunication with a medical observation device that generates a plurality of biological images generated by observing the inside of a living body and imaging the living body in a plurality of wavelength bands.
At least one of the plurality of biological images, or at least an image generated from the plurality of biological images, based on the plurality of biological images and situation information regarding the situation when observing the inside of the living body. A program for realizing a recommended function of determining any of them as a recommended image according to the above situation.
(24)
A medical observation device that observes the inside of a living body and generates a plurality of biological images by imaging the living body in a plurality of wavelength bands.
At least one of the plurality of biological images, or a plurality of the plurality of biological images, based on the plurality of biological images generated by the medical observation device and situation information regarding the situation when observing the inside of the biological images. An information processing device having a recommended unit for determining at least one of the images generated from the biological image of the above as a recommended image according to the situation.
A medical observation system equipped with.

1 Medical observation system 10 Medical observation equipment 20 Information processing device 101 Control unit 103 Observation unit 105 Light source unit 107 Imaging unit 109 Display unit 201 Information acquisition unit 203 Image recommendation processing unit 205 Output processing unit 207 Display control unit 209 Storage unit 211 Situation identification part 213 Recommended part 215 Recommended image generation part

The present disclosure relates to an information processing system, an information processing method and program.

Claims (24)

Based on a plurality of biological images generated by imaging the living body in a plurality of wavelength bands by a medical observation device for observing the inside of the living body, and situation information regarding the situation when observing the inside of the living body. An information processing apparatus comprising a recommendation unit that determines at least one of the plurality of biological images or at least one of images generated from the plurality of biological images as a recommended image according to the situation. The plurality of biological images include at least an image of the living body in the visible light band.
A display control unit that controls the display of the display screen on which the biological image is displayed is further provided.
The display control unit displays an image of a living body in the visible light band on the display screen, and when the user selects the display of the recommended image, the display content of the display screen is displayed as the recommended image. The information processing apparatus according to claim 1, which is switched to. The recommendation unit refers to a recommended image database in which the situation when the inside of the living body is observed and the type of the recommended image that can be presented in each of the situations are associated with each other in advance, and the situation is changed to the above situation. The information processing apparatus according to claim 1, wherein a recommended image is determined accordingly. The recommended image using at least a part of the plurality of biological images or an image that visualizes secondary information generated by analyzing at least one of the plurality of biological images. The information processing apparatus according to claim 1, further comprising a recommended image generation unit. The plurality of biological images include at least an image of the living body in the visible light band and an image of the living body in the near infrared band.
The recommended image generator is
Of the image of the living body in the near-infrared band, the color tone of the region where the light in the near-infrared band is detected is changed to a predetermined color tone, and the region in which the light in the near-infrared band is not detected After changing the brightness value to zero, the superimposed image obtained by superimposing the changed image of the living body in the near-infrared band on the image of the living body in the visible light band is used as the recommended image. The information processing apparatus according to claim 4, which is generated. The plurality of biological images include at least an image of the living body in the visible light band.
The recommendation unit displays information about the recommended image on a display screen on which an image of a living body captured in the visible light band is displayed, and allows the user to select whether or not to display the recommended image. Item 1. The information processing apparatus according to item 1. The information processing apparatus according to claim 6, wherein the information regarding the recommended image is at least one of text information indicating the type of the recommended image and thumbnail image of the recommended image. The information processing apparatus according to claim 1, wherein the recommended unit displays the recommended image on a display screen different from the display screen on which the captured image of the living body in the visible light band is displayed. Further provided with a situation specifying part for specifying the situation when the inside of the living body is observed,
The situation specifying unit is set information regarding the setting of the medical observation device, characteristic information regarding the characteristics of the medical observation device, and operation information regarding the operating state of the medical observation device and peripheral devices linked with the medical observation device. The information processing apparatus according to claim 1, wherein the information processing apparatus according to claim 1 specifies a situation when the inside of the living body is observed based on at least one of the biological image or secondary information generated based on the biological image. .. The situation specifying unit is set information regarding the setting of the medical observation device, characteristic information regarding the characteristics of the medical observation device, and operation information regarding the operating state of the medical observation device and peripheral devices linked with the medical observation device. , The biological image, or at least one of the secondary information generated based on the biological image, and the situation when the biological image is generated, refer to a situation-specific database in which the biological image is associated with each other in advance, and the living body. The information processing apparatus according to claim 9, which specifies a situation when the inside of the device is observed. The situation-specific database further records information on the spectral reflectance or extinction coefficient of the organs and biological substances existing inside the living body.
The situation specifying unit generates information on spectral reflectance as secondary information generated based on the biological image, or specifies the extinction coefficient in a specific wavelength band with reference to the biological image, and the above-mentioned The information processing apparatus according to claim 10, wherein the situation when the inside of the living body is observed is specified by comparing with the information on the spectral reflectance or the extinction coefficient of the organs existing inside the living body and the biological substance. .. The situation-specific database further records image information about organs existing inside the living body and medical instruments used for medical practice.
The situation specifying unit generates, as secondary information generated based on the biological image, information on the recognition result of performing image recognition processing on the biological image using the image information.
The information processing apparatus according to claim 10, wherein the information processing device according to claim 10 specifies a situation when the inside of the living body is observed by using the information regarding the recognition result. The biological image includes at least an image of the living body in the visible light band and an image of the living body in the near infrared band.
The information processing according to claim 1, wherein the recommended unit determines an image of a living body in the near-infrared band as the recommended image when the information that blood is detected is acquired as the situation information. apparatus. The biological image includes at least an image of the living body in the visible light band and an image of the living body in the near infrared band.
When the recommended part acquires at least one of the information that smoke or mist is detected or the information about the use of the medical energy device as the situation information, the living body in the near infrared band The information processing apparatus according to claim 1, wherein the captured image is determined as the recommended image. The recommendation unit is displayed on the display screen when at least one of the information that smoke or mist is no longer detected or the information that the use of the medical energy device is finished is acquired as the status information. The information processing apparatus according to claim 14, wherein the image to be processed is switched from the recommended image to an image of a living body in the visible light band. The recommended part distributes the oxygen saturation generated as secondary information based on the biological image when the information that the blood flow blocking medical device or the vascular anastomosis medical device is detected is acquired as the situation information. The information processing apparatus according to claim 1, wherein an image visualized from the above or an image of the living body in the near infrared band corresponding to the distribution of blood is determined as the recommended image. The recommendation unit determines, as the recommended image, an image of the living body in a wavelength band suitable for visually recognizing the blood vessel when the information indicating that the presence of the blood vessel is detected is acquired as the situation information. Item 1. The information processing apparatus according to item 1. The medical observation device has a multi-spectral sensor including at least pixels that distinguish light intensities in a plurality of wavelength bands for each wavelength band and detect them at a time, and from the output results of the multi-spectral sensor, a plurality of. The information processing apparatus according to claim 1, wherein the biological image is generated in the wavelength band of. The medical observation device has a plurality of image pickup elements corresponding to a specific wavelength band, and the biological image in a plurality of wavelength bands is generated from the output results of the respective image pickup devices. The information processing device described in. The medical observation device according to claim 1, wherein the medical observation device generates the biological image in a plurality of wavelength bands by performing imaging while switching the wavelength band of the light imaged on the image pickup device of the medical observation device. The information processing device described. The information processing device according to claim 1, wherein the medical observation device is a medical endoscope or a medical microscope. Based on a plurality of biological images generated by imaging the living body in a plurality of wavelength bands by a medical observation device for observing the inside of the living body, and situation information regarding the situation when observing the inside of the living body. An information processing method comprising determining at least one of the plurality of biological images, or at least one of images generated from the plurality of biological images, as a recommended image according to the situation. A computer capable of intercommunication with a medical observation device that generates a plurality of biological images generated by observing the inside of a living body and imaging the living body in a plurality of wavelength bands.
At least one of the plurality of biological images, or at least an image generated from the plurality of biological images, based on the plurality of biological images and situation information regarding the situation when observing the inside of the living body. A program for realizing a recommended function of determining any of them as a recommended image according to the above situation. A medical observation device that observes the inside of a living body and generates a plurality of biological images by imaging the living body in a plurality of wavelength bands.
At least one of the plurality of biological images, or a plurality of the plurality of biological images, based on the plurality of biological images generated by the medical observation device and situation information regarding the situation when observing the inside of the biological images. An information processing device having a recommended unit for determining at least one of the images generated from the biological image of the above as a recommended image according to the situation.
A medical observation system equipped with.

Images