WO2022209156A1 - Medical observation device, information processing device, medical observation method, and endoscopic surgery system - Google Patents

Abstract

A medical observation device (100) is equipped with: a light source device (4) for illuminating an object (9) with light; a sensor (35) for detecting that the amount of change in the brightness of light incident to each of a plurality of pixels (352) has exceeded a threshold value; and a control unit (5) that changes a light illumination profile of the light source device (4), and measures the distance to the object (9) on the basis of the detection result which is acquired from the sensor (35) and is generated as a result of the change of the light illumination profile.

Description

Medical observation device, information processing device, medical observation method, and endoscopic surgery system

The present disclosure relates to a medical observation device, an information processing device, a medical observation method, and an endoscopic surgery system.

It is known to control an arm that supports an endoscope based on an image captured by the endoscope (see Patent Document 1, for example).

JP 2015-123201 A WO2018/123456

　There is a demand for distance measurement in endoscopic and microsurgery, and distance measurement technology for use in such surgeries is still under consideration.

One aspect of the present disclosure aims to provide a distance measurement technique suitable for surgery.

A medical observation apparatus according to one aspect includes a light source device that irradiates light onto an object, a sensor that detects that the amount of change in brightness due to light incident on each of a plurality of pixels exceeds a predetermined threshold, and a light source device. a control unit that changes the irradiation profile of light and measures the distance to the object based on the detection result of the sensor generated by the change in the irradiation profile of light.

An information processing device according to one aspect changes the light irradiation profile of a light source device that irradiates light onto an object, and calculates the distance to the object based on the sensor detection result generated by the change in the light irradiation profile. and a control unit for measuring, and the sensor detects that the amount of change in luminance due to light incident on each of the plurality of pixels exceeds a predetermined threshold.

In a medical observation method according to one aspect, an information processing device changes a light irradiation profile of a light source device that irradiates light onto an object, and based on a sensor detection result generated by the change in the light irradiation profile, measuring the distance to the object, and the sensor detects that the amount of change in brightness due to light incident on each of the plurality of pixels exceeds a predetermined threshold.

An endoscopic surgery system according to one aspect includes an endoscope including a sensor that detects that a luminance change due to light incident on each of a plurality of pixels exceeds a predetermined threshold, and an arm that supports the endoscope. and a light source device for irradiating an object with light, and a control device for controlling the endoscope, the arm portion, and the light source device, wherein the control device changes the light irradiation profile of the light source device to irradiate the light. The distance to the object is measured based on the detection result of the sensor caused by the profile change.

1 is a diagram showing an example of a schematic configuration of a medical observation device according to an embodiment; FIG. It is a figure which shows the example of a schematic structure of an endoscope. It is a figure which shows the example of a schematic structure of a sensor (EVS). It is a figure which shows the example of the schematic structure of a pixel. 4 is a flow chart showing an example of processing executed in the medical observation device; FIG. 5 is a diagram schematically showing an example of changes in the irradiation range; It is a figure which shows typically the example of the relationship between an incident angle and an irradiation range. It is a figure which shows typically the combination of several irradiation. 4 is a flowchart showing an example of processing including temporary suspension of irradiation (turning off the light); It is a figure which shows the example of a schematic structure of a light source device. 1 is a diagram showing an example of a schematic configuration of an endoscopic surgery system; FIG. 4 is a flow chart showing an example of processing executed in the endoscopic surgery system; 4 is a flow chart showing an example of processing executed in the endoscopic surgery system; It is a figure which shows the example of the hardware constitutions of an apparatus.

Below, embodiments of the present disclosure will be described in detail based on the drawings. In addition, in each of the following embodiments, the same reference numerals are given to the same elements to omit redundant description.

The present disclosure will be described according to the order of items shown below.
1. Embodiment of Medical Observation Device 2 . Example of irradiation profile 3 . Embodiment of Endoscopic Surgery System4. Example of hardware configuration5. Example of effect

1. Embodiment of Medical Observation Apparatus FIG. 1 is a diagram showing a schematic configuration of a medical observation apparatus according to an embodiment. The medical observation device 100 includes an endoscope 1, a light source device 4, and a CCU5.

The endoscope 1 images the operative field (operative site) within the body cavity of the patient. An object within the operative field is referred to as object 9 and illustrated. Examples of the object 9 are organs, surgical tools, and the like. The endoscope 1 may be a forward viewing scope or a perspective scope. FIG. 1 illustrates a lens barrel 2 and a camera head 3 as components of an endoscope 1, details of which will be described later with reference to FIG.

In addition, in the present disclosure, "imaging" may mean including "shooting". Hereinafter, "imaging" and/or "video" will simply be referred to as "imaging" unless otherwise specified. As long as there is no contradiction, "imaging" may be read as "shooting". Also, the term "image" may include "video". Hereinafter, "image" and/or "video" are simply referred to as "image" unless otherwise specified. As long as there is no contradiction, "image" may be read as "video".

The light source device 4 irradiates the object 9 with light. The light source device 4 supplies the endoscope 1 with light for irradiating the surgical field. The light source device 4 may include a light source such as an LED (Light Emitting Diode) light source, a laser light source, or the like. The light source device 4 includes a light exit port 43 through which light from the light source is emitted. The light exit port 43 is connected to the endoscope 1 via a cable C14. The cable C14 includes a light guide (for example, an optical waveguide such as an optical fiber) that propagates light. The light exit port 43 can also be said to be a light guide insertion port into which the light guide is inserted.

An example of the light that the light source device 4 irradiates onto the object 9 is illumination light for illuminating the surgical field. An example of illumination light is white light, but it is not particularly limited to this.

Another example of the light with which the light source device 4 irradiates the object 9 is light in a predetermined wavelength band corresponding to special light observation. Special light observation may be so-called narrow band imaging. In that case, for example, by utilizing the wavelength dependence of light absorption in body tissues, the object 9 is irradiated with light that is narrower than the irradiation light (for example, white light) during normal observation, and as a result, Predetermined tissues such as blood vessels on the mucosal surface are imaged with high contrast. Special light observation may be fluorescence observation. In that case, an image is obtained by fluorescence generated by irradiation of the object 9 with excitation light. For example, when the object 9 is a body tissue, the body tissue is irradiated with excitation light and fluorescence from the body tissue is observed (autofluorescence observation). A reagent such as indocyanine green (ICG) may be locally injected into the body tissue, and the body tissue may be irradiated with excitation light corresponding to the fluorescence wavelength of the reagent to obtain a fluorescence image. The light source device 4 may be configured to be capable of supplying narrow-band light and/or excitation light corresponding to such special light observation.

Another example of the light that the light source device 4 irradiates onto the object 9 is light for measuring the distance to the object 9 (for distance measurement). Various lights described above, including illumination light, may be used as the light for distance measurement. The light for distance measurement is not limited to visible light, and may be infrared light or ultraviolet light. The light source device 4 may be configured to change the wavelength of the light supplied to the endoscope 1, which will be explained later with reference to FIG.

The CCU 5 can function as a control unit in the medical observation device 100. CCU 5 is a camera control unit that controls endoscope 1 . CCU 5 is connected to endoscope 1 via cable C15. The cable C<b>15 includes signal lines and the like, and enables control of the endoscope 1 by the CCU 5 . The CCU 5 includes, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and the like.

The CCU 5 supervises the medical observation device 100, including the operation of a display device (corresponding to the display device 5041 in FIG. 11 described later) that displays an image of the surgical site in the patient's body cavity captured by the endoscope 1. control. The CCU 5 can also be said to be a control device that performs overall control of the medical observation device 100 . The CCU 5 can also be said to be an information processing device that processes various information handled by the medical observation device 100 .

The CCU 5 controls irradiation of light to the object 9 by controlling the light source device 4 . CCU5 is connected to light source device 4 via cable C45. The cable C45 includes signal lines and the like, and enables control of the light source device 4 by the CCU5. For example, the CCU 5 transmits to the light source device 4 an illumination profile that defines the mode of illumination. The irradiation profile will be explained again later. The light source device 4 supplies light to the endoscope 1 as defined by the received illumination profile.

FIG. 2 is a diagram showing an example of a schematic configuration of an endoscope. As mentioned earlier, the endoscope 1 includes a lens barrel 2 and a camera head 3. The illustrated lens barrel 2 is rigid, and the endoscope 1 is configured as a so-called rigid scope. The lens barrel 2 extends with the camera head 3 as its base end. An example of the size of the lens barrel 2 is several millimeters to ten and several millimeters in diameter and several hundred millimeters in length. Note that the lens barrel 2 may be flexible and the endoscope 1 may be configured as a rigid endoscope.

The lens barrel 2 is inserted into the patient's body cavity for a predetermined length from its tip. As components of the lens barrel 2, a mounting portion 21, a light guide 22, lenses 23a to 23g, a mounting portion 24, and an opening 25 are exemplified. Also, an opening at the tip of the lens barrel 2 on the side opposite to the camera head 3 (on the side of the object 9) is referred to as an opening 2a.

The attachment portion 21 is a portion to which the cable C14 from the light source device 4 is attached. Light from the light source device 4 is supplied to the endoscope 1 via the cable C14 and the mounting portion 21 .

The light guide 22 extends inside the lens barrel 2 and guides the light from the light source device 4 to the tip of the lens barrel 2 so that the light from the light source device 4 irradiates the object 9 from a predetermined position and in a predetermined direction. The tip of the light guide 22 at the tip of the lens barrel 2 is referred to as an emitting end 22a and illustrated. The light from the light source device 4 is emitted from the emission end 22 a to irradiate the object 9 . The position of the output end 22a corresponds to the predetermined position, and the direction of light emitted from the light guide 22 corresponds to the predetermined direction. These predetermined positions and predetermined directions are grasped by the medical observation device 100, more specifically by the CCU 5, and enable distance measurement, which will be described later. At least part of the light applied to the object 9 is reflected by the object 9 , and at least part of the reflected light enters the opening 25 . The light incident on the opening 2a from the object 9 in this way may be simply referred to as "light from the object 9" below.

The mounting portion 23 is a portion (for example, an eyepiece) that connects the base end of the lens barrel 2 to the camera head 3 .

The lenses 24a to 24g guide light from the object 9 to the camera head 3. In this example, the lenses 24a to 24g are provided inside the lens barrel 2 in order from the tip (the end on the opening 2a side) of the lens barrel 2 toward the base end. The lens 24a positioned closest to the object 9 can function as an objective lens. Note that the number of lenses provided in the lens barrel 2 is merely an example. More lenses than shown may be provided inside the barrel 2, for example several tens or even 100 or more. Light from the object 9 reaches the camera head 3 through many lenses provided in such a lens barrel 2 .

The camera head 3 detects light from the object 9. As components of the camera head 3, a mounting portion 31, a lens 32a, an optical element 32b, sensors 33 to 35, and a vibration blur correction mechanism 36 are exemplified.

The attachment portion 31 is a portion to which the base end portion of the lens barrel 2 is attached, and the attachment portion 23 of the lens barrel 2 described above is connected. Light from the object 9 enters the camera head 3 through the mounting portion 31 .

The lens 32a and the optical element 32b collect light from the object 9 onto the sensors 33 to 35, respectively. The lens 32a guides observation light to the optical element 32b. The optical element 32b guides the light to be detected by the sensor 33 to the sensor 33, the light to be detected by the sensor 34 to the sensor 34, and the light to be detected by the sensor 35 to the sensor 35, respectively. The optical element 32b includes various known optical elements such as mirrors and prisms.

The sensor 33 is an imaging device (first imaging sensor) that captures an image of the object 9 . For example, the sensor 33 is an imaging element for visible light observation, and includes RGB pixels and the like. In that case, the optical element 32b described above guides the light from the object 9, especially the visible light, to the sensor 33. FIG. The sensor 33 generates an electrical signal (image signal) through photoelectric conversion. The generated image signal is transmitted to CCU5.

The sensor 34 is an imaging device (second imaging sensor) that captures an image of the object 9 . For example, the sensor 34 is an imaging element for special light observation, and includes infrared light pixels and the like. In that case, the optical element 32b described above guides light from the object 9, particularly infrared light, to the sensor . The sensor 34 generates an electrical signal (image signal) through photoelectric conversion. The generated image signal is transmitted to CCU5.

The camera head 3 may be provided with a drive mechanism or the like for appropriately driving the optical system such as the lens 32a and the optical element 32b. Adjustment of magnification, focal length, etc. becomes possible. Also, for example, in order to cope with stereoscopic vision (3D display) or the like, a plurality of sensors 33 or a plurality of sensors 34 may be provided. In this case, a plurality of relay optical systems may be provided inside the lens barrel 2 in order to guide the observation light to each of the plurality of imaging elements.

The sensor 35 is a sensor (brightness change detection sensor) that detects a brightness change due to light from the object 9 . The sensor 35 is also called an EVS (Event Vision Sensor) or the like. Sensor 35 will be described with reference also to FIGS.

FIG. 3 is a diagram showing an example of the schematic configuration of the sensor (EVS). The sensor 35 includes a pixel array section 351 including a plurality of pixels 352 , a driving circuit 353 , a signal processing section 354 , an arbiter section 355 (arbitration section), and a column processing section 356 . The drive circuit 353 , signal processing section 354 , arbiter section 355 and column processing section 356 are peripheral circuit sections of the pixel array section 351 . The sensor 35 is a sensor that detects that the luminance change amount due to light incident on each of the plurality of pixels 352 exceeds a predetermined threshold.

A plurality of pixels 352 are arranged in a matrix in this example. Each pixel 352 can generate a voltage corresponding to a photocurrent generated by photoelectric conversion as a pixel signal. Each pixel 352 can detect the presence or absence of an event by comparing a change in photocurrent corresponding to a change in luminance of incident light with a predetermined threshold. In other words, each pixel 352 can detect an event based on a luminance change exceeding a predetermined threshold.

When detecting an event, each pixel 352 can output a request to the arbiter unit 355 requesting output of event data representing the occurrence of the event. Each pixel 352 outputs the event data to the driving circuit 353 and the signal processing unit 354 when receiving a response indicating permission to output the event data from the arbiter unit 355 . Also, the pixel 352 that has detected the event outputs a pixel signal generated by photoelectric conversion to the column processing unit 356 .

The drive circuit 353 can drive each pixel 352 of the pixel array section 351 . For example, the drive circuit 353 detects an event, drives the pixel 352 that outputs the event data, and outputs the pixel signal of the corresponding pixel 352 to the column processing unit 356 .

The arbiter unit 355 arbitrates requests requesting output of event data supplied from each pixel 352, responds based on the arbitration result (permission/non-permission of event data output), and resets event detection. A reset signal can be sent to the pixel 352 to do so.

The column processing unit 356 can perform processing for converting analog pixel signals output from the pixels 352 of the corresponding column into digital signals for each column of the pixel array unit 351 . The column processing unit 356 can also perform CDS (Correlated Double Sampling) processing on digitized pixel signals.

The signal processing unit 354 performs predetermined signal processing on the digitized pixel signals supplied from the column processing unit 356 and the event data output from the pixel array unit 351, and converts the signal-processed event data ( time stamp information, etc.) and pixel signals can be output. These event data, pixel signals, etc. are transmitted to the CCU 5 .

A change in the photocurrent generated by the pixel 352 can be regarded as a change in the amount of light (luminance change) incident on the pixel 352 . Therefore, an event can be said to be a luminance change of pixel 352 exceeding a predetermined threshold. Furthermore, the event data representing the occurrence of an event can include at least positional information such as coordinates representing the position of the pixel 352 where the change in the amount of light has occurred as an event. Event data including such position information is transmitted to CCU5.

FIG. 4 is a diagram showing an example of a schematic configuration of a pixel. The pixel 352 includes a light receiving portion 3521 , a pixel signal generation portion 3522 and an event detection portion 3523 .

The light receiving section 3521 can photoelectrically convert incident light to generate a photocurrent. Then, the light receiving unit 3521 can supply a voltage signal corresponding to the photocurrent to either the pixel signal generation unit 3522 or the event detection unit 3523 under the control of the driving circuit 353 .

The pixel signal generation unit 3522 can generate the signal supplied from the light receiving unit 3521 as a pixel signal. Then, the pixel signal generation section 3522 can supply the generated analog pixel signals to the column processing section 356 via vertical signal lines VSL (not shown) corresponding to the columns of the pixel array section 351 .

The event detection unit 3523 can detect whether an event has occurred based on whether the amount of change in photocurrent from the light receiving unit 3521 has exceeded a predetermined threshold. The events can include, for example, an ON event indicating that the amount of change in photocurrent has exceeded the upper threshold, and an OFF event indicating that the amount of change has fallen below the lower threshold. Note that the event detection unit 3523 may detect only on-events.

When an event occurs, the event detection unit 3523 can output to the arbiter unit 355 a request to output event data representing the occurrence of the event. Then, when the event detection unit 3523 receives a response to the request from the arbiter unit 355 , it can output event data to the drive circuit 353 and the signal processing unit 354 .

With the configuration as described above, the sensor 35 achieves a high dynamic range, high robustness in fast-moving subject detection, high temporal resolution, and the like. By having a particularly high dynamic range, light from the object 9 can be detected with high sensitivity.

Returning to FIG. 2, the vibration blur correction mechanism 36 is provided for the sensor 35, and suppresses blur that may occur in light detection by the sensor 35 due to vibration of the endoscope 1 for light detection by the sensor 35 or the like. Various known mechanisms may be adopted as the vibration blur correction mechanism 36 .

The CCU 5 can function as a distance measuring unit that measures the distance to the object 9 using the endoscope 1 and light source device 4 described above. The CCU 5 changes the light irradiation profile of the light source device 4 and measures the distance to the object 9 based on the detection result of the sensor 35 generated by the change in the light irradiation profile. A command instructing to change the irradiation profile may be transmitted from the CCU 5 to the light source device 4 , or the changed irradiation profile may be transmitted from the CCU 5 to the light source device 4 . In any case, the light source device 4 irradiates the object 9 with light as defined by the changed irradiation profile. The light incident on the sensor 35 from the object 9 also changes. In the pixel array section 351 of the sensor 35, a change in brightness due to the change in light occurs and is detected as an event. As described above, event data including location information of pixel 352 where the event occurred is sent to CCU 5 .

The CCU 5 calculates the distance to the object 9 based on the generated detection result of the sensor 35, that is, the event data transmitted from the sensor 35 to the CCU 5. The calculated distance of the target object 9 may be the distance (depth) from an arbitrary position grasped by the CCU 5 to the target object 9 . For example, the CCU 5 calculates the distance from the tip of the barrel 2 of the endoscope 1 to the object 9, the distance from the sensor 35 to the object 9, and the like. As described above, the light from the light source device 4 irradiates the object 9 from a predetermined position and a predetermined direction ascertained by the CCU 5 . Further, the event data transmitted from the sensor 35 of the camera head 3 of the endoscope 1 to the CCU 5 includes the position information of the pixel 352 where the event was detected, that is, the information of the light receiving position of the object 9. . The CCU 5 calculates the distance to the target object 9 using various known methods using these pieces of information. For example triangulation may be used, in which case the light source device 4 acts as a laser light source and the sensor 35 acts as a position sensor. A scanning rangefinder method may be used, in which case the light source 41 may output, for example, a scannable stripe laser light.

As described above, the CCU 5 measures the distance to the object 9. The distance measurement results can be used in various ways. For example, based on the distance measurement results, three-dimensional distance measurement data of the object 9 may be constructed. It is also possible to generate a three-dimensional map containing (three-dimensional distance measurement data of) the object 9 . If the object 9 is an organ, a three-dimensional map of the patient's body cavity including the position of the organ can be generated. Self-supporting control (robot control) of the endoscope 1 based on the three-dimensional map is also possible.

The medical observation device 100 may measure the distance to the object 9 at designated timing. For example, the distance to the object 9 may be measured at a timing designated by user operation. An example of the user operation may be a voice instruction such as pressing a button (distance measurement button) for instructing distance measurement. The user operation may include not only these but various operations that can instruct distance measurement.

FIG. 5 is a flowchart showing an example of processing (medical observation method) executed in the medical observation device. Processing related to distance measurement is exemplified. Explanations that overlap with the above will be omitted as appropriate.

In step S1, it is determined whether or not to measure the distance. The CCU 5 determines that distance measurement should be performed, for example, when distance measurement is instructed by the above-described user operation. If distance measurement is to be performed (step S1: Yes), the process proceeds to step S2. Otherwise (step S1: No), the process of step S1 is repeated.

In step S2, the irradiation profile is changed. CCU5 changes the illumination profile. The light from the object 9 also changes, causing a brightness change at the sensor 35 .

In step S3, luminance change is detected. A sensor 35 detects luminance changes. Event data including the position information of the pixel 352 where the luminance change was detected (the event occurred) is sent to the CCU5.

In step S4, distance measurement is performed. The CCU 5 calculates the distance to the object 9 based on the detection result in the previous step S3, more specifically, the event data transmitted from the sensor 35 to the CCU 5. As described above, the results of distance measurement are used to construct three-dimensional distance measurement data of the object 9, generate a three-dimensional map including the data, and the like.

According to the medical observation apparatus 100 described above, the detection of an event by the sensor 35 is (forcibly) generated by changing the light irradiation profile of the light source device 4, and the target 9 is detected using the detection result. The distance to is measured.

As described above, the lens barrel 2 of the endoscope 1 has a fairly small optical system diameter and includes many lenses (lenses 24a to 24g, etc.). Therefore, the light from the object 9 is greatly attenuated by the lens barrel 2 . For example, it is conceivable to incorporate a TOF sensor into an endoscope, but many of the TOF sensors use infrared wavelengths for their wavelengths, so the above-mentioned problem of attenuation becomes apparent. The TOF sensor cannot sufficiently detect the light from the object 9, and it is difficult to obtain accurate distance measurement. Distance measurement by a stereo optical system (3D) is also conceivable, but it is necessary to provide two photographic optical systems in parallel within the lens barrel 2, and the diameter of each photographic optical system becomes narrower. hard to get By using the high-sensitivity sensor 35 as a distance-measuring sensor, as in the medical observation apparatus 100 described above, the light from the object 9 can be detected with high sensitivity, thereby improving the accuracy of distance measurement. can. Sufficient accuracy can be obtained even with the endoscope 1 .

According to the medical observation apparatus 100, the distance to the object 9 can be measured by changing the light irradiation profile of the light source device 4. Therefore, for example, the SfM robot operation disclosed in Patent Document 1 (mechanical motion of the sensor) is not required. In this sense as well, it is suitable for distance measurement with the endoscope 1 . Since the irradiation change can be performed more quickly (for example, instantaneously) than the mechanical operation, it is possible to measure the distance in a short time accordingly.

2. Example of Irradiation Profile The irradiation profile defines the method of light irradiation by the light source device 4 . The illumination profile may define any illumination scheme whose changes can change the light from the object 9 . Some examples of illumination profiles are described.

In one embodiment, the illumination profile may include an illumination range. CCU 5 may cause detection of an event by sensor 35 to measure the distance to object 9 by changing the illumination range.

Examples of the irradiation range are the entire operative field (equivalent to the screen on which the captured image is presented), the central part of the operative field, and the surrounding part of the central part of the operative field. It should be noted that the irradiation of the central portion and the peripheral portion may be performed intensively, and the existence of the irradiation of other portions is not excluded.

The irradiation range may have various shapes. For example, when the irradiation range is the central portion of the screen, the shape of the irradiation range may be circular, elliptical, polygonal, or the like. When the irradiation range is the peripheral portion of the screen, the shape of the irradiation range may be an annular (ring-shaped) circular shape, an elliptical shape, a polygonal shape, or the like.

When the irradiation range has an annular shape, the CCU 5 may change the size of the annular shape. For example, the CCU 5 may change the illumination range such that the annular shape gradually becomes larger and smaller. An example including such changes will be described with reference to FIG.

FIG. 6 is a diagram schematically showing an example of changes in the irradiation range. The irradiation range R1 shown in FIG. 6A is the irradiation range of the central portion of the screen, and has a circular shape in this example. Illuminated ranges R2 to R4 shown in (B) to (D) of FIG. 6 are the illuminated ranges of the peripheral portion of the screen, and in this example, have an annular shape. The annular shape of the irradiation range R3 is larger than the annular shape of the irradiation range R2. The annular shape of the irradiation range R4 is larger than the annular shape of the irradiation range R3. For example, the CCU 5 gradually changes the irradiation range so as to include such irradiation ranges R1 to R4. By changing the irradiation range in the radial direction of the circle in this way, the operative field can be efficiently scanned and the distance can be measured.

A change in the irradiation range such as the irradiation range R1 to the irradiation range R4 described above is realized, for example, by changing the incident angle of the light incident on the optical waveguide from the light source device 4 . This will be described with reference to FIG.

FIG. 7 is a diagram schematically showing an example of the relationship between the incident angle and the irradiation range. The optical waveguides are the cable C14 and the light guide 22 previously described with reference to FIG. The optical waveguide may be, for example, a bundle fiber or a liquid light guide in which a plurality of index guide type multimode optical fibers having a core diameter of about 10 μm to 100 μm are bundled. Such an optical waveguide has the characteristic of radiating light rays from the exit facet while preserving the angle of the light rays incident on the entrance facet.

As shown in the upper part of FIG. 7, when the incident angle from the light source device 4 to the optical waveguide is reduced, the angle of light emitted from the optical waveguide is also reduced. As shown in the lower part of FIG. 7, when the angle of incidence of light from the light source device 4 onto the optical waveguide is increased, the angle of light emitted from the optical waveguide also increases. In an optical waveguide, the incident angle of light rays is preserved, but the incident position of light rays is not preserved. Therefore, light rays that enter at a certain incident angle are emitted from the output end face as ring-shaped rays while maintaining that angle. be.

For example, by controlling the incident angle of light to the optical waveguide as described above, the irradiation range can be changed. The light source device 4 may include various optical elements for controlling the angle of incidence of light on the optical waveguides described above. Examples of optical elements include a collimator lens for obtaining parallel light, a mechanism for adjusting the incident angle (incident angle adjusting mechanism), and an optical system for coupling the light whose incident angle is adjusted to the optical waveguide (coupling optics system), etc. See Patent Document 2, for example, for a more specific configuration.

In one embodiment, the illumination profile may comprise a combination of illumination of the object 9 from different positions of the light of the light source device 4 . CCU 5 may cause detection of an event by sensor 35 and measure the distance to object 9 by changing the combination.

FIG. 8 is a diagram schematically showing a combination of multiple irradiations. FIG. 8 schematically shows the appearance of the front end (opening 2a side) of the lens barrel 2 when viewed from the front. Emission ends 22a of a plurality of light guides 22 are positioned around the opening 2a. The output ends 22a are referred to as output ends 22a-1 to 22a-12 so that the output ends 22a can be distinguished from each other. The light source device 4 is configured to supply light to each of the emission ends 22a-1 to 22a-12 and to individually control each light. An example of control is control of lighting of light, and the CCU 5 may change the lighting pattern of light emitted from the emission ends 22a-1 to 22a-12. For example, the CCU 5 may change the lighting pattern so that the output ends 22a-1 to 22a-12 emit light (light up) in this order (sequentially).

In one embodiment, the CCU 5 may temporarily stop the irradiation (turn off the light) when changing the light irradiation profile of the light source device 4 . A change in the illumination of the object 9 becomes more conspicuous, and an event can be detected more reliably by the sensor 35 .

FIG. 9 is a flowchart showing an example of processing including temporary suspension of irradiation (turning off the light). In step S11, it is determined whether or not to change the irradiation profile. For example, the CCU 5 determines that the irradiation profile should be changed when distance measurement is instructed by a user's operation. If the irradiation profile is to be changed (step S11: Yes), the process proceeds to step S12. Otherwise (step S11: No), the process of step S11 is repeated.

At step S12, the lights are turned off. The CCU 5 transmits to the light source device 4, for example, a command to stop light irradiation. The light source device 4 stops irradiating the object 9 with light.

In step S13, the irradiation profile is changed. CCU5 changes the illumination profile. The light source device 4 is ready to irradiate the object 9 with light as defined by the changed irradiation profile.

At step S14, lighting is performed. The CCU 5 transmits to the light source device 4, for example, a command instructing the start of light irradiation. The light source device 4 starts irradiating the object 9 with light. The object 9 is irradiated with light as defined by the irradiation profile after the change in the previous step S13.

In one embodiment, the illumination profile may include wavelengths of light from the light source device 4 . The CCU 5 may cause the sensor 35 to detect an event by changing the wavelength of light from the light source device 4 and measure the distance to the object 9 . Some objects 9 (for example, internal organs, etc.) exhibit different reflection characteristics depending on the wavelength of the irradiated light. Changing the wavelength of the light illuminating such an object 9 will also change the light from the object 9 and can cause detection of an event by the sensor 35 .

The light source device 4 is configured to be able to change the wavelength of light with which the object 9 is irradiated. An example of the configuration of such a light source device 4 will be described with reference to FIG.

FIG. 10 is a diagram showing an example of a schematic configuration of a light source device. In this example, the light source device 4 includes light sources 41a to 41f, mirrors 42a to 42f, and a lens 42g in addition to the light exit port 43 described above.

The light sources 41a to 41f output lights of different wavelengths (lights of different colors). In this example, the light source 41a outputs infrared light. The light source 41b outputs red light. The light source 41c outputs yellow light. The light source 41d outputs green light. The light source 41e outputs blue light. The light source 41f outputs violet light. The light sources 41a to 41f are individually controllable.

The mirrors 42a to 42f guide the light from the light sources 41a to 41f to the lens 42g. The mirror 42a reflects the light from the light source 41a toward the lens 42g. The mirror 42b reflects the light from the light source 41b toward the lens 42g. Moreover, the mirror 42b transmits the light of the light source 41a reflected by the mirror 42a. The mirror 42c reflects the light from the light source 41c toward the lens 42g. Also, the mirror 42c transmits the light of the light source 41a and the light source 41b reflected by the mirror 42a and the mirror 42b.

The mirror 42d reflects the light from the light source 41d toward the lens 42g. Also, the mirror 42d transmits the lights of the light sources 41a to 41c reflected by the mirrors 42a to 42c. The mirror 42e reflects the light from the light source 41e toward the lens 42g. Also, the mirror 42e transmits the lights of the light sources 41a to 41d reflected by the mirrors 42a to 42d. The mirror 42f reflects the light from the light source 41f toward the lens 42g. The mirror 42f transmits the lights of the light sources 41a to 41e reflected by the mirrors 42a to 42e.

For example, the mirror 42a may be a total reflection mirror. Mirrors 42b through 42f may be dichroic mirrors.

The lens 42g is a condensing lens that collects the light from the light sources 41a to 41f reflected by the mirrors 42a to 42g and guides them to the light exit port 43.

Since the light sources 41a to 41f are individually controllable as described above, the light source device 4 emits arbitrarily selected light from the light sources 41a to 41f (lights with different wavelengths) through the light exit 43. can be emitted from and illuminate the object 9 .

The CCU 5 may change the wavelength of the light from the light source device 4 according to the type of the object 9, such as the type and composition of the organ. A wavelength change is selected that causes a change in the light from the object 9 . To specify the type of the object 9, image recognition or the like for a captured image (for example, an image based on an image signal generated by the sensor 33 or the sensor 34) may be used. Various known image recognition algorithms, image recognition models (learned models), etc. may be used for image recognition.

The medical observation device 100 (the endoscope 1, the light source device 4, and the CCU 5) described above may be incorporated and used in an endoscopic surgery system including an endoscopic robot, for example. Such embodiments are described. In the following description, the endoscope robot will be described as an arm or the like that supports an endoscope.

3. Embodiment of Endoscopic Surgery System FIG. 11 is a diagram showing an example of a schematic configuration of an endoscopic surgery system. The illustrated endoscope 5001 (lens barrel 5003, camera head 5005), light source device 5043, and CCU 5039 are the endoscope 1 (lens barrel 2, camera head 3) and light source device of the medical observation apparatus 100 described so far. 4 and CCU 5 configuration and functions. Duplicate descriptions are omitted as appropriate.

FIG. 11 shows a surgeon 5067 performing surgery on a patient 5071 on a patient bed 5069 using the endoscopic surgery system 5000 . As shown in FIG. 11, an endoscopic surgery system 5000 includes an endoscope 5001, other surgical instruments (medical instruments) 5017, and a support arm device (medical arm) 5027 for supporting the endoscope 5001. , and a cart 5037 loaded with various devices for endoscopic surgery. Details of the endoscopic surgery system 5000 will be sequentially described below.

[Surgical tools]
In endoscopic surgery, instead of cutting the abdominal wall to open the abdomen, for example, a plurality of tubular opening instruments called trocars 5025a to 5025d are punctured into the abdominal wall. Then, the barrel 5003 of the endoscope 5001 and other surgical instruments 5017 are inserted into the body cavity (abdominal cavity) of the patient 5071 from the trocars 5025a to 5025d. In the example shown in FIG. 11 , a pneumoperitoneum tube 5019 , an energy treatment instrument 5021 and forceps 5023 are inserted into the body cavity of a patient 5071 as other surgical instruments 5017 . Also, the energy treatment tool 5021 is a treatment tool that performs tissue incision and ablation, blood vessel sealing, or the like, using high-frequency current or ultrasonic vibration. However, the surgical tool 5017 shown in FIG. 11 is merely an example, and examples of the surgical tool 5017 include various surgical tools generally used in endoscopic surgery, such as pestle, retractor, and the like.

[Support arm device]
The support arm device 5027 has an arm portion 5031 extending from the base portion 5029 . In the example shown in FIG. 11, the arm section 5031 is composed of joint sections 5033a, 5033b, and 5033c and links 5035a and 5035b, and is driven under the control of the arm control device 5045. The arm portion 5031 supports the endoscope 5001 and controls the position and attitude of the endoscope 5001 . As a result, stable position fixation of the endoscope 5001 can be realized.

[Various devices mounted on the cart]
First, the display device 5041 displays an image based on an image signal subjected to image processing by the CCU 5039 under the control of the CCU 5039 . When the endoscope 5001 is compatible with high-resolution imaging such as 4K (horizontal pixel number 3840×vertical pixel number 2160) or 8K (horizontal pixel number 7680×vertical pixel number 4320), and/or If the display device 5041 is compatible with 3D display, a device capable of high-resolution display and/or a device capable of 3D display is used as the display device 5041 . Further, a plurality of display devices 5041 having different resolutions and sizes may be provided depending on the application.

An image of the surgical site within the body cavity of the patient 5071 captured by the endoscope 5001 is displayed on the display device 5041 . The surgeon 5067 can use the energy treatment tool 5021 and the forceps 5023 to perform treatment such as excision of the affected area while viewing the image of the surgical area displayed on the display device 5041 in real time. Although illustration is omitted, the pneumoperitoneum tube 5019, the energy treatment instrument 5021, and the forceps 5023 may be supported by the surgeon 5067, an assistant, or the like during surgery.

The CCU 5039 can comprehensively control the operations of the endoscope 5001 and the display device 5041. Specifically, the CCU 5039 subjects the image signal received from the camera head 5005 to various image processing such as development processing (demosaicing) for displaying an image based on the image signal. Furthermore, the CCU 5039 provides the image signal subjected to the image processing to the display device 5041 . Also, the CCU 5039 transmits a control signal to the camera head 5005 to control its driving. The control signal can include information about imaging conditions such as magnification and focal length. As described above, the CCU 5039 also measures the distance to the object 9 .

The arm control device 5045 is composed of a processor such as a CPU, for example, and operates according to a predetermined program to control the driving of the arm portion 5031 of the support arm device 5027 according to a predetermined control method.

The input device 5047 is an input interface for the endoscopic surgery system 5000. The surgeon 5067 can input various information and instructions to the endoscopic surgery system 5000 via the input device 5047 . For example, the surgeon 5067 inputs various types of information regarding surgery, such as the patient's physical information and information about surgical techniques, via the input device 5047 . Further, for example, the surgeon 5067 gives an instruction to drive the arm unit 5031 via the input device 5047, or an instruction to change the imaging conditions (type of irradiation light, magnification, focal length, etc.) by the endoscope 5001. , an instruction to drive the energy treatment instrument 5021, and the like. The type of the input device 5047 is not limited, and the input device 5047 may be various known input devices. As the input device 5047, for example, a mouse, keyboard, touch panel, switch, footswitch 5057, and/or lever can be applied. For example, when a touch panel is used as the input device 5047 , the touch panel may be provided on the display surface of the display device 5041 .

Alternatively, the input device 5047 may be a device that is attached to a part of the body of the surgeon 5067, such as a glasses-type wearable device or an HMD (Head Mounted Display). In this case, various inputs are performed according to the gestures and line of sight of the surgeon 5067 detected by these devices. Also, the input device 5047 can include a camera capable of detecting the movement of the surgeon 5067, and various inputs are performed according to the gestures and line of sight of the surgeon 5067 detected from the image captured by the camera. may be broken. Furthermore, the input device 5047 can include a microphone capable of picking up the voice of the surgeon 5067, and various voice inputs may be made through the microphone. Since the input device 5047 is configured to be capable of inputting various kinds of information in a non-contact manner, a user belonging to a particularly clean area (for example, a surgeon 5067) can operate a device belonging to an unclean area without contact. becomes possible. In addition, since the surgeon 5067 can operate the device without taking his/her hand off the surgical tool, the convenience of the surgeon 5067 is improved.

The treatment instrument control device 5049 controls driving of the energy treatment instrument 5021 for tissue cauterization, incision, blood vessel sealing, or the like. The pneumoperitoneum device 5051 is inserted into the body cavity through the pneumoperitoneum tube 5019 in order to inflate the body cavity of the patient 5071 for the purpose of securing the visual field of the endoscope 5001 and securing the working space of the surgeon 5067 . send gas. The recorder 5053 is a device capable of recording various types of information regarding surgery. The printer 5055 is a device capable of printing various types of information regarding surgery in various formats such as text, images, and graphs.

[Support arm device (detailed configuration)]
Furthermore, an example of the detailed configuration of the support arm device 5027 will be described. The support arm device 5027 has a base portion 5029 as a base and an arm portion 5031 extending from the base portion 5029 . In the example shown in FIG. 11, the arm portion 5031 is composed of a plurality of joint portions 5033a, 5033b, 5033c and a plurality of links 5035a, 5035b connected by the joint portion 5033b. Therefore, the configuration of the arm portion 5031 is simplified for illustration. Specifically, the shape, number and arrangement of the joints 5033a to 5033c and the links 5035a and 5035b, the direction of the rotation axis of the joints 5033a to 5033c, etc. are appropriately set so that the arm 5031 has a desired degree of freedom. can be For example, the arm portion 5031 may preferably be configured to have 6 or more degrees of freedom. As a result, the endoscope 5001 can be freely moved within the movable range of the arm portion 5031, so that the barrel 5003 of the endoscope 5001 can be inserted into the body cavity of the patient 5071 from a desired direction. be possible.

The joints 5033a to 5033c are provided with actuators, and the joints 5033a to 5033c are configured to be rotatable around a predetermined rotation axis by driving the actuators. By controlling the driving of the actuator by the arm control device 5045, the rotation angles of the joints 5033a to 5033c are controlled, and the driving of the arm 5031 is controlled. Thereby, control of the position and attitude of the endoscope 5001 can be realized. At this time, the arm control device 5045 can control the driving of the arm section 5031 by various known control methods such as force control or position control.

For example, when the surgeon 5067 appropriately performs an operation input via the input device 5047 (including the foot switch 5057), the arm control device 5045 appropriately controls the driving of the arm section 5031 according to the operation input. The position and orientation of the scope 5001 may be controlled. Note that the arm portion 5031 may be operated by a so-called master-slave method. In this case, the arm section 5031 (slave) can be remotely controlled by the surgeon 5067 via the input device 5047 (master console) installed at a location remote from or within the operating room.

Here, in general, in endoscopic surgery, the endoscope 5001 was supported by a doctor called a scopist. In contrast, in the embodiment of the present disclosure, the use of the support arm device 5027 makes it possible to more reliably fix the position of the endoscope 5001 without manual intervention, so that the image of the surgical site is can be stably obtained, and the operation can be performed smoothly.

It should be noted that the arm control device 5045 does not necessarily have to be provided on the cart 5037. Also, the arm control device 5045 does not necessarily have to be one device. For example, the arm control device 5045 may be provided at each joint portion 5033a to 5033c of the arm portion 5031 of the support arm device 5027, and the arm portion 5031 is driven by the cooperation of the plurality of arm control devices 5045. Control may be implemented.

For example, by incorporating the distance measurement technique by the medical observation device 100 described above into the endoscopic surgery system 5000 described above, distance measurement of objects such as organs in the body cavity of the patient 5071 and the surgical instrument 5017 can be performed. becomes possible. The distance measurement result can be utilized for autonomous control (autonomous drive) of the arm section 5031 that supports the endoscope 5001, and the like.

It is important to construct a three-dimensional map inside the patient's 5071 body cavity. By referring to the three-dimensional map, the environment around the tip of the endoscope 5001 (peripheral environment) can be grasped, and the endoscope 5001 can avoid collision, excessive proximity, etc. to organs in the abdominal cavity. can be controlled. The three-dimensional map may be generated in advance by CT scanning or the like and held in the endoscopic surgery system 5000 .

In some cases, during surgery, the tip of the endoscope 5001 reaches an area not covered by the previously generated three-dimensional map. In that case, an object 9 not shown on the three-dimensional map, that is, an object 9 for which distance information (such as three-dimensional distance measurement data) is not held may appear around the distal end of the endoscope 5001 . The CCU 5 may measure the distance to the object 9 when it does not have the distance information of the object 9 existing around the distal end of the endoscope 5001 . Measurement method As described above, the description will not be repeated. The CCU 5 may construct 3D range measurement data for the object 9 based on the range measurement results and generate a 3D map containing (the 3D range measurement data for) the object 9 . The generation of this 3D map may be complementary to a previously generated 3D map.

FIG. 12 is a flowchart showing an example of processing executed in the endoscopic surgery system. In step S21, it is determined whether or not surrounding scan information is held. For example, when the target object 9 existing around the distal end of the endoscope 5001 is included in the pre-generated three-dimensional map, the CCU 5 determines that the peripheral scan information is held. If the surrounding scan information is held (step S21: Yes), the process of step S21 is repeated. Otherwise (step S21: No), the process proceeds to step S22.

In step S22, transition to the distance measurement mode. CCU 5039 initiates distance measurement according to the techniques described so far. Specifically, the processes of steps S23 to S25 are executed. These processes are the same as the processes of steps S2 to S4 in FIG. 6 described above. The CCU 5039 changes the light irradiation profile of the light source device 5043 and measures the distance to the object 9 based on the detection result of the sensor 35 generated by the change in the light irradiation profile.

In step S26, three-dimensional distance measurement data is constructed. The CCU 5039 constructs three-dimensional distance measurement data of the object 9 based on the distance measurement result to the object 9 in the previous step S25. The CCU 5039 also complements the pre-generated 3D map to include the constructed 3D distance measurement data of the object 9 .

For example, as described above, it is possible to construct the 3D distance measurement data of the object 9 in an area that has not been grasped so far, and to obtain a 3D map including it.

Also, the CCU 5 may measure the distance to the object 9 when the position of the object 9 existing around the distal end of the endoscope 5001 changes. A change in the position of the object 9 may be detected by image recognition or the like for a captured image (for example, an image based on an image signal generated by the sensor 33 or the sensor 34). For example, when an assistant or the like lifts and moves the liver during surgery, a change in the position of the liver is detected. Based on the distance measurement results, the CCU 5 may construct three-dimensional distance measurement data of the object 9 after the position change and generate a three-dimensional map containing the object 9 . The generation of this 3D map may update a previously generated 3D map.

FIG. 13 is a flowchart showing an example of processing executed in the endoscopic surgery system. In step S31, it is determined whether or not the surrounding circumstances have changed. For example, the CCU 5039 determines that the surrounding situation has changed when detecting a change in the position of the object 9 included in the captured image. If the surrounding situation has changed (step S31: Yes), the process proceeds to step S32. Otherwise (step S31: No), the process of step S31 is repeated.

The processing of steps S32 to S35 is the same as the processing of steps S22 to S25 in FIG. 13 described above. The CCU 5039 transitions to the distance measurement mode, changes the light irradiation profile of the light source device 5043, and measures the distance to the object 9 based on the detection result of the sensor 35 generated by the change in the light irradiation profile.

In step S36, three-dimensional distance measurement data is constructed. The CCU 5039 constructs three-dimensional distance measurement data of the object 9 based on the distance measurement result in the previous step S35. The CCU 5039 also updates the pre-generated 3D map to include the constructed 3D distance measurement data of the object 9 .

For example, as described above, it is possible to construct the 3D distance measurement data of the object 9 after the position has changed, and furthermore obtain a 3D map including it.

In the above, an example of applying the distance measurement technique by the medical observation device 100 to the endoscopic surgery system 5000 has been described. However, the distance measurement technique by the medical observation device 100 may be applied to other surgical systems. Another example of a surgical system is a surgical system that includes an operating microscope.

4. Example of Hardware Configuration FIG. 14 is a diagram showing an example of the hardware configuration of the apparatus. The CCU5, CCU5039, etc. described so far are implemented by, for example, the computer 1000 shown in FIG.

The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input/output interface 1600. Each part of computer 1000 is connected by bus 1050 .

The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400 and controls each section. For example, the CPU 1100 loads programs stored in the ROM 1300 or HDD 1400 into the RAM 1200 and executes processes corresponding to various programs.

The ROM 1300 stores boot programs such as the BIOS (Basic Input Output System) executed by the CPU 1100 when the computer 1000 is started, programs dependent on the hardware of the computer 1000, and the like.

The HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by the CPU 1100 and data used by such programs. Specifically, the HDD 1400 is a recording medium that records programs for the medical observation method and information processing method according to the present disclosure, which are examples of the program data 1450 .

A communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (for example, the Internet). For example, CPU 1100 receives data from another device via communication interface 1500, and transmits data generated by CPU 1100 to another device.

The input/output interface 1600 is an interface for connecting the input/output device 1650 and the computer 1000 . For example, the CPU 1100 receives data from input devices such as a keyboard and mouse via the input/output interface 1600 . The CPU 1100 also transmits data to an output device such as a display, speaker, or printer via the input/output interface 1600 . Also, the input/output interface 1600 may function as a media interface for reading a program or the like recorded on a predetermined computer-readable recording medium. Media include, for example, optical recording media such as DVD (Digital Versatile Disc) PD (Phase change rewritable Disk), magneto-optical recording media such as MO (Magneto-Optical Disk), tape media, magnetic recording media, or semiconductor memory. be.

For example, when the computer 1000 functions as the CCU5 and CCU5039 described above, the CPU 1100 of the computer 1000 implements the functions of the CCU5 and CCU5039 by executing a program for estimating the depth loaded on the RAM 1200. do. Further, HDD 1400 may store programs for executing the processes of CCU5 and CCU5039. Note that CPU 1100 reads program data 1450 from HDD 1400 and executes it, but as another example, the program may be obtained from another device via external network 1550 .

CCU5 and CCU5039 may be applied to a system consisting of multiple devices, such as cloud computing, which assumes connection to a network (or communication between devices).

Each of the above components may be configured using general-purpose members, or may be configured with hardware specialized for the function of each component. Such a configuration can be changed as appropriate according to the technical level of implementation.

5. Example of Effect The technique described above is specified as follows, for example. As described with reference to FIGS. 1 to 10 and the like, the medical observation device 100 includes the light source device 4, the sensor 35, and the control unit (CCU 5). The light source device 4 irradiates the object 9 with light. The sensor 35 detects that the amount of luminance change due to the light incident on each of the plurality of pixels 352 exceeds a predetermined threshold. The controller changes the light irradiation profile of the light source device 4 and measures the distance to the object 9 based on the detection result of the sensor 35 generated by the change in the light irradiation profile.

According to the medical observation apparatus 100 described above, the light irradiation profile of the light source device 4 is changed, and the distance to the object 9 is measured based on the detection result of the sensor 35 generated by the change in the light irradiation profile. be. For example, distance measurement accuracy in surgery such as endoscopic surgery can be improved.

As described with reference to FIGS. 6 and 7, changes in the irradiation profile may include changes in the irradiation range. The illuminated area may comprise an annular shape, and the change in illuminated area may comprise a change in the size of the annular shape. For example, by varying the illumination profile in this way, the distance to the object 9 can be measured.

As described with reference to FIG. 8 and the like, changes in the irradiation profile may include changes in combinations of irradiation of the light from the light source device 4 onto the object 9 from a plurality of different positions. Changes in illumination combinations may include changes in lighting patterns. For example, by varying the illumination profile in this way, the distance to the object 9 can be measured.

As described with reference to FIG. 9 and the like, the control unit may temporarily stop irradiation when changing the irradiation profile. As a result, changes in the illumination of the object 9 become more conspicuous, and detection by the sensor 35 can be more reliably generated.

As described with reference to FIG. 10 and the like, changes in the irradiation profile may include changes in the wavelength of light. For example, by varying the illumination profile in this way, the distance to the object 9 can be measured.

The CCU 5 described with reference to FIGS. 1 to 10 and the like can also be called an information processing device that performs processing related to distance measurement, and such an information processing device is also one of the embodiments. The information processing device changes the light irradiation profile of the light source device 4 that irradiates the object 9 with light, and calculates the distance to the object 9 based on the detection result of the sensor 35 generated by the change in the light irradiation profile. Measure. The sensor 35 detects that the amount of luminance change due to the light incident on each of the plurality of pixels exceeds a predetermined threshold. Such an information processing apparatus can also improve the accuracy of distance measurement in surgery, as described above.

The medical observation method described with reference to FIG. 5 etc. is also one of the embodiments. In the medical observation method, the information processing unit (CCU 5) changes the light irradiation profile of the light source device 4 that irradiates the object 9 with light, and based on the detection result of the sensor 35 generated by the change in the light irradiation profile. and measuring the distance to the object 9 (steps S2 to S4). The sensor 35 detects that the amount of luminance change due to the light incident on each of the plurality of pixels 352 exceeds a predetermined threshold. Such a medical observation method can also improve the accuracy of distance measurement in surgery, as described above.

The endoscopic surgery system 5000 described with reference to FIGS. 11 to 13 and the like is also one of the embodiments. The endoscopic surgery system 5000 includes an endoscope 5001 (corresponding to the endoscope 1) including a sensor 35 for detecting that the luminance change amount due to light incident on each of a plurality of pixels 352 exceeds a predetermined threshold, An arm portion 5031 that supports the endoscope 5001, a light source device 5043 (corresponding to the light source device 4) that irradiates the object 9 with light, and a control device that controls the endoscope 5001, the arm portion 5031 and the light source device 5043 CCU 5039). The control device (CCU 5039) changes the light irradiation profile of the light source device 5043, and measures the distance to the object 9 based on the detection result of the sensor 35 generated by the change in the light irradiation profile. Such an endoscopic surgery system 5000 can also improve the accuracy of distance measurement in surgery as described above.

The control device (CCU 5039) may measure the distance when it does not hold the distance information of the object 9 existing around the tip of the endoscope 5001. The control unit (CCU 5039) may measure the distance when the position of the object 9 existing around the distal end of the endoscope 5001 changes. The control unit (CCU 5039) may generate a three-dimensional map containing the object 9 based on the distance measurement results. As a result, distance information of the object 9, which has not been grasped so far, can be grasped (construction of three-dimensional distance measurement data, etc.), and a three-dimensional map including it can be obtained. The control device (CCU 5039) may control the driving of the arm section 5031 based on the generated three-dimensional map including the object 9. FIG. More appropriate control of the arm section 5031 based on the latest three-dimensional map becomes possible.

It should be noted that the effects described in this disclosure are merely examples and are not limited to the disclosed content. There may be other effects.

Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the embodiments described above, and various modifications are possible without departing from the gist of the present disclosure. Moreover, you may combine the component over different embodiment and modifications suitably.

Note that the present technology can also take the following configuration.
(1)
a light source device for irradiating an object with light;
a sensor that detects that a luminance change amount due to light incident on each of the plurality of pixels exceeds a predetermined threshold;
a control unit that changes the light irradiation profile of the light source device and measures the distance to the object based on the detection result of the sensor generated by the change in the light irradiation profile;
comprising a
Medical observation device.
(2)
The change in the irradiation profile includes a change in irradiation range.
The medical observation device according to (1).
(3)
The irradiation range includes an annular shape,
The change in the irradiation range includes a change in the size of the annular shape.
The medical observation device according to (2).
(4)
A change in the irradiation profile includes a change in combination of irradiation of the object from a plurality of different positions of the light of the light source device.
A medical observation device according to any one of (1) to (3).
(5)
The change in the irradiation combination includes a lighting pattern,
The medical observation device according to (4).
(6)
wherein the control unit temporarily stops the irradiation when changing the irradiation profile;
The medical observation device according to any one of (1) to (5).
(7)
the change in the irradiation profile includes a change in the wavelength of the light;
A medical observation device according to any one of (1) to (6).
(8)
changing a light irradiation profile of a light source device that irradiates light onto an object, and measuring a distance to the object based on a sensor detection result generated by the change in the light irradiation profile;
An information processing device,
The sensor detects that a luminance change amount due to light incident on each of the plurality of pixels exceeds a predetermined threshold.
Information processing equipment.
(9)
An information processing device changes a light irradiation profile of a light source device that irradiates light onto an object, and measures a distance to the object based on a sensor detection result generated by the change in the light irradiation profile. thing,
including
The sensor detects that a luminance change amount due to light incident on each of the plurality of pixels exceeds a predetermined threshold.
Medical observation method.
(10)
an endoscope including a sensor that detects that a luminance change due to light incident on each of a plurality of pixels exceeds a predetermined threshold;
an arm that supports the endoscope;
a light source device for irradiating an object with light;
a control device that controls the endoscope, the arm portion, and the light source device;
with
The control device changes the light irradiation profile of the light source device, and measures the distance to the object based on the detection result of the sensor generated by the change in the light irradiation profile.
Endoscopic surgery system.
(11)
The control device measures the distance when the distance information of the object existing around the tip of the endoscope is not held.
The endoscopic surgery system according to (10).
(12)
The control device measures the distance when the position of the object existing around the tip of the endoscope changes.
The endoscopic surgery system according to (10) or (11).
(13)
The control device generates a three-dimensional map including the object based on the distance measurement result.
The endoscopic surgery system according to (11) or (12).
(14)
The endoscopic surgery system according to (13), wherein the control device controls driving of the arm based on the generated three-dimensional map including the object.
(15)
wherein the light source device irradiates the object with light from a predetermined position and a predetermined direction;
A medical observation device according to any one of (1) to (7).
(16)
The light emitted from the light source device is applied to the object from a predetermined position and in a predetermined direction.
The information processing device according to (8).
(17)
The light emitted from the light source device is applied to the object from a predetermined position and in a predetermined direction.
The medical observation method according to (9).
(18)
wherein the light source device irradiates the object with light from a predetermined position and a predetermined direction;
The endoscopic surgery system according to any one of (10) to (14).

1 endoscope 2 lens barrel 3 camera head 33 sensor (imaging sensor)
34 sensor (imaging sensor)
35 sensor (brightness change detection sensor)
4 light source device 41 light source 5 CCU (control unit, information processing device)
100 medical observation device 5000 endoscopic surgery system 5001 endoscope 5031 arm unit 5039 CCU (control unit, information processing unit)
5043 light source device

Claims (14)

1. a light source device for irradiating an object with light;
   a sensor that detects that a luminance change amount due to light incident on each of the plurality of pixels exceeds a predetermined threshold;
   a control unit that changes the light irradiation profile of the light source device and measures the distance to the object based on the detection result of the sensor generated by the change in the light irradiation profile;
   comprising a
   Medical observation device.
2. The change in the irradiation profile includes a change in irradiation range.
   The medical observation device according to claim 1.
3. The irradiation range includes an annular shape,
   The change in the irradiation range includes a change in the size of the annular shape.
   The medical observation device according to claim 2.
4. A change in the irradiation profile includes a change in combination of irradiation of the object from a plurality of different positions of the light of the light source device.
   The medical observation device according to claim 1.
5. The change in the irradiation combination includes a lighting pattern,
   The medical observation device according to claim 4.
6. wherein the control unit temporarily stops the irradiation when changing the irradiation profile;
   The medical observation device according to claim 1.
7. the change in the irradiation profile includes a change in the wavelength of the light;
   The medical observation device according to claim 1.
8. changing the light irradiation profile of a light source device that irradiates light onto an object, and measuring the distance to the object based on the sensor detection result generated by the change in the light irradiation profile;
   An information processing device,
   The sensor detects that a luminance change amount due to light incident on each of the plurality of pixels exceeds a predetermined threshold.
   Information processing equipment.
9. An information processing device changes a light irradiation profile of a light source device that irradiates light onto an object, and measures a distance to the object based on a sensor detection result generated by the change in the light irradiation profile. thing,
   including
   The sensor detects that a luminance change amount due to light incident on each of the plurality of pixels exceeds a predetermined threshold.
   Medical observation method.
10. an endoscope including a sensor that detects that a luminance change due to light incident on each of a plurality of pixels exceeds a predetermined threshold;
    an arm that supports the endoscope;
    a light source device for irradiating an object with light;
    a control device that controls the endoscope, the arm portion, and the light source device;
    with
    The control device changes the light irradiation profile of the light source device, and measures the distance to the object based on the detection result of the sensor generated by the change in the light irradiation profile.
    Endoscopic surgery system.
11. The control device measures the distance when the distance information of the object existing around the tip of the endoscope is not held.
    The endoscopic surgery system according to claim 10.
12. The control device measures the distance when the position of the object existing around the tip of the endoscope changes.
    The endoscopic surgery system according to claim 10.
13. The control device generates a three-dimensional map including the object based on the distance measurement result.
    The endoscopic surgery system according to claim 11.
14. The endoscopic surgery system according to claim 13, wherein the control device controls driving of the arm based on the generated three-dimensional map including the object.

Images