JP2019005266A - Information processing device, program, method and system - Google Patents

Abstract

To provide an information processing device, program, method and system for analyzing a state of blood in a blood vessel in particular in a biological tissue.SOLUTION: The information processing device includes: a memory for storing a prescribed instruction command and storing an image including a blood vessel of a biological tissue imaged by a probe; and a processor configured so as to respectively generate an index showing a state of blood in the blood vessel in one or more coordinate positions in the image and execute the instruction command stored in the memory in order to be able to output each generated index in association with each of the coordinate positions.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to an information processing apparatus for analyzing a state of a living tissue, a program and method executed by the information processing apparatus, and a system using the information processing apparatus.

  2. Description of the Related Art Conventionally, in order to analyze a state of a living tissue, an apparatus that detects light reflected from the living tissue by irradiating the living tissue with light is known. For example, Patent Document 1 describes an apparatus for irradiating a living tissue with excitation light from a probe connected to an excitation light source and detecting the intensity of fluorescence emitted from the excited living tissue. .

JP 2003-220033 A

  In view of the above-described techniques, the present disclosure provides an information processing apparatus, a program, a method, and a system for analyzing a state of blood in a blood vessel, in particular, a blood vessel.

  According to one aspect of the present disclosure, “a memory in which a predetermined instruction command is stored and an image including a blood vessel of a living tissue imaged by a probe; and one or a plurality of coordinate positions in the image In order to generate an index indicating the state of blood in the blood vessel in each and output the generated index in association with each of the coordinate positions, the instruction command stored in the memory is executed An information processing apparatus including a configured processor ”is provided.

  According to one aspect of the present disclosure, “a computer including a memory in which an image including a blood vessel of a biological tissue photographed by a probe is stored is stored in a state of blood in the blood vessel at one or a plurality of coordinate positions in the image. Is generated, and a program configured to function as a processor configured to execute processing for outputting the generated indices in association with the coordinate positions is provided.

  According to one aspect of the present disclosure, “a method performed by a processor executing a predetermined instruction stored in a memory, the step including storing an image including a blood vessel of a living tissue imaged by a probe; Each of generating an index indicating the state of blood in the blood vessel at one or a plurality of coordinate positions in the image, and outputting each generated index in association with each of the coordinate positions. A method of including "is provided.

  According to one aspect of the present disclosure, “of an information processing device, a light source that is communicably connected to the information processing device and capable of emitting a plurality of lights having different peak wavelength ranges, and light emitted from the light source” And a probe that includes an image sensor that detects light reflected from the surface of the biological tissue.

  According to various embodiments of the present disclosure, it is possible to provide an information processing apparatus, a program, a method, and a system for analyzing a state of blood in a blood vessel, in particular, a blood vessel.

  In addition, the said effect is only an illustration for the convenience of description, and is not restrictive. In addition to the above effect or instead of the above effect, any effect described in the present disclosure or an effect apparent to those skilled in the art can be achieved.

FIG. 1 is a diagram for describing a configuration of a system 1 according to an embodiment of the present disclosure. FIG. 2 is a block diagram illustrating an exemplary configuration of the information processing apparatus 100, the probe 200, and the light source control apparatus 300 that configure the system 1 according to an embodiment of the present disclosure. FIG. 3 a is a conceptual diagram illustrating a cross-sectional configuration of the probe 200 according to an embodiment of the present disclosure. FIG. 3 b is a conceptual diagram illustrating a configuration of the bottom surface of the probe 200 according to an embodiment of the present disclosure. FIG. 4 is a conceptual diagram illustrating a usage pattern of the probe 200 according to an embodiment of the present disclosure. FIG. 5 is a diagram illustrating a processing flow executed in the system 1 according to an embodiment of the present disclosure. FIG. 6 is a diagram illustrating a processing flow executed in the information processing apparatus 100 according to an embodiment of the present disclosure. FIG. 7 a is a diagram illustrating an example of an image captured through the probe 200 according to an embodiment of the present disclosure. FIG. 7B is a diagram illustrating an example of an image processed in the information processing apparatus 100 according to an embodiment of the present disclosure. FIG. 7c is a diagram illustrating an example of an image processed in the information processing apparatus 100 according to an embodiment of the present disclosure. FIG. 7d is a diagram illustrating an example of an image processed in the information processing apparatus 100 according to an embodiment of the present disclosure. FIG. 7E is a diagram illustrating an example of an image processed in the information processing apparatus 100 according to the embodiment of the present disclosure. FIG. 7F is a diagram illustrating an example of an image processed in the information processing apparatus 100 according to an embodiment of the present disclosure. FIG. 8 is a diagram illustrating a processing flow executed in the information processing apparatus 100 according to an embodiment of the present disclosure. FIG. 9 is a diagram illustrating an example of an image output from the information processing apparatus 100 according to an embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of an image output from the information processing apparatus 100 according to an embodiment of the present disclosure.

  Various embodiments of the present disclosure will be described with reference to the accompanying drawings. In addition, the same referential mark is attached | subjected to the common component in drawing.

1. Overview of System According to Present Disclosure A system according to various embodiments of the present disclosure includes, as an example, a blood vessel such as a fine artery (eg, an arteriole, a capillary vessel, or a venule) of a living tissue (eg, an organ) using a probe. ) And based on the image including the imaged blood vessel, an index indicating the state of blood (for example, blood oxygen saturation) included in the blood vessel at one or more coordinate positions in the image is generated. The generated index is associated with each of the coordinate positions and output.

  As a specific example, a capillary is imaged on the surface of a human biological tissue with a probe. Then, the captured image is transferred from the probe to the information processing apparatus, and various image processing and image analysis are performed in the information processing apparatus to estimate the oxygen saturation of blood at one or a plurality of coordinate positions in the image. Then, an index generated by estimating the oxygen saturation is arranged (mapped) so as to be superimposed on the coordinate position on the captured image, and the result is displayed on the display or the like of the information processing apparatus.

  The index indicating the state of blood in the blood vessel may be any index that can be acquired from an image captured using a probe, but preferred examples include oxygen saturation of blood, The total concentration of hemoglobin, or a combination thereof.

  In addition, the index indicating the state of blood in the blood vessel may be, for example, a calculated or estimated value of oxygen saturation or the total concentration itself, or may be obtained by classifying the numerical value for each predetermined range. Good. That is, the index does not mean only the numerical value of the calculated or estimated oxygen saturation or total concentration, but may be information processed based on the numerical value.

  In addition, when outputting to a display or the like, the image to which the generated index is mapped may be the image captured by the probe itself, or image processing such as smoothing, binarization, and normalization is performed. It may be a later image.

2. Configuration of System 1 According to One Embodiment FIG. 1 is a diagram for describing a system according to an embodiment of the present disclosure. Referring to FIG. 1, a system 1 includes a probe 200 for capturing an image of a biological tissue, an information processing apparatus 100 that performs processing of the captured image, and a light source that controls a light source included in the probe 200. The control device 300 is connected to each other so that various information, instruction commands, data, and the like can be transmitted and received. Of these, the probe 200 is used in contact with the surface of a living tissue, thereby enabling dark field imaging instead of general bright field imaging.

  Although the light source control device 300 is provided in FIG. 1, the light source control device 300 may be omitted by controlling the light source of the probe 200 by the information processing device 100 or the microcomputer in the probe 200. Is possible. In addition, although the information processing apparatus 100 is described as one component, the information processing apparatuses can be distributed for each of various processes and stored information.

  FIG. 2 is a block diagram illustrating an exemplary configuration of the information processing apparatus 100, the probe 200, and the light source control apparatus 300 that configure the system 1 according to an embodiment of the present disclosure. Note that the information processing apparatus 100, the probe 200, and the light source control apparatus 300 do not have to include all of the components illustrated in FIG. 2, and may have a configuration in which some of the components are omitted. It is also possible to add.

  Referring to FIG. 2, the information processing apparatus 100 includes a display 111, a processor 112, a touch panel 114 and an input interface 113 including a hard key 115, a communication processing circuit 116, a memory 117, and an I / O circuit 118. These components are electrically connected to each other via a control line or a data line.

  The display 111 functions as a display unit that reads out image information stored in the memory 117 and performs various outputs in accordance with instructions from the processor 112. Specifically, the display 111 displays various setting screens for displaying an image in which an index indicating the state of blood in the blood vessel generated by the processor 112 is mapped on the blood vessel image or generating a mapping image. Or display an image of the generation process. The display 111 is composed of a liquid crystal display, for example.

  The processor 112 is constituted by a CPU (microcomputer) as an example, and functions as a control unit for executing an instruction command (program) stored in the memory 117 and controlling other connected components. . For example, the processor 112 executes various image analysis programs stored in the memory 117, and displays an index indicating the state of blood in the blood vessel at one or more coordinate positions of the image including the blood vessel imaged by the probe 200. Or the generated index is arranged on one or a plurality of coordinate positions of the captured image and displayed on the display 111. The processor 112 may be composed of a single CPU, but may be composed of a plurality of CPUs. Also, other types of processors such as GPUs specialized for image processing may be combined as appropriate.

  The input interface 113 includes a touch panel 114 and / or hard keys 115 and functions as an operation unit that receives various instructions and inputs from the user. The touch panel 114 is arranged so as to cover the display 111, and outputs position coordinate information corresponding to the image data displayed on the display 111 to the processor 112. As the touch panel system, a known system such as a resistive film system, a capacitive coupling system, and an ultrasonic surface acoustic wave system can be used.

  The communication processing circuit 116 performs processing such as modulation and demodulation in order to transmit / receive information to / from a remotely installed server device and other information processing devices via a connected antenna (not shown). Do. For example, the communication processing circuit 116 performs processing for transmitting a mapping image obtained as a result of executing the program according to the present embodiment to a server device or another information processing device. The communication processing circuit 116 is processed based on a wideband wireless communication system typified by a W-CDMA (Wideband-Code Division Multiple Access) system, but a wireless LAN as typified by IEEE802.11. It is also possible to perform processing based on a method related to narrowband wireless communication such as Bluetooth or Bluetooth (registered trademark). The communication processing circuit 116 can also use known wired communication.

  The memory 117 includes a ROM, a RAM, a nonvolatile memory, an HDD, and the like, and functions as a storage unit. The ROM stores an instruction command for executing image processing and the like according to the present embodiment and a predetermined OS as a program. The RAM is a memory used for writing and reading data while the program stored in the ROM is being processed by the processor 112. The nonvolatile memory and the HDD are memories in which data is written and read by executing the program, and the data written therein is stored even after the execution of the program is completed. As an example, the non-volatile memory and the HDD store images such as an image captured by the probe 200, an image such as a mapping image, and information on a user who is an object captured by the probe 200.

  The I / O circuit 118 is connected to an I / O circuit included in each of the probe 200 and the light source control device 300, and serves as an information input / output unit for inputting / outputting information between the probe 200 and the light source control device 300. Function. Specifically, the I / O circuit 118 functions as an interface for receiving an image captured by the probe 200 and transmitting a control signal for controlling the image sensor 212 included in the probe 200. . The I / O circuit 118 can employ a known connection format such as a serial port, a parallel port, or a USB as desired.

  Referring to FIG. 2, the probe 200 includes a light source 211, an image sensor 212, and an I / O circuit 213. These components are electrically connected to each other via a control line or a data line.

  The light source 211 is composed of at least one LED. As an example, the light source 211 is an LED for emitting blue light having a peak wavelength of 470 nm and a half-value width of 30 nm, and an LED for emitting green light having a peak wavelength of 527 nm and a half-value width of 30 nm to a living tissue or blood vessel. It is composed of a plurality of light sources having different peak wavelengths. The light emission color of the light source is preferably only included in the wavelength range of 400 nm to 600 nm, which is the wavelength region where the light absorption of hemoglobin contained in blood is dominant, and only the specific emission color described above. It is not limited to. Moreover, although two types of light sources of blue light and green light have been described, it is also possible to provide additional light sources having different peak wavelength ranges. When a light source (for example, red light) having no peak wavelength in the light absorption wavelength region is used, the difference in light absorption between the blood vessel portion and its peripheral portion is reduced. On the other hand, when a light source having a peak wavelength region in the light absorption wavelength region of hemoglobin is used, the difference in light absorption between the blood vessel portion and its peripheral portion is sufficiently large. Therefore, the difference in pixel value between the blood vessel portion and the peripheral portion (background portion) in the image captured by the probe 200 becomes clearer, and the blood vessel portion can be preferably extracted.

  Although not shown, the light source 211 has a known switching circuit for periodically switching its emission color (peak wavelength) based on a control signal received from the processor 311 of the light source control device 300. May be.

  The image sensor 212 detects light that is scattered in the biological tissue and reflected from the surface of the biological tissue, images the imaging target, and outputs an image signal output to the information processing apparatus 100 via the I / O circuit 213. Generate. As the image sensor 212, a known image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor can be used. The generated image signal is processed by each circuit such as a CDS circuit, an AGC circuit, and an A / D converter, and then transmitted to the information processing apparatus 100 as a digital image signal.

  The I / O circuit 213 is connected to an I / O circuit included in each of the information processing apparatus 100 and the light source control apparatus 300, and information for inputting / outputting information between the information processing apparatus 100 and the light source control apparatus 300 Functions as an input / output unit. Specifically, the I / O circuit 213 receives a digital image signal generated by the image sensor 212 or the like in the information processing apparatus 100, or receives a control signal for controlling the light source 211 and the image sensor 212. 100 and the light source control device 300 function as an interface for receiving. The I / O circuit 213 can adopt a known connection format such as a serial port, a parallel port, or a USB as desired.

  Referring to FIG. 2, the light source control device 300 includes a processor 311, a memory 312, an input interface 313, and an I / O circuit 314. These components are electrically connected to each other via a control line or a data line.

  The processor 311 is configured by a CPU (microcomputer) as an example, and controls for controlling other connected components in order to execute instruction commands (for example, various programs) stored in the memory 312. It functions as a part. For example, the processor 311 executes a light source control program stored in the memory 312 and outputs a control signal for periodically switching the color of light output from the light source 211 provided in the probe 200. The processor 311 may be composed of a single CPU, but may be composed of a plurality of CPUs.

  The memory 312 includes a ROM, a RAM, a nonvolatile memory, an HDD, and the like, and functions as a storage unit. The ROM stores an instruction command for executing control of the light source according to the present embodiment and a predetermined OS as a program. The RAM is a memory used for writing and reading data while the program stored in the ROM is processed by the processor 311. The nonvolatile memory and the HDD are memories in which data is written and read by executing the program, and the data written therein is stored even after the execution of the program is completed. For example, setting information such as a peak wavelength of the light source and a light emission period of light emitted from the light source (or a switching period when a plurality of light emission colors are used) is stored in the nonvolatile memory or the HDD.

  The input interface 313 is configured by a hard key or the like, and functions as an operation unit that receives various setting information of the light source by the user.

  The I / O circuit 314 is connected to an I / O circuit included in each of the information processing apparatus 100 and the probe 200, and serves as an information input / output unit for inputting / outputting information between the information processing apparatus 100 and the probe 200. Function. Specifically, the I / O circuit 314 functions as an interface for transmitting a control signal for controlling the light source 211 of the probe 200 to the probe 200. The I / O circuit 314 can adopt a known connection format such as a serial port, a parallel port, or a USB as desired.

3. Configuration of Probe 200 FIG. 3A is a conceptual diagram illustrating a cross-sectional configuration of the probe 200 according to an embodiment of the present disclosure. FIG. 3B is a conceptual diagram illustrating a configuration of the bottom surface of the probe 200 according to an embodiment of the present disclosure. Referring to FIGS. 3a and 3b, the probe 200 passes between the contact surface 225 that contacts the surface of the living tissue and transmits light to the image sensor 221 included in the camera. An optical path 224 disposed between them is provided. In the optical path 224, a lens 233, an optical filter, or the like can be arranged in accordance with desired image data or the arrangement of the image sensor 221.

  In addition, the probe 200 includes a plurality of LEDs 222 as light sources around the optical path 224 and a separation wall 232 configured to physically separate the optical path 224 and the LED 222 around the optical path 224. The LED 222 is optically completely separated from the optical path 224 leading to the image sensor 221 by the separation wall 232 in order to photograph a living tissue by a dark field photographing method (specifically, a side stream dark field photographing method). Arranged. Specifically, the LED 222 is installed such that the optical axis of the light applied to the living tissue that is the subject is inclined by a predetermined angle (for example, about 50 degrees) with respect to the optical axis of the light passing through the optical path 224. Is done. Since the light emitted from the LED 222 has directivity, not only can the LED 222 be optically completely separated from the optical path 224, but also the intensity of the light emitted to the living tissue as a subject is increased. It becomes possible. In the example of FIGS. 3 a and 3 b, the probe 200 includes six multi-color LEDs 222 around the optical path 224. Thus, by arranging the plurality of LEDs 222 evenly, it is possible to irradiate the subject with light uniformly.

  In the example of FIGS. 3a and 3b, a plurality of light colors (for example, blue light and green light) are switched and used in a predetermined cycle in response to a control signal from the light source control device 300 using a multi-color LED. As an example of the switching cycle, 500 msec, preferably 200 msec, more preferably 100 msec can be mentioned.

  However, the present invention is not limited to this, and it is also possible to arrange a plurality of types of light sources having different emission colors in advance. Further, in the example of FIGS. 3a and 3b, six LEDs 222 are used, but the number thereof can be increased or decreased as desired. For example, only one LED or eight LEDs can be used. .

  The probe 200 includes a cover 223 on the contact surface with the living tissue so as to cover the LED 222. The cover 223 is made of, for example, a silicone resin, and prevents the LED 222 from being directly contacted with a living tissue or a secretion thereof to be contaminated.

  FIG. 4 is a conceptual diagram illustrating a usage pattern of the probe 200 according to an embodiment of the present disclosure. In the present embodiment, the image captured by the probe 200 is an image captured by a dark field imaging method. Therefore, the light emitted from the LED 222 needs to be optically separated from the optical path 224. Therefore, the image is captured in a state where the contact surface 225 and the cover surface 226 of the probe 200 are in contact with the surface 231 of the living tissue 228.

  Specifically, referring to FIG. 4, light (for example, blue light or green light) emitted from the LED 222 passes through the cover surface 226 and the surface 231 of the living tissue 228 and enters the living tissue 228. The incident light 227 is scattered in the living tissue like the light 227a. At this time, since the incident light 227 has a peak wavelength in the absorption wavelength region of hemoglobin of red blood cells, a portion of the scattered light 227a (for example, light 227b) is hemoglobin 230 of red blood cells included in the capillary 229 near the surface 231. To be absorbed. On the other hand, a part of the light (for example, light 227c) not absorbed by the red blood cell hemoglobin 230 passes through the surface 231 of the living tissue 228 and the contact surface 225 of the probe 200, enters the optical path 224, and finally It reaches the image sensor 221 and is imaged by the image sensor 221.

  Thus, in this embodiment, since the probe 200 is used by contacting the contact surface 225 and the cover surface 226 of the probe 200 with the surface 231 of the living tissue 228 as described above, the probe 200 is used on the surface 231 of the living tissue 228. The reflection of light can be reduced. In addition, since the dark field imaging method is used, clearer imaging of the capillary 229 is possible.

4). Overview of Processing Performed by Information Processing Device 100 FIG. 5 is a diagram illustrating a processing flow performed in the system 1 according to an embodiment of the present disclosure. Specifically, FIG. 5 is a diagram illustrating a processing flow performed when the processor 112 of the information processing apparatus 100, the processor 311 of the light source control apparatus 300, and the like execute instruction commands stored in the memories 117 and 312. It is.

  According to FIG. 5, the probe 200 receives the control signal from the processor 311 and the control signal for photographing from the processor 112 of the information processing apparatus 100 by setting the LED 222 as the light source in the light source control apparatus 300. Thus, the processing flow is started. First, the probe 200 that has received the control signal irradiates light on a living tissue to be imaged by controlling the peak wavelength of light emitted from the LED 222 and its switching cycle. The probe 200 detects the scattered light received by the image sensor 212 and captures an image including a blood vessel of the living tissue (S101). At this time, as described above, the blue light and the green light are switched at a predetermined switching cycle, and the image sensor 212 also detects the blue light and the green light by separating them. Therefore, in the imaging process, two spectral images, that is, a spectral image by blue light and a spectral image by green light are obtained.

  As an example, the depth of focus is 5.6 mm, the color switching period of the LED 222 is 100 msec, the frame rate is 30 fps, and an image of 640 × 640 pixels is generated.

  Next, each captured spectral image is transmitted to the information processing apparatus 100 via the I / O circuit 213 of the probe 200 and the I / O circuit 118 of the information processing apparatus 100. Each spectral image is stored in the memory 117 under the control of the processor 112. Then, the processor 112 reads each spectral image and an instruction command (program) for processing from the memory 117, and executes a blood vessel region extraction process in each spectral image (S102). As an example, the blood vessel extraction processing can be used by appropriately combining, for each spectral image, tubular structure extraction processing based on the Hessian matrix, binarization processing, analysis processing based on pixel values, and the like.

  Next, when the coordinate position of the blood vessel included in the image is specified by the blood vessel extraction process, the processor 112 performs optical measurement at one or more coordinate positions indicating the blood vessel based on the instruction command stored in the memory 117. Processing for calculating the density is executed (S103). For example, the optical density is calculated based on the pixel value of the portion extracted as a blood vessel and the average pixel value of the surrounding background portion.

  Next, the processor 112 executes processing for generating an index indicating the oxygen saturation level of blood at one or more coordinate positions based on the instruction command stored in the memory 117 (S104). The oxygen saturation calculation process is performed using the calculated optical density, the molar absorption coefficient of oxygenated hemoglobin and deoxygenated hemoglobin, and the like.

  Next, when an index indicating oxygen saturation at one or a plurality of coordinate positions indicating a blood vessel in the image is generated by the oxygen saturation calculation process, the processor 112 is based on the instruction command stored in the memory 117. Then, a process of outputting the index in association with each coordinate position is executed (S105). As an example, on the display 111 of the information processing apparatus 100, any one of the spectral images received from the probe 200, or a processed image created based on each spectral image in the process described above, at each coordinate position. Based on this, each index is arranged and output.

  As described above, an index based on the oxygen saturation is generated as an index indicating the state of blood in the blood vessel from the image captured by the probe 200, and a series of processes until the index is output to the display 111 is completed. Details of each process will be described later.

5). The blood vessel extraction processing FIG. 6 is a diagram showing a processing flow executed in the information processing apparatus 100 according to an embodiment of the present disclosure. Specifically, FIG. 6 is a diagram illustrating a processing flow performed when the processor 112 of the information processing apparatus 100 executes an instruction command stored in the memory 117.

  First, the processor 112 receives the image including the blood vessel of the living tissue imaged by the probe 200 in the I / O circuit 118 of the information processing apparatus 100, and stores the I / O circuit 118 and the memory 117 so as to store in the memory 117. Control (S201).

  Here, FIG. 7a is a diagram illustrating an example of an image captured through the probe 200 according to an embodiment of the present disclosure. Specifically, FIG. 7a is an image showing an example of an image (spectral image) including a blood vessel of a living tissue imaged by the probe 200 stored in the memory 117 in S201. Note that, as described above, in the present embodiment, spectral images photographed with two emission colors of blue light and green light are stored. Therefore, although not particularly illustrated, at least two spectral images shown in FIG. 7a are stored.

  Next, returning to FIG. 6, the processor 112 reads each spectral image stored in the memory 117 and executes normalization processing for each pixel by a known method (S202). As an example, a process of expanding the brightness in the image so that the darkest point in the image is “black” and the brightest point is brightened to the maximum is executed. Each image (normalized image) after the processing is stored in the memory 117 at one end.

  FIG. 7B is a diagram illustrating an example of an image processed in the information processing apparatus 100 according to an embodiment of the present disclosure. Specifically, FIG. 7b is a diagram illustrating an example of a normalized process image. As is apparent from comparison with the spectral image of FIG. 7a, by performing the normalization process, it is possible to make the dark part (the part corresponding to the blood vessel in the present embodiment) in the image more prominent with respect to the background. It becomes possible.

  Next, returning to FIG. 6, the processor 112 reads out each normalized processing image from the memory 117, and analyzes the image by the Hessian matrix for each pixel, so that a tubular structure (that is, a structure corresponding to a blood vessel) is obtained. The process for extracting is executed (S203). In addition, about the said process, a well-known method can be utilized including the method reported in "A. F. Frangi et al. Multiscale vessel enhancement filtering, Proceedings of MICCAI, 130-137, 1998". Each image (tubular structure extraction image) after the tubular structure is extracted by image analysis using the Hessian matrix is stored in the one-end memory 117.

  FIG. 7c is a diagram illustrating an example of an image processed in the information processing apparatus 100 according to an embodiment of the present disclosure. Specifically, FIG. 7c is a diagram illustrating an example of a tubular structure extraction image. In FIG. 7b, the portion analyzed as a tubular structure among the blood vessels displayed in “black” or a color close to black is displayed in white.

  Next, returning to FIG. 6, the processor 112 reads each tubular structure extracted image from the memory 117, and executes binarization processing for each pixel (S <b> 204). Specifically, the processor 112 executes a process of converting each pixel value indicated by a gray scale of 0 to 255 to two gradations of white and black by comparing with a predetermined threshold value. Note that the threshold value can be appropriately set as desired. Each image (binarized image) after the binarization processing is stored in the memory 117 at one end.

  FIG. 7d is a diagram illustrating an example of an image processed in the information processing apparatus 100 according to an embodiment of the present disclosure. Specifically, FIG. 7d is a diagram illustrating an example of a binarized image. As is clear from FIG. 7d, the image of FIG. 7c drawn in gray scale is converted into two gradations of white and black and displayed. As a result, the subsequent processing can be speeded up.

  Note that, as shown in FIGS. 7a and 7d, there are regions in the living tissue that are unclearly displayed due to the region 12 where blood vessels overlap and the region 11 shifted in the depth direction. When a tubular structure extraction process or a binarization process is performed on an image including such an area, the area should be extracted as a tubular structure (area displayed in white in FIGS. 7c and 7d). Nevertheless, there are areas that are not extracted as tubular structures (areas shown in black in areas 13 and 14 in FIGS. 7c and 7d).

  Next, returning to FIG. 6, the processor 112 reads each spectral image of S201 from the memory 117 again, and executes a smoothing process (S205). For this process, a known smoothing process such as a process using a moving average filter or a process using a Gaussian filter can be used. Each image (smoothed image) that has been subjected to the smoothing process is stored in the memory 117.

  Next, the processor 112 reads out each smoothed image from the memory 117, and the region other than the region analyzed as a tubular structure (blood vessel) by the binarization processing in S204 (that is, black in FIG. 7d). With respect to the area indicated by the gradation, an analysis process based on the pixel value is executed for each pixel (S206). Specifically, the processor 112 calculates an average pixel value of the entire image for each smoothed image. Then, the processor 112 compares the pixel value of each pixel with the threshold value using the calculated average pixel value as a threshold value. If the pixel value is smaller than the threshold value, the processor 112 assigns a white gradation because it is a portion that should be recognized as a blood vessel. In addition, when the pixel value is larger than the threshold value, the processor 112 assigns a black gradation. Each image (pixel value analysis image) after this pixel value analysis processing is performed is stored in the memory 117 at one end.

  FIG. 7E is a diagram illustrating an example of an image processed in the information processing apparatus 100 according to an embodiment of the present disclosure. Specifically, it is a diagram illustrating an example of a pixel value analysis image. As described in FIGS. 7c and 7d, there are regions that are not recognized as tubular structures in the tubular structure extraction image even though they are blood vessels (regions 13 and 14 in FIGS. 7c and 7d, etc.). In the pixel value analysis processing image shown in FIG. 7e, a white gradation is assigned to an area to be analyzed as a blood vessel among such areas 13 and 14. Accordingly, by using a combination of the tubular structure extraction process and the pixel value analysis process, it is possible to analyze the blood vessel region more accurately.

  Next, returning to FIG. 6, the processor 112 reads the binarized image and the pixel value analysis image stored in the memory 117 and performs a process of synthesizing both images (S207). As the synthesis method, any known synthesis method may be applied. The synthesized image (synthesized image) is stored in the memory 117.

  FIG. 7F is a diagram illustrating an example of an image processed in the information processing apparatus 100 according to an embodiment of the present disclosure. Specifically, FIG. 7f is a diagram illustrating an example of a composite image. In the composite image, a region indicated by white gradation is a region recognized as a blood vessel. As is clear from FIG. 7f, the region that could not be analyzed as a blood vessel in FIGS. 7c and 7d is interpolated by the process shown in FIG. 7e, so that a more accurate blood vessel region can be analyzed.

  The above process is performed on each spectral image captured with blue light and green light.

  Thus, the process for extracting blood vessels from each spectral image captured by the probe 200 is completed.

6). Optical Density Calculation Processing FIG. 8 is a diagram illustrating a processing flow executed in the information processing apparatus 100 according to an embodiment of the present disclosure. Specifically, FIG. 8 is a diagram illustrating a processing flow performed when the processor 112 of the information processing apparatus 100 executes an instruction command stored in the memory 117.

  First, the processor 112 reads the composite image (image generated in S207) stored in the memory 117, and executes black and white inversion processing (S301). The inversion process is executed by a known method. Then, the processor 112 identifies a region that is not recognized as a blood vessel by the processing described in FIG. 6, that is, a background region, based on the image after reversal (reversed image). Next, the processor 112 reads out each smoothed image in S205 of FIG. 6 from the memory 117, and extracts the pixel value of each pixel included in the area specified as the background area (S302). Next, the processor 112 calculates an average pixel value of the background area based on each extracted pixel value (S303). As the average pixel value of the background region, the average pixel value of the background region around the coordinate position (x, y) is used with respect to the coordinate position (x, y) of the blood vessel part for calculating the optical density. The surrounding background area may have a predetermined size centered on the coordinate position (x, y) as a peripheral area, or the entire background image is divided by a grid to include the coordinate position (x, y). The background area may be a peripheral area. Then, the processor 112 calculates the optical density from the pixel value of each pixel and the calculated average pixel value of the background region for the region in the smoothed image corresponding to the region recognized as a blood vessel in FIG. 7F (S304). .

式（Ｉ）において、Ｄ（ｘ，ｙ）は座標位置（ｘ，ｙ）における光学濃度を、Ｉ（ｘ，ｙ）は座標位置（ｘ、ｙ）における透過光強度を、Ｉ_ｉｎ（ｘ，ｙ）は座標位置（ｘ、ｙ）における入射光強度を表す。ここで、透過光強度は、平滑化画像中の血管部の座標位置（ｘ、ｙ）で特定される画素の画素値が用いられる。また、入射光強度は、Ｓ３０３によって算出された背景領域の平均画素値が用いられる。 Specifically, the optical density D (x, y) in each pixel is calculated by the following formula (I).

In equation (I), D (x, y) is the optical density at the coordinate position (x, y), I (x, y) is the transmitted light intensity at the coordinate position (x, y), and I _in (x, y). y) represents the incident light intensity at the coordinate position (x, y). Here, the pixel value of the pixel specified by the coordinate position (x, y) of the blood vessel part in the smoothed image is used as the transmitted light intensity. For the incident light intensity, the average pixel value of the background area calculated in S303 is used.

  With the above formula (I), the optical density is calculated for each pixel for each smoothed image. Thus, the optical density calculation process is completed.

式（ＩＩ）において、Ｄ（λ）は、Ｓ３０４において算出された座標位置（ｘ、ｙ）における光学濃度を、ｓは座標位置（ｘ、ｙ）における血液の酸素飽和度、ε_ＨｂＯ２及びε_Ｈｂはそれぞれ酸素化ヘモグロビンと脱酸素化ヘモグロビンのモル吸収係数、ｃはヘモグロビンの全濃度、ｄは血管径を示す。 7). Based on the instruction command stored in the memory 117, the oxygen saturation calculation processor 112 estimates the oxygen saturation of blood by using information obtained based on the processes shown in FIGS. Execute the process. Specifically, for each coordinate position (x, y), the oxygen saturation is estimated using the following formula (II).

In the formula (II), D (λ) is the optical density at the coordinate position (x, y) calculated in S304, s is the oxygen saturation of blood at the coordinate position (x, y), ε _{HbO 2} and ε _Hb Represents the molar absorption coefficient of oxygenated hemoglobin and deoxygenated hemoglobin, c represents the total concentration of hemoglobin, and d represents the blood vessel diameter.

式（ＩＩＩ）において、Ψは青色光（λ₁）で撮像された画像と緑色光（λ₂）で撮像された画像の座標位置（ｘ、ｙ）における光学濃度の比（Ｄ（λ₂）／Ｄ（λ₁））を、Δλ_ｎは［εＨｂＯ_２（λ_ｎ）−εＨｂ（λ_ｎ）］（ｎは１または２）を表す。 Here, the oxygen saturation s is obtained by solving simultaneous equations of an expression obtained by substituting each numerical value calculated from an image captured with blue light and an expression obtained by substituting each numerical value calculated from an image captured with green light. It is calculated by. That is, the oxygen saturation s is calculated by the following formula (III).

In formula (III), [psi ratio of the optical density at the coordinate position of the image captured by the image and the green light captured by the blue light _{(λ 1) (λ 2)} (x, y) (D (λ 2) / D (λ ₁ )), Δλ _n represents [εHbO ₂ (λ _n ) −εHb (λ _n )] (n is 1 or 2).

  The processor 112 estimates the oxygen saturation of one or more coordinate positions (x, y) corresponding to the blood vessels included in the image based on the above formula (III).

8). The output processor 112 executes a process of outputting the estimated oxygen saturation as an index indicating the state of blood in the blood vessel based on the instruction command stored in the memory 117. FIG. 9 is a diagram illustrating an example of an image output from the information processing apparatus 100 according to an embodiment of the present disclosure. Specifically, the processor 112 classifies each gradation from blue to red according to the oxygen saturation estimated for each coordinate position, and displays the pixel corresponding to the coordinate position in the classified gradation. Control. At this time, among the pixels constituting the spectral image stored in the memory 117, the pixel corresponding to the coordinate position where the oxygen saturation is estimated is replaced with the classified gradation and displayed.

  As described above, in the present embodiment, since the oxygen saturation can be calculated as an index indicating the state of blood in the blood vessel for each coordinate position, it is possible to create a distribution map of blood oxygen saturation. It becomes possible to more accurately analyze a point with low or high oxygen saturation.

9. In the above embodiment, the oxygen saturation of blood is estimated as an index indicating the state of blood in the blood vessel. However, it is possible to estimate the total concentration of hemoglobin instead of or together with the oxygen saturation. is there. Specifically, in the formula (II), there are two unknowns, oxygen saturation s, total hemoglobin concentration c, and blood vessel diameter d, cd. The blood vessel diameter d can be calculated by a known method, for example, from the distribution (profile) of pixel values in the direction perpendicular to the blood vessel, using the half-value width as the blood vessel diameter. Therefore, in addition to the image obtained from the blue light and the green light, the equation (II) is obtained from the image obtained by irradiating another light (for example, blue-green light) whose peak wavelength is included in the absorption wavelength region of hemoglobin. ), It is possible to estimate the total hemoglobin concentration c in addition to the oxygen saturation s.

  In addition, the oxygen saturation and / or the total concentration of hemoglobin was used as an index indicating the state of blood in the blood vessel. However, the index is not the estimated numerical value itself but the estimated numerical value for each predetermined range. The classification may be divided into That is, the index may be each estimated numerical value itself or information processed based on the numerical value.

  In addition, as an index indicating the state of blood in the blood vessel, a predetermined coordinate position in the spectral image is replaced with a predetermined gradation and displayed on the display 111, but the spectral image is not necessarily used. For example, a normalized image, a smoothed image, a synthesized image, etc. stored in the memory 117 can be used. Further, when the image is displayed on the display 111, it is output in the map image format as shown in FIG. You may display with the indicating line shown. Furthermore, although it has been described that the display is displayed on the display 111 as an output format, it may be output from a printer connected to the information processing apparatus 100.

  Further, as an index indicating the state of blood in the blood vessel, it is output in a two-dimensional map image format as shown in FIG. 9, but the oxygen saturation estimated along the extracted blood vessel is plotted on the graph. May be. FIG. 10 is a diagram illustrating an example of an image output from the information processing apparatus 100 according to an embodiment of the present disclosure. Specifically, a graph in which each oxygen saturation calculated on the line 16 in FIG. 9 is plotted for each distance is shown. In this way, it is possible to display the index on the display 111 not in the map image format but in the graph format.

  Further, in the above embodiment, the case where the same LED 222 of the probe 200 emits light by periodically switching blue light and green light has been described. However, LEDs emitting blue light and green light may be arranged in advance, respectively. Is possible. Moreover, although multicolor LED was used as a light source and the color was switched periodically, it is also possible to use white light. In this case, it is desirable to use a so-called spectroscopic camera instead of a camera including a normal image sensor, or to capture each spectroscopic image using blue light and green light using a spectroscopic filter.

  In the blood vessel extraction process of the above-described embodiment, normalization process, binarization process, smoothing process, analysis process using pixel values, synthesis process, and the like are performed. However, these processes are not necessarily performed. That is, it is only necessary to extract a blood vessel portion from each captured spectral image, and if the accuracy is sufficient, it is possible to perform only the analysis process using the Hessian matrix.

  In the above embodiment, the image sensor 212 and the like are arranged on the probe 200. However, the probe 200 does not have to be provided exclusively for the system 1. That is, a light source can be arranged at the distal end portion of an endoscope or a laparoscope as in the present embodiment and used as a probe.

  Further, in the above-described embodiment, a threshold value for determining whether the state of the oxygen saturation level or the total concentration of hemoglobin is good or bad is set in advance, and the state of blood in the blood vessel is good or bad according to the threshold value. May be notified. For example, when the state of blood in the blood vessel is bad, a display that prompts attention such as “re-examination required” or “care is required during surgery” may be displayed on the display 111.

  The processes and procedures described in this specification can be realized not only by those explicitly described in the embodiment but also by software, hardware, or a combination thereof. Specifically, the processes and procedures described in this specification are realized by mounting logic corresponding to the processes on a medium such as an integrated circuit, a volatile memory, a nonvolatile memory, a magnetic disk, or an optical storage. Is done. The processes and procedures described in this specification can be implemented as a computer program and executed by various computers including an information processing apparatus and a server apparatus.

  Even though the processes and procedures described herein are described as being performed by a single device, software, component, or module, such processes or procedures may include multiple devices, multiple software, It may be executed by multiple components and / or multiple modules. Further, even if it is described that various types of information described in this specification are stored in a single memory or storage unit, such information may be stored in a plurality of memories or a plurality of memories provided in a single device. It can be distributed and stored in a plurality of memories arranged in a distributed manner. Further, the software and hardware elements described herein may be realized by integrating them into fewer components or by disassembling them into more components.

DESCRIPTION OF SYMBOLS 100 Information processing apparatus 200 Probe 300 Light source control apparatus

Claims (15)

A memory in which a predetermined instruction command is stored and an image including a blood vessel of a living tissue imaged by a probe;
An index indicating the state of blood in the blood vessel at one or a plurality of coordinate positions in the image is generated, and each generated index is stored in the memory in association with each of the coordinate positions. A processor configured to execute the directed instruction
An information processing apparatus including:   The information processing apparatus according to claim 1, wherein the index is generated by calculating blood oxygen saturation.   The information processing apparatus according to claim 2, wherein the oxygen saturation is calculated based on an optical density calculated at a coordinate position of a blood vessel specified from the image.   The information processing apparatus according to claim 1, wherein the image is an image captured in a state where the probe is in contact with a surface of the biological tissue.   The information processing apparatus according to claim 1, wherein the image is an image photographed by a dark field photographing method. The information processing apparatus further includes a display,
The processor executes the instruction command to output the indicator to the display;
The information processing apparatus according to any one of claims 1 to 5.   The said processor performs the said instruction | indication command so that the said parameter | index may be arrange | positioned and output in the said coordinate position on the said image or the other image produced | generated based on the said image. The information processing apparatus according to item.   The information processing apparatus according to claim 1, wherein the image is an image captured by irradiating a plurality of lights having different peak wavelength ranges.   The information processing apparatus according to claim 1, wherein the probe includes an LED light source capable of emitting a plurality of lights having different peak wavelength ranges to the living tissue.   The information processing apparatus according to claim 8 or 9, wherein the peak wavelength ranges of the plurality of lights are all included in a light absorption wavelength range of hemoglobin contained in blood.   The information processing apparatus according to claim 10, wherein the plurality of lights include at least blue light and green light.   The information processing apparatus according to claim 1, wherein the coordinate position is a coordinate position corresponding to a blood vessel specified from the image. A computer including a memory in which an image including a blood vessel of a living tissue imaged by a probe is stored;
An index indicating the state of blood in the blood vessel at one or a plurality of coordinate positions in the image is generated, and a process for outputting the generated indices in association with each of the coordinate positions is executed. Configured processor,
Program to function as. A method performed by a processor executing a predetermined instruction instruction stored in a memory,
Storing an image including a blood vessel of a living tissue imaged by a probe;
Generating each index indicating the state of blood in the blood vessel at one or more coordinate positions in the image;
Outputting each generated index in association with each of the coordinate positions;
Including methods. The information processing apparatus according to any one of claims 1 to 11,
A light source that is communicably connected to the information processing apparatus and capable of irradiating a plurality of lights having different peak wavelength ranges; Including a probe;
Including system.

Images