JP2023123874A - Photoacoustic imaging system, photoacoustic imaging system control method, and program - Google Patents

Abstract

To provide an information processing device used for a system for generating a display image capable of grasping easily a structure of a contrast object by photoacoustic imaging.SOLUTION: There is provided an image processing device for processing three dimensional image data in time series including three dimensional image data corresponding to each of a plurality of times of light irradiation, generated based on an input signal of a photoacoustic wave generated by the plurality of times of light irradiation to an analyte. The image processing device has flow information acquisition means for acquiring flow information of a light absorber in the analyte, based on the three dimensional image data in time series.SELECTED DRAWING: Figure 1

Description

The present invention relates to information processing used in systems that generate images by photoacoustic imaging.

2. Description of the Related Art Photoacoustic imaging (also referred to as “photoacoustic imaging”) using a contrast medium is known for examining blood vessels, lymphatic vessels, and the like. Patent Document 1 describes a photoacoustic image generating apparatus that evaluates a contrast agent used for imaging lymph nodes, lymph vessels, etc., and emits light with a wavelength that is absorbed by the contrast agent and generates a photoacoustic wave. is described.

WO2017/002337

However, in the photoacoustic imaging described in Patent Document 1, there are cases where it is difficult to grasp the structure of the imaging target inside the subject (for example, running of blood vessels, lymphatic vessels, etc.).

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide an information processing apparatus used in a system that generates an image that facilitates understanding of the structure of a contrast target by photoacoustic imaging.

The present invention
Time-series three-dimensional image data including three-dimensional image data corresponding to each of the plurality of light irradiations, generated based on received signals of photoacoustic waves generated by the plurality of light irradiations to the subject. An image processing device that processes
Provided is an image processing apparatus comprising flow information acquisition means for acquiring flow information of a light absorber in the subject based on the time-series three-dimensional image data.
The present invention also provides
Time-series three-dimensional image data including three-dimensional image data corresponding to each of the plurality of light irradiations, generated based on received signals of photoacoustic waves generated by the plurality of light irradiations to the subject. An image processing method for processing
An image processing method is provided, comprising a flow information acquiring step of acquiring flow information of a light absorber in the subject based on the time-series three-dimensional image data.
The present invention also provides
A method for displaying an image obtained by repeatedly irradiating light pulses and obtaining substantially continuously three-dimensional images of a light absorber distribution of a subject in a specific region of the subject, the image displaying method comprising a series of continuously obtained images. is repeatedly reproduced and displayed at a predetermined speed.
The present invention also provides
Time-series three-dimensional image data including three-dimensional image data corresponding to each of the plurality of light irradiations, generated based on received signals of photoacoustic waves generated by the plurality of light irradiations to the subject. A program for causing a computer to execute an image processing method for processing
The image processing method provides a program characterized by having a flow information obtaining step of obtaining flow information of a light absorber in the subject based on the time-series three-dimensional image data.

ADVANTAGE OF THE INVENTION According to this invention, the information processing apparatus used for the system which produces|generates the image which can grasp|ascertain the structure of contrast-enhanced object easily by photoacoustic imaging can be provided.

1 is a block diagram of a system according to an embodiment of the invention; FIG. 1 is a block diagram showing a specific example of an image processing apparatus and its peripheral configuration according to an embodiment of the present invention; FIG. Detailed block diagram of a photoacoustic device according to an embodiment of the present invention Schematic diagram of a probe according to one embodiment of the present invention 1 is a flowchart of an image processing method according to an embodiment of the present invention; FIG. FIG. 2 is a flow diagram of acquisition of lymphatic flow information according to one embodiment of the present invention; Contour graph of the calculated value of formula (1) corresponding to the contrast agent when the combination of wavelengths is changed A line graph showing the calculated value of formula (1) corresponding to the contrast agent when the concentration of ICG is changed. Graph showing Moller absorption coefficient spectra of oxyhemoglobin and deoxyhemoglobin FIG. 4 is a diagram showing a GUI according to an embodiment of the present invention; Diagram explaining how the image processing device acquires lymphatic flow information Spectroscopic image of the extensor side of the right forearm with varying concentrations of ICG Spectroscopic image of the left forearm extensor side when the concentration of ICG is changed Spectroscopic images of the inner right lower leg and inner left lower leg with varying concentrations of ICG

Preferred embodiments of the present invention will be described below with reference to the drawings. However, the dimensions, materials, shapes, and relative positions of the components described below should be appropriately changed according to the configuration of the device to which the invention is applied and various conditions. Therefore, it is not intended to limit the scope of the present invention to the following description.

The photoacoustic image obtained by the system according to the present invention reflects the amount and rate of absorption of light energy. A photoacoustic image is an image representing the spatial distribution of at least one object information such as the generated sound pressure (initial sound pressure) of the photoacoustic wave, the light absorption energy density, and the light absorption coefficient. The photoacoustic image may be an image representing a two-dimensional spatial distribution, or an image (volume data) representing a three-dimensional spatial distribution. A system according to this embodiment generates a photoacoustic image by imaging a subject into which a contrast medium has been introduced. Note that the photoacoustic image may be an image representing a two-dimensional spatial distribution or an image representing a three-dimensional spatial distribution in the depth direction from the surface of the subject in order to grasp the three-dimensional structure of the imaging target.

Also, the system according to the present invention can generate a spectroscopic image of the subject using a plurality of photoacoustic images corresponding to a plurality of wavelengths. The spectroscopic image of the present invention is an image generated using photoacoustic signals corresponding to each of a plurality of wavelengths based on photoacoustic waves generated by irradiating a subject with light of a plurality of wavelengths different from each other. be.
Note that the spectroscopic image may be an image showing the concentration of a specific substance in the subject, generated using photoacoustic signals corresponding to each of a plurality of wavelengths. When the light absorption coefficient spectrum of the contrast medium used differs from the light absorption coefficient spectrum of the specific substance, the image value of the contrast medium in the spectral image differs from the image value of the specific substance in the spectral image. Therefore, it is possible to distinguish the region of the contrast agent from the region of the specific substance according to the image value of the spectral image. Note that specific substances include substances that constitute a subject, such as hemoglobin, glucose, collagen, melanin, fat, and water. Also in this case, it is necessary to select a contrast agent having a light absorption spectrum different from the light absorption coefficient spectrum of the specific substance. Also, the spectroscopic image may be calculated by different calculation methods depending on the type of the specific substance.

ここで、Ｉ^λ _１（ｒ）は第１波長λ_１の光照射により発生した光音響波に基づいた計測値であり、Ｉ^λ _２（ｒ）は第２波長λ_２の光照射により発生した光音響波に基づいた計測値である。ε_Ｈｂ ^λ _１は第１波長λ_１に対応するデオキシヘモグロビンのモラー吸収係数［ｍｍ^－１ｍｏｌ^－１］であり、ε_Ｈｂ ^λ _２は第２波長λ_２に対応するデオキシヘモグロビンのモラー吸収係数［ｍｍ^－１ｍｏｌ^－１］である。ε_ＨｂＯ ^λ _１は第１波長λ_１に対応するオキシヘモグロビンのモラー吸収係数［ｍｍ^－１ｍｏｌ^－１］であり、ε_ＨｂＯ ^λ _２は第２波長λ_２に対応するオキシヘモグロビンのモラー吸収係数［ｍｍ^－１ｍｏｌ^－１］である。ｒは位置である。なお、計測値Ｉ^λ _１（ｒ）、Ｉ^λ _２（ｒ）としては、吸収係数μ_ａ ^λ _１（ｒ）、μ_ａ ^λ _２（ｒ）を用いてもよいし、初期音圧Ｐ_０ ^λ _１（ｒ）、Ｐ_０ ^λ _２（ｒ）を用いてもよい。 In the embodiments described below, an image calculated using the oxygen saturation calculation formula (1) will be described as a spectral image. The present inventors have found that the oxygen saturation of blood hemoglobin (which may be an index correlated with oxygen saturation) is calculated based on photoacoustic signals corresponding to each of a plurality of wavelengths. The range of possible values for the oxygen saturation of hemoglobin when substituting the measured value I(r) of the photoacoustic signal obtained with a contrast agent that shows a different trend in the wavelength dependence of the absorption coefficient from that of oxyhemoglobin and deoxyhemoglobin. It has been found that a calculated value Is(r) deviating greatly from is obtained. Therefore, if a spectroscopic image having this calculated value Is(r) as an image value is generated, a hemoglobin region (blood vessel region) and a contrast agent existing region (for example, a lymphatic vessel where the contrast agent is introduced into the subject) can be obtained. In this case, it becomes easy to separate (distinguish) from the area of the lymphatic vessel) on the image.

Here, I ^λ ₁ (r) is a measured value based on a photoacoustic wave generated by light irradiation with a first wavelength λ ₁ , and I ^λ ₂ (r) is a measured value generated by light irradiation with a second wavelength λ ₂ These are measured values based on photoacoustic waves. ε _Hb ^λ ₁ is the Molar absorption coefficient of deoxyhemoglobin corresponding to the first wavelength λ ₁ [mm ⁻¹ mol ⁻¹ ], and ε _Hb ^λ ₂ is the Molar absorption coefficient of deoxyhemoglobin corresponding to the second wavelength λ ₂ [ mm ⁻¹ mol ⁻¹ ]. ε _HbO ^λ ₁ is the Molar absorption coefficient of oxyhemoglobin corresponding to the first wavelength λ ₁ [mm ⁻¹ mol ⁻¹ ], and ε _HbO ^λ ₂ is the Molar absorption coefficient of oxyhemoglobin corresponding to the second wavelength λ ₂ [ mm ⁻¹ mol ⁻¹ ]. r is the position. As the measured values I ^λ ₁ (r) and I ^λ ₂ (r), the absorption coefficients μ _a ^λ ₁ (r) and μ _a ^λ ₂ (r) may be used, or the initial sound pressure P ₀ ^λ ₁ (r) and P ₀ ^λ ₂ (r) may be used.

Substituting the measured value based on the photoacoustic wave generated from the region where hemoglobin exists (blood vessel region) into Equation (1), the calculated value Is(r) is the oxygen saturation of hemoglobin (or index) is obtained. On the other hand, in a subject into which a contrast agent has been introduced, when the measured value based on the acoustic wave generated from the region where the contrast agent exists (for example, the lymphatic region) is substituted into Equation (1), the calculated value Is(r) is obtained as a pseudo A typical concentration distribution of the contrast agent is obtained. Even when calculating the concentration distribution of the contrast agent, the numerical value of the Moller absorption coefficient of hemoglobin may be used as it is in the equation (1). The spectroscopic image Is(r) thus obtained is such that both the hemoglobin-existing region (blood vessel) and the contrast agent-existing region (e.g., lymphatic vessel) inside the subject are separable (distinguishable) from each other. A rendered image.

In the present embodiment, the image value of the spectral image is calculated using the equation (1) for calculating the oxygen saturation. A calculation method other than (1) may be used. As the index and its calculation method, a known one can be used, so a detailed explanation is omitted here.

In addition, the system according to the present invention includes a first photoacoustic image based on the photoacoustic wave generated by the light irradiation of the first wavelength λ ₁ and a second photoacoustic wave generated by the light irradiation of the second wavelength λ ₂ An image showing the ratio of the two photoacoustic images may be used as the spectral image. That is, the ratio of the first photoacoustic image based on the photoacoustic wave generated by the light irradiation of the first wavelength λ ₁ and the second photoacoustic image based on the photoacoustic wave generated by the light irradiation of the second wavelength λ ₂ is The image on which it is based may be the spectral image. Note that the image generated according to the modified equation of formula (1) can also be expressed by the ratio of the first photoacoustic image and the second photoacoustic image, so based on the ratio of the first photoacoustic image and the second photoacoustic image can be said to be an image (spectral image).

Note that the spectral image may be an image representing a two-dimensional spatial distribution or an image representing a three-dimensional spatial distribution in the depth direction from the surface of the subject in order to grasp the three-dimensional structure of the contrast target.

The configuration of the system and the image processing method of this embodiment will be described below.
A system according to this embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of the system according to this embodiment. The system according to this embodiment includes a photoacoustic device 1100 , a storage device 1200 , an image processing device 1300 , a display device 1400 and an input device 1500 . Transmission and reception of data between devices may be performed by wire or wirelessly.
The photoacoustic device 1100 generates a photoacoustic image by imaging a subject into which a contrast medium has been introduced, and outputs the photoacoustic image to the storage device 1200 . The photoacoustic device 1100 is a device that generates characteristic value information corresponding to each of a plurality of positions within a subject using received signals obtained by receiving photoacoustic waves generated by light irradiation. That is, the photoacoustic device 1100 is a device that generates the spatial distribution of characteristic value information derived from photoacoustic waves as medical image data (photoacoustic image).

The storage device 1200 may be a storage medium such as a ROM (Read only memory), a magnetic disk, or a flash memory. The storage device 1200 may also be a storage server via a network such as PACS (Picture Archiving and Communication System).

The image processing device 1300 is a device that processes information such as photoacoustic images stored in the storage device 1200 and incidental information of the photoacoustic images.
A unit that performs the arithmetic function of the image processing apparatus 1300 can be configured by a processor such as a CPU or a GPU (Graphics Processing Unit) and an arithmetic circuit such as an FPGA (Field Programmable Gate Array) chip. These units may be composed not only of a single processor or arithmetic circuit, but also of a plurality of processors or arithmetic circuits.

The unit responsible for the storage function of the image processing apparatus 1300 can be composed of a non-temporary storage medium such as a ROM (Read only memory), a magnetic disk, or a flash memory. Also, the unit responsible for the storage function may be a volatile medium such as RAM (Random Access Memory). Note that the storage medium in which the program is stored is a non-temporary storage medium. It should be noted that the unit that performs the storage function may not only be composed of one storage medium, but may also be composed of a plurality of storage media.

A unit responsible for the control function of the image processing apparatus 1300 is composed of an arithmetic element such as a CPU. A unit responsible for control functions controls the operation of each component of the system. The unit responsible for the control function may receive instruction signals from various operations such as starting measurement from the input section and control each configuration of the system. Units responsible for control functions may also read program code stored in computer 150 to control the operation of each component of the system.

The display device 1400 is a display such as a liquid crystal display or an organic EL (Electro Luminescence). In addition, the display device 1400 may display an image or a GUI for operating the device.

As the input device 1500, a user-operable operation console composed of a mouse, a keyboard, and the like can be adopted. Alternatively, the display device 1400 may be configured with a touch panel and used as the input device 1500 .

FIG. 2 shows a specific configuration example of an image processing apparatus 1300 according to this embodiment. The image processing apparatus 1300 according to this embodiment is composed of a CPU 1310 , a GPU 1320 , a RAM 1330 , a ROM 1340 and an external storage device 1350 . A liquid crystal display 1410 as a display device 1400 , a mouse 1510 and a keyboard 1520 as input devices 1500 are connected to the image processing device 1300 . Furthermore, the image processing apparatus 1300 is a PACS (Picture Archiving and Communication
System) is connected to an image server 1210 as a storage device 1200 . As a result, image data can be saved on the image server 1210 and image data on the image server 1210 can be displayed on the liquid crystal display 1410 .

Next, a configuration example of the devices included in the system according to this embodiment will be described. FIG. 3 is a schematic block diagram of devices included in the system according to the present embodiment.

A photoacoustic device 1100 according to this embodiment has a drive unit 130 , a signal collection unit 140 , a computer 150 , a probe 180 and an introduction unit 190 . The probe 180 has a light emitting section 110 and a receiving section 120 . FIG. 4 shows a schematic diagram of the probe 180 according to this embodiment. A measurement target is the subject 100 into which the contrast medium has been introduced by the introduction section 190 . The drive unit 130 drives the light irradiation unit 110 and the reception unit 120 to perform mechanical scanning. The light irradiation unit 110 irradiates the subject 100 with light, and acoustic waves are generated within the subject 100 . Acoustic waves generated by the photoacoustic effect due to light are also called photoacoustic waves. The receiving unit 120 outputs an electric signal (photoacoustic signal) as an analog signal by receiving the photoacoustic wave.

The signal collector 140 converts the analog signal output from the receiver 120 into a digital signal and outputs the digital signal to the computer 150 . The computer 150 stores the digital signal output from the signal collection unit 140 as signal data derived from photoacoustic waves.

The computer 150 generates a photoacoustic image by performing signal processing on the stored digital signals. Further, the computer 150 outputs the photoacoustic image to the display unit 160 after performing image processing on the obtained photoacoustic image. The display unit 160 displays an image based on the photoacoustic image. The display image is saved in a memory in the computer 150 or in a storage device 1200 such as a data management system connected to the modality via a network, based on a save instruction from the user or the computer 150 .

The computer 150 also drives and controls components included in the photoacoustic device. Also, the display unit 160 may display a GUI or the like in addition to the image generated by the computer 150 . The input unit 170 is configured so that the user can input information. The user can use the input unit 170 to perform operations such as starting and ending measurement and instructing to save the created image.
The details of each configuration of the photoacoustic device 1100 according to this embodiment will be described below.

(Light irradiation unit 110)
The light irradiation unit 110 includes a light source 111 that emits light and an optical system 112 that guides the light emitted from the light source 111 to the subject 100 . The light includes pulsed light such as so-called rectangular waves and triangular waves.

Considering thermal confinement conditions and stress confinement conditions, the pulse width of the light emitted from the light source 111 is preferably 100 ns or less. Also, the wavelength of light may be in the range of about 400 nm to 1600 nm. When imaging blood vessels with high resolution, wavelengths (400 nm or more and 700 nm or less) that are highly absorbed by blood vessels may be used. When imaging a deep part of a living body, light having a wavelength (700 nm or more and 1100 nm or less) that is typically less absorbed in the background tissues (water, fat, etc.) of the living body may be used.

A laser or a light emitting diode can be used as the light source 111 . Moreover, when measuring using light of multiple wavelengths, the light source may be one whose wavelengths can be changed. In the case of irradiating a subject with a plurality of wavelengths, it is also possible to prepare a plurality of light sources that generate light of mutually different wavelengths and alternately irradiate from each light source. Even when a plurality of light sources are used, they are collectively expressed as a light source. Various lasers such as solid lasers, gas lasers, dye lasers, and semiconductor lasers can be used as the laser. For example, a pulsed laser such as an Nd:YAG laser or an alexandrite laser may be used as the light source. Alternatively, a Ti:sa laser or OPO (Optical Parametric Oscillators) laser that uses Nd:YAG laser light as excitation light may be used as a light source. Alternatively, a flash lamp or a light emitting diode may be used as the light source 111 . Alternatively, a microwave source may be used as the light source 111 .

Optical elements such as lenses, mirrors, and optical fibers can be used for the optical system 112 . When the breast or the like is used as the subject 100, the light emitting portion of the optical system may be configured with a diffusion plate or the like for diffusing light in order to widen the beam diameter of the pulsed light for irradiation. On the other hand, in the photoacoustic microscope, in order to increase the resolution, the light emitting part of the optical system 112 may be configured with a lens or the like, and the beam may be focused and irradiated.
Note that the light irradiation unit 110 may directly irradiate the subject 100 with light from the light source 111 without the optical system 112 .

(Receiver 120)
The receiver 120 includes a transducer 121 that outputs an electrical signal by receiving acoustic waves, and a support 122 that supports the transducer 121 . Also, the transducer 121 may be a transmitting means for transmitting acoustic waves. A transducer as a receiving means and a transducer as a transmitting means may be a single (common) transducer, or may be configured separately.

A piezoelectric ceramic material typified by PZT (lead zirconate titanate), a polymeric piezoelectric film material typified by PVDF (polyvinylidene fluoride), or the like can be used as a member constituting the transducer 121 . Elements other than piezoelectric elements may also be used. For example, a transducer using a capacitance type transducer (CMUT: Capacitive Micro-machined Ultrasonic Transducers) can be used. Any transducer may be employed as long as it can output an electrical signal by receiving an acoustic wave. Also, the signal obtained by the transducer is a time-resolved signal. That is, the amplitude of the signal obtained by the transducer represents a value based on the sound pressure received by the transducer at each time (for example, a value proportional to the sound pressure).
The frequency components that make up the photoacoustic wave are typically 100 kHz to 100 MHz,
A transducer 121 that can detect these frequencies may be employed.

The support 122 may be made of a metal material or the like having high mechanical strength. In order to allow more irradiation light to enter the subject, the surface of the support 122 on the side of the subject 100 may be mirror-finished or light-scattering. In this embodiment, the support 122 has a hemispherical shell shape and is configured to support a plurality of transducers 121 on the hemispherical shell. In this case, the pointing axes of the transducers 121 arranged on the support 122 are concentrated near the center of curvature of the hemisphere. Then, when an image is formed using the signals output from the plurality of transducers 121, the image quality near the center of curvature is enhanced. Note that the support 122 may have any configuration as long as it can support the transducer 121 . The support 122 may arrange multiple transducers side by side in a flat or curved surface, such as a 1D array, 1.5D array, 1.75D array, or 2D array. A plurality of transducers 121 correspond to a plurality of receiving means.

Further, the support 122 may function as a container for storing the acoustic matching material. That is, the support 122 may be a container for placing the acoustic matching material between the transducer 121 and the subject 100 .

The receiver 120 may also include an amplifier that amplifies the time-series analog signal output from the transducer 121 . Further, the receiving unit 120 may include an A/D converter that converts time-series analog signals output from the transducer 121 into time-series digital signals. That is, the receiver 120 may include a signal collector 140, which will be described later.

A space between the receiving unit 120 and the subject 100 is filled with a medium through which photoacoustic waves can propagate. For this medium, a material that allows propagation of acoustic waves, matches the acoustic characteristics at the interface with the object 100 and the transducer 121, and has the highest possible photoacoustic wave transmittance is adopted. For example, this medium can be water, ultrasonic gel, or the like.

FIG. 4 shows a side view of probe 180 . A probe 180 according to this embodiment has a receiving section 120 in which a plurality of transducers 121 are three-dimensionally arranged on a hemispherical support 122 having an opening. In addition, a light exit part of the optical system 112 is arranged at the bottom of the support 122 .

In this embodiment, as shown in FIG. 4, the shape of the object 100 is held by contacting the holding part 200 .
A space between the receiving unit 120 and the holding unit 200 is filled with a medium through which photoacoustic waves can propagate. For this medium, a material that allows photoacoustic waves to propagate, has matching acoustic characteristics at the interface with the object 100 and the transducer 121, and has a photoacoustic wave transmittance that is as high as possible is used. For example, this medium can be water, ultrasonic gel, or the like.

A holding part 200 as holding means is used to hold the shape of the subject 100 during measurement. By holding the subject 100 by the holding section 200 , the movement of the subject 100 can be suppressed and the position of the subject 100 can be kept within the holding section 200 . A resin material such as polycarbonate, polyethylene, or polyethylene terephthalate can be used as the material of the holding portion 200 .

The holding portion 200 is attached to the attachment portion 201 . The attachment section 201 may be configured to be able to replace a plurality of types of holding sections 200 according to the size of the subject. For example, the mounting portion 201 may be configured to be replaceable with a holding portion having a different radius of curvature, center of curvature, or the like.

(Driving unit 130)
The driving unit 130 is a part that changes the relative position between the subject 100 and the receiving unit 120 . Driving unit 130 includes a motor such as a stepping motor that generates driving force, a driving mechanism that transmits the driving force, and a position sensor that detects position information of receiving unit 120 . A lead screw mechanism, a link mechanism, a gear mechanism, a hydraulic mechanism, or the like can be used as the drive mechanism. As the position sensor, a potentiometer using an encoder, a variable resistor, a linear scale, a magnetic sensor, an infrared sensor, an ultrasonic sensor, or the like can be used.

The drive unit 130 is not limited to changing the relative position between the subject 100 and the receiving unit 120 in the XY directions (two-dimensional), but may change the position one-dimensionally or three-dimensionally.
Note that the driving unit 130 may fix the receiving unit 120 and move the subject 100 as long as the relative positions of the subject 100 and the receiving unit 120 can be changed. When the subject 100 is moved, a configuration in which the subject 100 is moved by moving a holding unit that holds the subject 100 can be considered. Also, both the subject 100 and the receiving unit 120 may be moved.

The driving unit 130 may move the relative position continuously or may move by step-and-repeat. The drive unit 130 may be an electric stage that moves along a programmed trajectory, or may be a manual stage.
In the present embodiment, the drive unit 130 simultaneously drives the light irradiation unit 110 and the reception unit 120 to perform scanning. may
It should be noted that if the probe 180 is of a handheld type provided with a grip portion, the photoacoustic device 1100 does not need to have the driving portion 130 .

(Signal collection unit 140)
The signal collector 140 includes an amplifier that amplifies the electrical signal, which is an analog signal output from the transducer 121, and an A/D converter that converts the analog signal output from the amplifier into a digital signal. A digital signal output from the signal acquisition unit 140 is stored in the computer 150 . The signal acquisition unit 140 is also called a Data Acquisition System (DAS). In this specification, an electric signal is a concept including both analog signals and digital signals. A light detection sensor such as a photodiode may detect light emission from the light irradiation unit 110, and the signal collection unit 140 may start the above processing in synchronization with the detection result as a trigger.

(Computer 150)
A computer 150 as an information processing device is configured with hardware similar to that of the image processing device 1300 . That is, the unit that performs the arithmetic function of the computer 150 can be configured by a processor such as a CPU or GPU (Graphics Processing Unit), and an arithmetic circuit such as an FPGA (Field Programmable Gate Array) chip. These units may be composed not only of a single processor or arithmetic circuit, but also of a plurality of processors or arithmetic circuits.
The unit responsible for the storage function of computer 150 may be a volatile medium such as RAM (Random Access Memory). Note that the storage medium in which the program is stored is a non-temporary storage medium. Note that the unit that performs the storage function of the computer 150 may be configured not only from one storage medium but also from a plurality of storage media.

A unit that performs the control function of the computer 150 is composed of an arithmetic element such as a CPU. The unit responsible for the control function of the computer 150 controls the operation of each component of the photoacoustic device. The unit responsible for the control function of the computer 150 may receive instruction signals from the input unit 170 for various operations such as starting measurement, and control each component of the photoacoustic device. Also, the unit responsible for the control function of the computer 150 reads the program code stored in the unit responsible for the storage function, and controls the operation of each component of the photoacoustic device. That is, the computer 150 can function as a control device of the system according to this embodiment.

Note that the computer 150 and the image processing apparatus 1300 may be configured with the same hardware. A piece of hardware may serve both the functions of the computer 150 and the image processing device 1300 . That is, the computer 150 may serve the functions of the image processing device 1300 . Also, the image processing device 1300 may serve the function of the computer 150 as an information processing device.

(Display unit 160)
The display unit 160 is a display such as a liquid crystal display or an organic EL (Electro Luminescence). The display unit 160 may also display an image or a GUI for operating the device.
Note that the display unit 160 and the display device 1400 may be the same display. That is, one display may have the functions of both the display section 160 and the display device 1400 .

(Input unit 170)
As the input unit 170, an operation console configured by a user-operable mouse, keyboard, or the like can be adopted. Alternatively, the display unit 160 may be configured with a touch panel and used as the input unit 170 .
Note that the input unit 170 and the input device 1500 may be the same device. That is, one device may serve both the functions of the input unit 170 and the input device 1500 .

(Introduction part 190)
The introduction unit 190 is configured to be able to introduce a contrast agent into the subject 100 from the outside of the subject 100 . For example, the introduction part 190 can include a contrast agent container and an injection needle for piercing the subject. However, the introduction part 190 is not limited to this, and various types can be applied as long as the introduction part 190 can introduce the contrast agent into the subject 100 . The introduction part 190 may in this case be, for example, a known injection system or injector. Note that the contrast agent may be introduced into the subject 100 by the computer 150 as a control device controlling the operation of the introduction section 190 . Alternatively, the user may operate the introduction unit 190 to introduce the contrast medium into the subject 100 .

(Subject 100)
Although the subject 100 does not constitute a system, it will be described below. The system according to the present embodiment can be used for purposes such as diagnosing malignant tumors and vascular diseases in humans and animals, monitoring the course of chemotherapy, and the like. Therefore, the subject 100 is assumed to be a living body, specifically, a diagnosis target part such as a human body or animal breast, organs, blood vessel network, head, neck, abdomen, limbs including fingers or toes. be. For example, if the human body is the object of measurement, oxyhemoglobin or deoxyhemoglobin, blood vessels containing many of them, new blood vessels formed in the vicinity of a tumor, or the like may be the object of the light absorber. In addition, plaque on the wall of the carotid artery may be used as a light absorber. In addition, melanin, collagen, lipids, and the like contained in the skin and the like may be used as light absorbers. The contrast agent introduced into the subject 100 can be a light absorber. Contrast agents used in photoacoustic imaging may include dyes such as indocyanine green (ICG) and methylene blue (MB), fine gold particles, or externally introduced substances accumulated or chemically modified with them. Alternatively, a phantom imitating a living body may be used as the subject 100 .

Each component of the photoacoustic device may be configured as a separate device, or may be configured as an integrated device. Also, at least a part of the photoacoustic device may be configured as one device integrated.
Note that each device constituting the system according to the present embodiment may be configured with separate hardware, or all the devices may be configured with one piece of hardware. The functions of the system according to this embodiment may be configured with any hardware.

Next, the image generation method according to this embodiment will be described using the flowchart shown in FIG. Note that the flowchart shown in FIG. 5 includes steps showing the operation of the system according to the present embodiment and steps showing the operation of a user such as a doctor.

(S100: Step of acquiring information on inspection)
The computer 150 of the photoacoustic device 1100 acquires information regarding inspection. For example, the computer 150 acquires examination order information transmitted from an in-hospital information system such as HIS (Hospitai Information System) or RIS (Radiology Information System). The examination order information includes information such as the type of modality used for examination and the contrast medium used for examination.

In addition, when the modality is photoacoustic imaging, the inspection order information includes information on irradiated light. Since the main embodiments of the present invention are capable of calculating flow information based on light of a single wavelength, it is only necessary to obtain information of at least a single wavelength. Information about the light may also include the pulse length, repetition frequency, intensity, etc. of the light.

Here, the lymphatic flow information obtained by the flow described below may include various types of information on how lymph flows, such as the frequency, flow rate, and flow velocity of lymph flowing in lymphatic vessels. Flow information may include the standard deviation over time and peak-to-peak (PP) of luminance values in the image. The flow information can be expressed in any manner, and for example, a method of expressing images (including moving images), a method of expressing physically, a method of expressing by words, and the like can be adopted.

In addition, in the present invention, when spectroscopic images are generated using light of a plurality of wavelengths different from each other and lymphatic flow information is acquired based on the spectroscopic images, the examination order information does not need to include information on each wavelength. be. When using multiple wavelengths, in setting the wavelengths, when an oxygen saturation image is generated as a spectral image according to formula (1), the blood vessel region in the spectral image is an image corresponding to the actual oxygen saturation While the values are calculated, it is preferable to take into account that the image values of the region of the contrast agent in the spectroscopic image change greatly depending on the wavelength used and the absorption coefficient spectrum of the contrast agent. That is, in order to facilitate understanding of the three-dimensional structure of the contrast agent, it is possible to use a wavelength that makes the image value of the contrast agent region in the spectroscopic image distinguishable from the image value of the blood vessel region. preferable. Specifically, when generating an image using formula (1) as a spectral image, the oxygen saturation of arteries and veins is generally within the range of 60% to 100%, and contrast enhancement in the spectral image is used. It is preferable to use two wavelengths such that the value of formula (1) corresponding to the agent is less than 60% (eg negative) or greater than 100%. For example, when using ICG as a contrast agent, two wavelengths of 700 nm or more and less than 820 nm and a wavelength of 820 nm or more and 1020 nm or less are selected, and a spectroscopic image is generated according to formula (1). and blood vessel regions can be distinguished well. Thus, 700 nm to 820 nm and 820 nm to 102 nm
Using a wavelength of 0 nm is a preferred example of this embodiment.

In addition, the user may use the input unit 170 to specify the type of modality used for the examination, information about light when the modality is photoacoustic imaging, the type of contrast agent used for the examination, and the concentration of the contrast agent. . In this case, the computer 150 can acquire examination information via the input unit 170 .

FIG. 10 shows an example of a GUI displayed on the display unit 160. As shown in FIG. A GUI item 2500 displays examination order information such as a patient ID, an examination ID, and an imaging date and time. The item 2500 may have a display function for displaying inspection order information acquired from an external device such as HIS or RIS, and an input function for allowing the user to input inspection order information using the input unit 170 . A GUI item 2600 displays information about the contrast medium, such as the type of contrast medium and the concentration of the contrast medium. The item 2600 may have a display function for displaying information about the contrast agent acquired from an external device such as HIS or RIS, and an input function for allowing the user to input information about the contrast agent using the input unit 170. good. In the item 2600, it may be possible to input information on the contrast agent such as the type and concentration of the contrast agent by a method such as pull-down from a plurality of options. Note that the GUI shown in FIG. 10 may be displayed on the display device 1400 .

It should be noted that the image processing apparatus 1300 may acquire information on a contrast agent set by default from information on a plurality of contrast agents when an instruction to input information on the contrast agent is not received from the user. In the case of this embodiment, a case where ICG is set as the type of contrast medium and 1.0 mg/mL as the concentration of the contrast medium by default will be described. In this embodiment, the GUI item 2600 displays the type and concentration of the contrast agent set by default, but the information on the contrast agent may not be set by default. In this case, the information about the contrast medium may not be displayed in the item 2600 of the GUI on the initial screen.

(S200: Step of introducing contrast medium)
The introduction unit 190 introduces a contrast medium into the subject. When the user introduces the contrast agent into the subject using the introduction unit 190, the user operates the input unit 170 to output a signal indicating that the contrast agent has been introduced from the input unit 170 as a control device. It may be sent to computer 150 . In addition, the introduction unit 190 may transmit a signal indicating that the contrast agent has been introduced into the subject 100 to the computer 150 . Note that the contrast medium may be administered to the subject without using the introduction section 190 . For example, the contrast agent may be administered by inhaling the sprayed contrast agent into the living body as the subject.
After introduction of the contrast medium, the subsequent processing may be performed after a period of time until the contrast medium spreads over the object to be contrast-enhanced in the subject 100 .

Here, a spectroscopic image obtained by photographing a living body into which ICG has been introduced using a photoacoustic device will be described. 12 to 14 show spectroscopic images taken when ICG was introduced at different concentrations. In each imaging, 0.1 mL of ICG was introduced subcutaneously or intradermally into the hand or foot. Since the ICG introduced subcutaneously or intradermally is selectively taken into the lymphatic vessel, the lumen of the lymphatic vessel is imaged. All images were taken within 5 to 60 minutes after ICG introduction. Moreover, both spectral images are spectral images generated from photoacoustic images obtained by irradiating a living body with light with a wavelength of 797 nm and light with a wavelength of 835 nm.

FIG. 12(A) shows a spectroscopic image of the extensor side of the right forearm when ICG was not introduced. On the other hand, FIG. 12(B) shows a spectroscopic image of the extensor side of the right forearm when ICG was introduced at a concentration of 2.5 mg/mL. Lymphatic vessels are depicted in the regions indicated by dashed lines and arrows in FIG. 12(B).
FIG. 13(A) shows a spectroscopic image of the extensor side of the left forearm when ICG was introduced at a concentration of 1.0 mg/mL. FIG. 13(B) shows a spectroscopic image of the extensor side of the left forearm when ICG was introduced at a concentration of 5.0 mg/mL. Lymphatic vessels are depicted in the regions indicated by dashed lines and arrows in FIG. 13(B).
FIG. 14(A) shows a spectroscopic image of the inner right leg when ICG at a concentration of 0.5 mg/mL was introduced. FIG. 14(B) shows a spectroscopic image of the inner left leg when ICG with a concentration of 5.0 mg/mL was introduced. Lymphatic vessels are depicted in the regions indicated by dashed lines and arrows in FIG. 14(B).

According to the spectroscopic images shown in FIGS. 12 to 14, it is understood that increasing the concentration of ICG improves the visibility of lymphatic vessels in the spectroscopic images. Moreover, according to FIGS. 12 to 14, it is understood that the lymphatic vessels can be well visualized when the ICG concentration is 2.5 mg/mL or higher. That is, when the concentration of ICG is 2.5 mg/mL or more, the linear lymphatic vessels can be clearly visually recognized. Therefore, when ICG is employed as a contrast medium, its concentration may be 2.5 mg/mL or higher. Considering the dilution of ICG in vivo, the concentration of ICG may be greater than 5.0 mg/mL. However, considering the solubility of diagnogreen, it is difficult to dissolve it in an aqueous solution at a concentration of 10.0 mg/mL or higher.

As described above, the concentration of ICG to be introduced into the living body is preferably 2.5 mg/mL or more and 10.0 mg/mL or less, preferably 5.0 mg/mL or more and 10.0 mg/mL or less.

Therefore, the computer 150 is configured to selectively accept an instruction from the user indicating the concentration of ICG within the above numerical range when ICG is input as the type of contrast agent in item 2600 of the GUI shown in FIG. may be That is, in this case, the computer 150 may be configured not to accept an instruction from the user indicating an ICG concentration outside the above numerical range. Therefore, when the computer 150 acquires information indicating that the type of contrast medium is ICG, the computer 150 receives an instruction from the user indicating an ICG concentration of less than 2.5 mg/mL or greater than 10.0 mg/mL. may be configured not to accept In addition, when the computer 150 acquires information indicating that the type of contrast agent is ICG, the computer 150 receives an instruction from the user indicating an ICG concentration of less than 5.0 mg/mL or greater than 10.0 mg/mL. It may be configured not to accept it.

The computer 150 may configure the GUI so that the user cannot specify an ICG concentration outside the above numerical range on the GUI. In other words, the computer 150 may display the GUI so that the user cannot specify an ICG concentration outside the above numerical range on the GUI. For example, the computer 150 may display a pull-down on the GUI for selectively instructing the concentration of ICG within the above numerical range. The computer 150 may display ICG densities outside the above numerical range in the pull-down menu in a grayed out manner, and configure the GUI so that the grayed out densities cannot be selected.
Further, the computer 150 may issue an alert when the user instructs an ICG concentration outside the above numerical range on the GUI. As a notification method, any method such as displaying an alert on the display unit 160 or lighting a sound or a lamp can be adopted.
Further, when ICG is selected as the type of contrast agent on the GUI, the computer 150 may cause the display unit 160 to display the above numerical range as the concentration of ICG to be introduced into the subject.

Note that the concentration of the contrast agent introduced into the subject is not limited to the numerical range shown here, and a suitable concentration can be adopted according to the purpose. In addition, although an example in which the type of contrast agent is ICG has been described here, the above configuration can be similarly applied to other contrast agents.

By constructing the GUI in this way, it is possible to assist the user in introducing an appropriate concentration of the contrast medium into the subject according to the type of contrast medium to be introduced into the subject.

Next, the change in the image value corresponding to the contrast agent in the spectral image when the combination of wavelengths is changed will be described. FIG. 7 shows simulation results of image values (oxygen saturation values) corresponding to contrast agents in spectral images for each combination of two wavelengths. The vertical and horizontal axes in FIG. 7 represent the first wavelength and the second wavelength, respectively. FIG. 7 shows contour lines of image values corresponding to the contrast agent in the spectroscopic image. FIGS. 7(a) to 7(d) show contrast agents in spectroscopic images when the concentrations of ICG are 5.04 μg/mL, 50.4 μg/mL, 0.5 mg/mL, and 1.0 mg/mL, respectively. indicates the image value corresponding to . As shown in FIG. 7, depending on the combination of selected wavelengths, the image value corresponding to the contrast agent in the spectral image may be 60% to 100%. As described above, if such a combination of wavelengths is selected, it becomes difficult to distinguish between the blood vessel region and the contrast agent region in the spectroscopic image. Therefore, it is preferable to select a combination of wavelengths such that the image value corresponding to the contrast agent in the spectral image is less than 60% or greater than 100% in the wavelength combinations shown in FIG. Furthermore, in the combination of wavelengths shown in FIG. 7, it is preferable to select a combination of wavelengths such that the image value corresponding to the contrast medium in the spectral image is a negative value.

For example, consider the case where 797 nm is selected as the first wavelength and 835 nm is selected as the second wavelength. FIG. 8 shows the relationship between the concentration of ICG and the image value (value of formula (1)) corresponding to the contrast agent in the spectral image when 797 nm is selected as the first wavelength and 835 nm is selected as the second wavelength. is a graph showing According to FIG. 8, when 797 nm is selected as the first wavelength and 835 nm is selected as the second wavelength, contrast enhancement in the spectroscopic image is Image values corresponding to agents are negative. Therefore, according to the spectroscopic image generated by such a combination of wavelengths, since the oxygen saturation value of the blood vessel does not take a negative value in principle, the region of the blood vessel and the region of the contrast medium can be clearly distinguished. be able to.

Although the wavelength is determined based on the information about the contrast medium, the absorption coefficient of hemoglobin may be taken into consideration in determining the wavelength. FIG. 9 shows spectra of the Molar absorption coefficient of oxyhemoglobin (dashed line) and the Molar absorption coefficient of deoxyhemoglobin (solid line). In the wavelength range shown in FIG. 9, the magnitude relationship between the Molar absorption coefficient of oxyhemoglobin and the Molar absorption coefficient of deoxyhemoglobin is reversed at 797 nm. That is, it can be said that veins can be easily recognized at wavelengths shorter than 797 nm, and arteries can be easily recognized at wavelengths longer than 797 nm. By the way, in the treatment of lymphedema, lymphovenous anastomosis is used to form a bypass between a lymphatic vessel and a vein. For this preoperative examination, it is conceivable to image both veins and lymphatic vessels with accumulated contrast medium by photoacoustic imaging. In this case, by setting at least one of the plurality of wavelengths to a wavelength smaller than 797 nm, veins can be imaged more clearly. Further, it is advantageous for imaging veins to set at least one of the plurality of wavelengths to a wavelength at which the Molar absorption coefficient of deoxyhemoglobin is larger than that of oxyhemoglobin. In addition, when a spectral image is generated from a photoacoustic image corresponding to two wavelengths, both of the two wavelengths are set to wavelengths in which the Molar absorption coefficient of deoxyhemoglobin is larger than the Molar absorption coefficient of oxyhemoglobin. is advantageous. By selecting these wavelengths, it is possible to accurately image both the lymphatic vessel into which the contrast medium has been introduced and the vein in the preoperative examination of the lymphatic venule anastomosis.

By the way, if all of the plurality of wavelengths are wavelengths in which the absorption coefficient of the contrast agent is larger than that of the blood, artifacts derived from the contrast agent reduce the oxygen saturation accuracy of the blood. Therefore, in order to reduce artifacts derived from the contrast agent, at least one of the plurality of wavelengths may be a wavelength at which the absorption coefficient of the contrast agent is smaller than that of blood.

Here, a case of generating a spectral image according to formula (1) has been described, but a spectral image in which the image value corresponding to the contrast agent in the spectral image changes depending on the conditions of the contrast agent and the wavelength of the irradiation light It can also be applied when generating

(S300: Step of irradiating light)
The light irradiation unit 110 sets the wavelength determined based on the information acquired in S100 to the light source 111 . Light source 111 emits light of the determined wavelength. The light generated from the light source 111 is applied to the subject 100 as pulsed light through the optical system 112 . Then, the pulsed light is absorbed inside the subject 100, and a photoacoustic wave is generated by the photoacoustic effect. At this time, the introduced contrast agent also absorbs the pulsed light and generates photoacoustic waves. The light irradiation unit 110 may transmit a synchronization signal to the signal collection unit 140 together with transmission of the pulsed light. Also, the light irradiation unit 110 similarly performs light irradiation for each of the plurality of wavelengths.

The user may use the input unit 170 to specify control parameters such as irradiation conditions of the light irradiation unit 110 (repetition frequency and wavelength of irradiation light, etc.) and the position of the probe 180 . Computer 150 may set control parameters determined based on user instructions. Further, computer 150 may move probe 180 to a specified position by controlling drive unit 130 based on specified control parameters. When imaging at a plurality of positions is designated, the driving section 130 first moves the probe 180 to the first designated position. It should be noted that the drive unit 130 may move the probe 180 to a preprogrammed position when an instruction to start measurement is given.

(S400: Step of receiving photoacoustic waves)
Upon receiving the synchronization signal transmitted from the light irradiation section 110, the signal collection section 140 starts the signal collection operation. That is, the signal collection unit 140 generates an amplified digital electric signal by amplifying and AD converting the analog electric signal derived from the photoacoustic wave output from the receiving unit 120, and outputs it to the computer 150. . Computer 150 stores the signal transmitted from signal collector 140 . When imaging at a plurality of scanning positions is specified, the steps of S300 and S400 are repeatedly executed at the specified scanning positions, and the irradiation of pulsed light and the generation of digital signals derived from acoustic waves are repeated. Note that the computer 150 may acquire and store the position information of the receiving unit 120 at the time of light emission based on the output from the position sensor of the driving unit 130, using the light emission as a trigger.

In this embodiment, an example in which light with a plurality of wavelengths is irradiated in a time-division manner has been described, but the light irradiation method is not limited to this as long as signal data corresponding to each of the plurality of wavelengths can be obtained. . For example, when encoding is performed by light irradiation, there may be timings at which lights of a plurality of wavelengths are irradiated almost simultaneously.

(S500: Step of generating a photoacoustic image)
A computer 150 as photoacoustic image acquisition means generates a photoacoustic image based on the stored signal data. The computer 150 outputs the generated photoacoustic image to the storage device 1200 for storage. In this embodiment, one three-dimensional photoacoustic image (volume data) is generated by image reconstruction using a photoacoustic signal obtained by one irradiation of light to the subject. Furthermore, by performing light irradiation a plurality of times and performing image reconstruction for each light irradiation, time-series three-dimensional image data (time-series volume data) is acquired. Three-dimensional image data obtained by image reconstruction for each light irradiation of multiple times of light irradiation is generically referred to as three-dimensional image data corresponding to multiple times of light irradiation. Since light irradiation is performed a plurality of times in time series, the three-dimensional image data corresponding to the light irradiation a plurality of times forms time-series three-dimensional image data.

Reconstruction algorithms for transforming signal data into a two-dimensional or three-dimensional spatial distribution include analytical reconstruction methods such as backprojection in the time domain and backprojection in the Fourier domain, and model-based methods (repeated calculations). law) can be adopted. For example, backprojection methods in the time domain include universal back-projection (UBP), filtered back-projection (FBP), or delay-and-sum.

The computer 150 generates initial sound pressure distribution information (generated sound pressures at a plurality of positions) as a photoacoustic image by performing reconstruction processing on the signal data. Further, the computer 150 calculates the light fluence distribution inside the subject 100 of the light irradiated to the subject 100, divides the initial sound pressure distribution by the light fluence distribution, and converts the absorption coefficient distribution information into photoacoustic You may acquire it as an image. A known method can be applied as a method of calculating the light fluence distribution. In addition, the computer 150 can generate photoacoustic images corresponding to each of the multiple wavelengths of light. Specifically, the computer 150 can generate a first photoacoustic image corresponding to the first wavelength by performing reconstruction processing on signal data obtained by light irradiation of the first wavelength. Further, the computer 150 can generate a second photoacoustic image corresponding to the second wavelength by performing reconstruction processing on signal data obtained by light irradiation of the second wavelength. Thus, the computer 150 can generate multiple photoacoustic images corresponding to multiple wavelengths of light.

In this embodiment, the computer 150 acquires absorption coefficient distribution information corresponding to each of light of a plurality of wavelengths as a photoacoustic image. Let the absorption coefficient distribution information corresponding to the first wavelength be a first photoacoustic image, and let the absorption coefficient distribution information corresponding to the second wavelength be a second photoacoustic image.

In this embodiment, an example in which the system includes the photoacoustic device 1100 that generates a photoacoustic image has been described, but the present invention can also be applied to a system that does not include the photoacoustic device 1100 . The present invention can be applied to any system as long as the image processing device 1300 as a photoacoustic image acquiring means can acquire a photoacoustic image. For example, the present invention can be applied to a system that does not include the photoacoustic device 1100 but includes the storage device 1200 and the image processing device 1300 . In this case, the image processing device 1300 as a photoacoustic image acquiring means can acquire a photoacoustic image by reading out a specified photoacoustic image from among a group of photoacoustic images stored in advance in the storage device 1200. can.

(S600: Step of generating spectral image)
A computer 150 as spectral image acquisition means generates spectral images based on a plurality of photoacoustic images corresponding to a plurality of wavelengths. The computer 150 outputs the spectroscopic image to the storage device 1200 and stores it in the storage device 1200 . As described above, the computer 150 may generate, as a spectroscopic image, an image showing information corresponding to the concentration of substances that make up the subject, such as glucose concentration, collagen concentration, melanin concentration, fat and water volume fractions. good. Further, the computer 150 may generate an image representing the ratio between the first photoacoustic image corresponding to the first wavelength and the second photoacoustic image corresponding to the second wavelength as the spectral image. In this embodiment, an example will be described in which the computer 150 generates an oxygen saturation image as a spectroscopic image according to Equation (1) using the first photoacoustic image and the second photoacoustic image.

Note that the image processing device 1300 as spectral image acquisition means may acquire a spectral image by reading out a specified spectral image from a group of spectral images pre-stored in the storage device 1200 . In addition, the image processing device 1300 as a photoacoustic image acquisition means reads out from among the photoacoustic image groups pre-stored in the storage device 1200, at least one of the plurality of photoacoustic images used to generate the spectral image. You may acquire a photoacoustic image by reading.
Time-series three-dimensional image data corresponding to the multiple times of light irradiation is generated by performing multiple times of light irradiation and subsequent acoustic wave reception and image reconstruction. Photoacoustic image data and spectral image data can be used as the three-dimensional image data. The photoacoustic image data here refers to image data showing the distribution of absorption coefficients, etc., and the spectroscopic image data refers to photoacoustic image data corresponding to each wavelength when the subject is irradiated with light of multiple wavelengths. It refers to image data indicating density and the like generated based on the image data.

(S700: Step of acquiring and utilizing information on lymph flow based on photoacoustic image or spectroscopic image)
The image processing device 1300 reads out the photoacoustic image or the spectroscopic image from the storage device 1200, and obtains information about lymphatic flow based on the photoacoustic image or the spectroscopic image. In addition, as described above, the processing of this step can be performed based on a photoacoustic image derived from at least one wavelength. However, spectral images created from photoacoustic images derived from each of a plurality of wavelengths can also be used.

A method of performing flow information acquisition processing according to the present embodiment will be described with reference to the flowchart shown in FIG. This flow specifically describes the processing in step S700 of FIG. In the following description, the image processing device 1300 as flow information acquisition means is mainly responsible for information processing. However, the configuration of the flow information acquisition means is not limited to this, and any one of the components having the information processing function included in the present invention may perform the processing in the following flow.

(S710: Step of extracting lymphatic area from photoacoustic image or spectroscopic image)
Here, a case where image processing is performed on a photoacoustic image derived from a single wavelength will be described. The image processing device 1300 reads the photoacoustic image saved in the storage device 1200 in step S500 of FIG. The time range of the photoacoustic image to be read is arbitrary. In general, the flow of lymph fluid is intermittent, and the period is several tens of seconds to several minutes. It is preferable to sequentially read photoacoustic images corresponding to photoacoustic waves acquired over a relatively long time range in accordance with the progress of processing. The time range may be set, for example, from 40 seconds to 2 minutes.

Subsequently, the image processing device 1300 extracts a region where lymph exists from each of the read time-series photoacoustic images. As an example of an extraction method, the image processing device 1300 detects changes in luminance values between time-series photoacoustic images in view of the fact that lymph circulation occurs intermittently or periodically due to contraction of lymphatic vessels. , there is a method of judging that a portion with a large change in the luminance value is a lymph node. Note that the time range and criteria for judging whether or not it is a lymphatic area are examples, and are appropriately determined according to the condition of the lymphatic vessels in the subject and the conditions related to the contrast medium and light irradiation. For example, when the predetermined time range is 1 minute, if an area having a brightness value that is more than half that of a typical blood vessel is observed for 5 seconds in 1 minute, the area is identified as a lymph node. can be judged.
Note that when image processing is performed on the spectroscopic image to extract a lymph region, the image processing apparatus 1300 distinguishes between a region corresponding to blood and a region corresponding to a contrast medium, for example, based on the oxygen saturation value. , the lymphatic area may be extracted.

(S720: Step of acquiring lymph flow information based on lymph area)
The image processing device 1300 calculates lymphatic flow information based on the extracted lymphatic area information. In this embodiment, one three-dimensional photoacoustic image (one frame of volume data) is generated using photoacoustic signals derived from photoacoustic waves generated substantially simultaneously by one light irradiation of the subject. be done. Therefore, subject information at each position of a photoacoustic image of one frame is derived from one light irradiation. That is, in one frame of the photoacoustic image, the lymph nodes are depicted at approximately the same timing. As a result, according to the image generation method according to the present embodiment, it is possible to improve the accuracy when the image processing apparatus 1300 acquires lymphatic flow information from time-series three-dimensional image data. Also, changes in image values at a plurality of positions in time-series three-dimensional image data represent temporal changes in image values at substantially the same timing at each of the plurality of positions. Therefore, according to the image generation method according to the present embodiment, accuracy is improved when flow information is acquired based on image values at a plurality of positions in time-series three-dimensional image data.
As flow information, for example, the image processing apparatus 1300 can calculate the frequency of luminance value changes per unit time. In this case, the image processing apparatus 1300 may calculate the number of luminance value peaks in the lymph region or the number of times the luminance value exceeds a predetermined threshold within a unit time (for example, 10 minutes).
As another example of flow information, the image processing device 1300 may calculate the moving speed of lymph. The moving speed of the lymph fluid can be obtained by calculating the moving distance of the lymph region between the photoacoustic images obtained at different timings. When calculating the moving speed, the image processing apparatus 1300 calculates the centroid of the lymphatic region and the particle density based on the luminance value distribution of the extracted lymphatic region, and performs weighting based on these values to obtain the moving speed. You may improve the precision of calculation. It is also possible to use an optical flow estimation technique for extracting and vectorizing the motion of an object from time-series images.
As another example of the flow information, the image processing device 1300 may calculate the volume and flow rate of lymph flowing through the lymphatic vessels. At that time, the image processing apparatus 1300 can perform system-dependent correction on the width of the lymphatic vessel in the photoacoustic image to obtain the diameter of the lymphatic vessel, and can calculate the volume and flow rate based on the value of the diameter. . Further, when measuring the flow velocity, the number of times of reconstruction averaging at the time of photoacoustic image generation may be reduced to a certain number or less.

Further, the image processing apparatus 1300 may set a plurality of associated regions in the image and acquire the flow information based on the change in luminance value of each of the plurality of regions. A plurality of regions here means at least two or more small regions, more preferably a plurality of adjacent or close small regions. When setting the small region, it is preferable to include at least part of the lymphatic region extracted in the above step. More preferably, a plurality of subregions are set at adjacent or close positions on the same lymphatic vessel (in other words, upstream and downstream of the same lymphatic vessel).
Here, as shown in FIG. 11(a), it is assumed that both of the two subregions are set to include the same lymphatic vessel. The image processing device 1300 calculates the luminance value of each of the first small region 2310a and the second small region 2310b three times in chronological order, compares each luminance value with a predetermined threshold value, and determines the light/ determine darkness. Then, when it is determined that the first small area is "bright→dark→dark" and the second small area is "dark→bright→dark", the image processing device 1300 starts from the first timing. It can be determined that the lymph flowed in the direction from the first small region to the second small region over the second timing. The image processing device 1300 can also calculate the flow velocity of the lymph based on the distance between the two small regions and the temporal relationship of luminance changes in each small region.
As the luminance value of the small region, any value that can reflect the density of the contrast medium, such as an average luminance value or a peak luminance value, can be used. Moreover, if the number of calculations of the luminance value is at least two, the flow can be detected.

In addition, the image processing apparatus 1300 can also calculate the flow velocity of lymph based on the degree of temporal change in the luminance value obtained by calculating the luminance change in a specific region, which is a specific lymph region in the image, in time series. can. That is, it can be said that the luminance value in a specific region reflects the volume of lymph existing in the specific region. Therefore, the time change of the luminance value in the specific region reflects the change in the volume of the lymph from the inflow of the lymph to the outflow of the specific region. Therefore, if the change in luminance value can be calculated at a sufficient frame rate, when the time change in luminance value is expressed as a function of time and luminance value, the function tends to increase, and the function tends to decrease during the period when lymph flows out. Therefore, the image processing apparatus 1300 uses the slope of the period (broken line 3110) that is an increasing function and the slope of the period (broken line 3120 ), the flow velocity of the lymph may be calculated. Since lymph generally flows intermittently, it can be assumed that a luminance waveform such as that shown in FIG. 11(b) appears periodically. Therefore, lymphatic flow information may be obtained by measuring the appearance frequency of this waveform.

The image processing apparatus 1300 can also quantitatively acquire the flow velocity of the lymph by another calculation method based on information extracted from time-series three-dimensional image data. Here, the image in FIG. 11(c) shows one of the three-dimensional image data acquired in time series. However, in this process, an image in which lymphatic vessels have already been emphasized may be used. First, the image processing device 1300 selects a region 2320 corresponding to a lymphatic vessel.
Subsequently, the image processing apparatus 1300 determines a target time range (for example, one minute) from the time-series three-dimensional image data. Then, in the determined time range, a representative value of luminance values in the time direction is determined for each position in the length direction on the region 2320 . Here, the maximum value of the luminance value function in the determined time range is used as the representative luminance value. FIG. 11(d) is a graph plotting the maximum value 3200 obtained above, with time as the horizontal axis and each position in the length direction of the region 2320 as the vertical axis. This graph shows that the position showing high brightness also moves according to the movement of lymph within the lymphatic vessel.
Subsequently, the image processing apparatus 1300 obtains an approximate function 3210 by performing fitting based on the plotted multiple maximum values 3200, and calculates the lymph flow velocity based on the slope of the approximate function 3210. FIG. Note that it is not always necessary to use the maximum value. For example, a graph may be created by displaying a color scale according to luminance values. For example, the method of least squares can be used as a fitting method for calculating the slope, but the method is not limited to this.
The time-series three-dimensional image data used to acquire the flow information may include three-dimensional image data generated by light pulses of at least two wavelengths. In this case, the flow information may be calculated from the temporal change of the time-series three-dimensional image data corresponding to at least two wavelengths.

The image processing device 1300 then stores the calculated flow information in the storage device 1200 . By the processing up to this point, the effect that the system according to the present embodiment can calculate the flow information of the lymph in the subject can be obtained.

(S730: Step of displaying acquired flow information)
Next, a method of displaying flow information will be described as an example of a method of using lymphatic flow information. Note that the processes from step S730 to step S750 may be executed as a series of processes, or each may be executed independently.
The image processing device 1300 as display control means generates image data using the flow information and causes the display device 1400 to display it. The display method is arbitrary, but a method that facilitates confirmation of the flow information by the user is preferable.

As an example of the display method, the image processing device 1300 can color the extracted lymphatic region with a predetermined color scale according to the flow frequency obtained in S720. For example, a color scale may be used in which a lymphatic area with a higher frequency of flow is displayed in a red color, and a lymphatic area with a lower frequency is displayed in a blue color. According to this method, it is possible to provide a criterion for judging whether or not a user has a certain lymphatic vessel flow is active. As another example of the display method, the image processing device 1300 may display the luminance of a certain lymphatic region with a luminance value corresponding to the maximum luminance value in the time-series photoacoustic images. Moreover, the display using the above color scale and the display using the maximum luminance value may be used together.
As another example of the display method, the system accepts position designation by the user via the input device 1500, and the image processing device 1300 causes the display device 1400 to separately display the calculation result of the flow information at the designated position. good too.
The system may also present the flow information to the user in the form of text, voice, etc. instead of or in addition to the image information. The image processing device 1300 may also display the direction of lymph flow using markers, arrows, or the like so that the user can easily understand. The image processing apparatus 1300 may also set a region of interest in the lymph region, calculate and display the amount of temporal variation in the lymph in the region of interest.

(S740: Step of saving acquired flow information data)
Next, as another example of the method of using the lymph flow information, a method of saving the acquired flow information in a format that is easy for the user to utilize will be described. As described above, the time during which lymph flows in lymphatic vessels is generally relatively short. Therefore, even if a plurality of photoacoustic images or spectroscopic images are acquired in time series, not all images are necessary for the user to confirm the flow of lymph. Therefore, in this step, the image processing apparatus 1300 selects images necessary for confirmation by the user and stores them in the storage device 1200 . As a method of selecting an image, for example, there is a method of comparing the luminance in the lymphatic region of temporally adjacent images, and storing in the storage device 1200 when there is a luminance change greater than or equal to a value at which it can be determined that there is a lymphatic flow. be. According to this method, an image to be checked by the user can be quickly displayed on the display device 1400 . In addition, since it is not necessary to save the data during the period when the lymph flow cannot be confirmed, the data capacity can be reduced. Using the data compressed and stored in this way, the image processing device 1300 generates a still image showing the flow of lymph and a moving image showing the flow of lymph repeatedly in a loop, and displays them on the display device 1400. be able to.

Incidentally, in the same way that a spectroscopic image can be used to extract a lymph region, oxygen saturation information of a spectroscopic image can also be used when selecting data with flow information in this step.
In this step, the image data from which it can be determined that there was a lymphatic flow may be discriminated by a method such as adding metadata. This method also enables the display device 1400 to quickly display an image to be checked by the user.

(S750: Step of executing diagnosis support by the user)
Next, as another example of a method of using lymphatic flow information, a method of presenting information to the user based on the acquired flow information and assisting diagnosis will be described. In this step, the image processing device 1300 calculates diagnostic support information based on the information acquired in each step and displays it on the display device 1400 . As an example, the image processing device 1300 quantitatively analyzes lymphatic flow information, and based on the results, calculates and presents information for diagnosing the stage of lymphedema or diabetes. The presented information includes, for example, stage suggestions and estimated values. The image processing device 1300 reads, for example, a table showing the relationship between the flow frequency, flow rate, flow velocity, and other values and the stage of lymphedema from the storage device 1400, and compares it with the flow information calculated in the above step to obtain presentation information. Calculate In addition, the image processing device 1300 compares the results of actual diagnosis by the user with the estimated stages, and updates the data for estimation based on the results of correctness/incorrectness judgments, thereby increasing the accuracy of estimation. It is also preferable to provide such a learning function.

As described above, the system according to this embodiment can acquire flow information in the lymph by executing the processing of each step in FIG. In addition, it is possible to display the flow information to the user as needed, save the flow information in a preferred format, and generate and present information for supporting the user based on the flow information.

(S800: Step of displaying spectral image)
The image processing device 1300 as display control means causes the display device 1400 to display the spectral image so that the region corresponding to the contrast agent and the other region can be distinguished. As a rendering method, any method such as maximum intensity projection (MIP), volume rendering, and surface rendering can be adopted. Here, the setting conditions such as the display area and line-of-sight direction when rendering the three-dimensional image two-dimensionally can be arbitrarily specified according to the observation target.

Here, a case will be described where a spectral image is generated according to Equation (1) using photoacoustic waves obtained by setting 797 nm and 835 nm as wavelengths of irradiation light. As shown in FIG. 8, when these two wavelengths are selected, the image value corresponding to the contrast agent in the spectroscopic image generated according to equation (1) is negative regardless of the concentration of the ICG. .

As shown in FIG. 10, the image processing device 1300 causes the GUI to display a color bar 2400 as a color scale indicating the relationship between the image values of the spectral images and the display colors. The image processing apparatus 1300 determines the numerical range of image values to be assigned to the color scale based on information about the contrast agent (for example, information indicating that the type of contrast agent is ICG) and information indicating the wavelength of the irradiation light. may decide. For example, the image processor 1300 may determine a numerical range that includes negative image values corresponding to arterial oxygen saturation, venous oxygen saturation, and contrast agents. The image processing apparatus 1300 may determine the numerical range of -100% to 100% and set the color bar 2400 by assigning -100% to 100% to the color gradation that changes from blue to red. With such a display method, in addition to identifying arteries and veins, regions corresponding to negative contrast agents can also be identified. Further, the image processing apparatus 1300 may display an indicator 2410 indicating the numerical range of image values corresponding to the contrast agent based on the information regarding the contrast agent and the information indicating the wavelength of the irradiation light. Here, in the color bar 2400, an indicator 2410 indicates a negative value area as the numerical range of the image values corresponding to the ICG. By displaying the color scale so that the display color corresponding to the contrast agent can be identified in this manner, the region corresponding to the contrast agent in the spectral image can be easily identified.

The image processing device 1300 as region determining means may determine the region corresponding to the contrast agent in the spectral image based on information about the contrast agent and information indicating the wavelength of the irradiation light. For example, the image processing apparatus 1300 may determine a region having a negative image value in the spectral image as a region corresponding to the contrast medium. Then, the image processing device 1300 may cause the display device 1400 to display the spectral image so that the region corresponding to the contrast agent and the other region can be distinguished. The image processing apparatus 1300 makes the area corresponding to the contrast agent different in display color from the other areas, blinks the area corresponding to the contrast agent, and displays an indicator (for example, a frame) indicating the area corresponding to the contrast agent. It is possible to adopt an identification display such as displaying.

Note that it may be possible to switch to a display mode for selectively displaying image values corresponding to the ICG by designating an item 2730 corresponding to the display of the ICG displayed on the GUI shown in FIG. For example, when the user selects the item 2730 corresponding to the display of ICG, the image processing device 1300 selects voxels with negative image values from the spectral image, and selectively renders the selected voxels. Regions of the ICG may be selectively displayed. Similarly, the user may select item 2710 corresponding to the display of arteries and item 2720 corresponding to the display of veins. Based on a user's instruction, the image processing device 1300 selectively selects an image value corresponding to arteries (eg, 90% or more and 100% or less) or an image value corresponding to veins (eg, 60% or more and less than 90%). You may switch to the display mode to display. The numerical ranges of the image values corresponding to arteries and the image values corresponding to veins may be changeable based on user instructions.

At least one of hue, brightness, and saturation is assigned to the image value of the spectral image, and the image obtained by assigning the remaining parameters of hue, brightness, and saturation to the image value of the photoacoustic image is displayed as a spectral image. may For example, an image obtained by assigning hue and saturation to the image value of the spectral image and assigning brightness to the image value of the photoacoustic image may be displayed as the spectral image. At this time, if the image value of the photoacoustic image corresponding to the contrast agent is larger or smaller than the image value of the photoacoustic image corresponding to the blood vessel, assigning the brightness to the image value of the photoacoustic image will give the blood vessel and the contrast agent It may be difficult to see both Therefore, the image value-to-brightness conversion table of the photoacoustic image may be changed according to the image value of the spectral image. For example, if the image values of the spectral image are included in the numerical range of the image values corresponding to the contrast agent, the brightness corresponding to the image values of the photoacoustic image may be made smaller than that corresponding to the blood vessel. That is, if the image values of the photoacoustic image are the same when the contrast agent region and the blood vessel region are compared, the brightness of the contrast agent region may be lower than that of the blood vessel region. Here, the conversion table is a table indicating brightness corresponding to each of a plurality of image values. Further, when the image value of the spectral image is included in the numerical range of the image value corresponding to the contrast agent, the brightness corresponding to the image value of the photoacoustic image may be made higher than that corresponding to the blood vessel. That is, if the image values of the photoacoustic image are the same when the contrast agent region and the blood vessel region are compared, the brightness of the contrast agent region may be made higher than that of the blood vessel region. Further, the numerical range of the image values of the photoacoustic image in which the image values of the photoacoustic image are not converted into brightness may differ depending on the image value of the spectral image.

The conversion table may be changed to one suitable for the type and concentration of the contrast agent and the wavelength of the irradiation light. Therefore, the image processing apparatus 1300 may determine a conversion table for converting the image value of the photoacoustic image into the brightness based on the information about the contrast agent and the information indicating the wavelength of the irradiation light. When the image value of the photoacoustic image corresponding to the contrast agent is estimated to be larger than that corresponding to the blood vessel, the image processing device 1300 sets the brightness corresponding to the image value of the photoacoustic image corresponding to the contrast agent to the blood vessel. It may be smaller than the corresponding one. Conversely, when the image value of the photoacoustic image corresponding to the contrast agent is estimated to be smaller than that corresponding to the blood vessel, the image processing device 1300 sets the brightness corresponding to the image value of the photoacoustic image corresponding to the contrast agent. may be larger than that corresponding to the blood vessel.

The GUI shown in FIG. 10 includes an absorption coefficient image (first photoacoustic image) 2100 corresponding to a wavelength of 797 nm, an absorption coefficient image (second photoacoustic image) 2200 corresponding to a wavelength of 835 nm, and an oxygen saturation image (spectral image) 2300. display. The GUI may indicate which wavelength of light is used to generate each image. Although both the photoacoustic image and the spectral image are displayed in this embodiment, only the spectral image may be displayed. Further, the image processing apparatus 1300 may switch between display of the photoacoustic image and display of the spectral image based on the user's instruction.

Note that the display unit 160 may be capable of displaying moving images. For example, the image processing device 1300 generates at least one of the first photoacoustic image 2100, the second photoacoustic image 2200, and the spectral image 2300 in time series, and generates moving image data based on the generated time-series images. It may be configured to generate and output to the display unit 160 . In view of the fact that the number of times that lymph flows is relatively small, it is also preferable to display as a still image or a time-compressed moving image in order to shorten the user's judgment time. In addition, in moving image display, it is also possible to repeatedly display how lymph flows. The moving image speed may be a predetermined speed defined in advance or a predetermined speed specified by the user.
Here, the present embodiment is a method for displaying an image obtained by repeatedly irradiating a light pulse and acquiring substantially continuously a three-dimensional image of the light absorber distribution of the subject in a specific region of the subject,
It can also be implemented as an image display method for repeatedly reproducing and displaying a series of continuously acquired images at a predetermined speed.
At that time, the material flow information of the specific region can be displayed on the same screen in association with the specific region at least by luminance display, color display, graph display, and/or numerical display.
Also, at least one of the specific regions can be highlighted.
Also, when displaying flow information as a moving image, fast-forward display may be enabled.

Also, it is preferable that the frame rate of the moving image is made variable in the display unit 160 capable of displaying the moving image. In order to make the frame rate variable, a window for the user to manually input the frame rate, a slide bar for changing the frame rate, or the like may be added to the GUI of FIG. Here, since the lymph intermittently flows through the lymphatic vessel, only a part of the obtained time-series volume data can be used to confirm the flow of the lymph. Therefore, if the real-time display is performed when confirming the flow of lymph, the efficiency may decrease. Therefore, by making the frame rate of the moving image displayed on the display unit 160 variable, it is possible to fast-forward the displayed moving image, so that the user can check the state of the fluid in the lymphatic vessel in a short time. Become.

Moreover, the display unit 160 may be capable of repeatedly displaying moving images within a predetermined time range. At that time, it is also preferable to add a GUI such as a window or a slide bar to FIG. 10 so that the user can specify the range of repeated display. This makes it easier for the user to grasp how the fluid flows in the lymphatic vessels, for example.

The method of displaying the flow information in the specific area is not limited to the above. For example, the image processing device 1300 as display control means associates the flow information in the specific region with the specific region, and displays it on the display device 1400 by at least one of brightness display, color display, graph display, and numerical display. They may be displayed on the same screen. Further, the image processing device 1300 as display control means may highlight at least one of the specific regions.

As described above, at least one of the image processing device 1300 and the computer 150 as an information processing device includes spectral image acquisition means, contrast agent information acquisition means, region determination means, photoacoustic image acquisition means, and display control means. It functions as a device with at least one. Note that each means may be composed of different hardware, or may be composed of one piece of hardware. Moreover, a plurality of means may be configured by one piece of hardware.

In this embodiment, by selecting a wavelength at which the image value corresponding to the contrast agent is a negative value, it is possible to distinguish between the blood vessel and the contrast agent. The image value corresponding to the contrast agent can be any value as long as it can identify . For example, the image processing described in this step can be applied even when the image value of the spectral image (oxygen saturation image) corresponding to the contrast agent is smaller than 60% or larger than 100%. .

In this embodiment, an example of applying image processing to a spectral image based on photoacoustic images corresponding to a plurality of wavelengths is described, but the image processing according to this embodiment is applied to a photoacoustic image corresponding to one wavelength. You may That is, the image processing apparatus 1300 determines a region corresponding to the contrast agent in the photoacoustic image based on the information about the contrast agent, and distinguishes between the region corresponding to the contrast agent and the region other than the region. A photoacoustic image may be displayed. In addition, the image processing apparatus 1300 may display a spectral image or a photoacoustic image so that a region having a numerical range of image values corresponding to a preset contrast agent and other regions can be identified. .

In the present embodiment, an example in which the computer 150 as an information processing apparatus generates a spectral image by irradiating light of a plurality of wavelengths has been described, but when generating a photoacoustic image by irradiating light of only one wavelength Alternatively, the wavelength may be determined by the wavelength determination method according to this embodiment. That is, computer 150 may determine the wavelength of the illuminating light based on information about the contrast agent. In this case, the computer 150 preferably determines a wavelength such that the image values of the contrast agent region in the optoacoustic image are distinguishable from the image values of the blood vessel region.

Note that the light irradiation unit 110 may irradiate the subject 100 with light having a wavelength set in advance so that the image value of the contrast agent region in the photoacoustic image and the image value of the blood vessel region can be distinguished. good. In addition, the light irradiation unit 110 may irradiate the subject 100 with light of a plurality of wavelengths set in advance so that the image value of the contrast medium region and the image value of the blood vessel region in the spectral image can be distinguished. good.

(Other examples)
The present invention is also realized by executing the following processing. That is, the software (program) that realizes the functions of the above-described embodiments is supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads the program. This is the process to be executed.

1100 photoacoustic device 1300 image processing device

The present invention
irradiating means for irradiating a subject having a blood vessel and a lymphatic vessel and having a light absorber introduced into the lymphatic vessel with light of a plurality of wavelengths;
a detecting means for detecting photoacoustic waves from the subject corresponding to the lights of the plurality of wavelengths;
generating means for generating spectral image data based on a plurality of received signals corresponding to the detected plurality of photoacoustic waves;
display control means for displaying the spectral image data;
with
Among the plurality of wavelengths of light irradiated by the irradiation means, the plurality of wavelengths is in the range of 700 to 1100 nm, the first light has a wavelength shorter than 820 nm in the plurality of wavelengths, and the second light has a wavelength shorter than 820 nm. The wavelength is a wavelength of 820 nm or more,
The display control means displays the spectroscopic image data such that the blood vessel and the lymphatic vessel into which the light absorber is introduced can be distinguished.
To provide a photoacoustic imaging system characterized by :
The present invention also provides
an irradiation step of irradiating a subject having a blood vessel and a lymphatic vessel and having a light absorber introduced into the lymphatic vessel with light of a plurality of wavelengths;
a detection step of detecting photoacoustic waves from the subject corresponding to the lights of the plurality of wavelengths;
a generating step of generating spectroscopic image data based on a plurality of received signals corresponding to the detected plurality of photoacoustic waves;
a display control step of displaying the spectral image data;
has
In the plurality of wavelengths of light irradiated in the irradiation step, the plurality of wavelengths are in the range of 700 to 1100 nm, the wavelength of the first light in the plurality of wavelengths is shorter than 820 nm, and the wavelength of the second light is The wavelength is a wavelength of 820 nm or more,
In the display control step, the spectroscopic image data is displayed such that the blood vessel and the lymphatic vessel into which the light absorber is introduced can be distinguished.
A control method for a photoacoustic imaging system characterized by :
The present invention also provides
A program characterized by causing a computer to function as each means of the photoacoustic imaging system described above is provided.

In addition, in the present invention, when spectroscopic images are generated using light of a plurality of wavelengths different from each other and lymphatic flow information is acquired based on the spectroscopic images, the examination order information does not need to include information on each wavelength. be. When using multiple wavelengths, in setting the wavelengths, when an oxygen saturation image is generated as a spectral image according to formula (1), the blood vessel region in the spectral image is an image corresponding to the actual oxygen saturation While the values are calculated, it is preferable to take into account that the image values of the region of the contrast agent in the spectroscopic image change greatly depending on the wavelength used and the absorption coefficient spectrum of the contrast agent. That is, in order to facilitate understanding of the three-dimensional structure of the contrast agent, it is possible to use a wavelength that makes the image value of the contrast agent region in the spectroscopic image distinguishable from the image value of the blood vessel region. preferable. Specifically, when generating an image using formula (1) as a spectral image, the oxygen saturation of arteries and veins is generally within the range of 60% to 100%, and contrast enhancement in the spectral image is used. The value of formula (1) corresponding to the agent is 60%
It is preferable to use two wavelengths that are smaller (eg negative) or larger than 100%. For example, when ICG is used as a contrast agent, two wavelengths of 700 nm or more and shorter than 820 nm and a wavelength of 820 nm or more and 1020 nm or less are selected, and a spectral image is generated by formula (1). and blood vessel regions can be distinguished well. Thus, using wavelengths of 700 nm to 820 nm and 820 nm to 1020 nm is a preferred example of this embodiment.

Although the wavelength is determined based on the information about the contrast medium, the absorption coefficient of hemoglobin may be taken into consideration in determining the wavelength. FIG. 9 shows spectra of the Molar absorption coefficient of oxyhemoglobin (dashed line) and the Molar absorption coefficient of deoxyhemoglobin (solid line). In the wavelength range shown in FIG. 9, the magnitude relationship between the Molar absorption coefficient of oxyhemoglobin and the Molar absorption coefficient of deoxyhemoglobin is reversed at 797 nm. That is, it can be said that veins can be easily recognized at wavelengths shorter than 797 nm, and arteries can be easily recognized at wavelengths longer than 797 nm. By the way, in the treatment of lymphedema, lymphovenous anastomosis is used to form a bypass between a lymphatic vessel and a vein. For this preoperative examination, it is conceivable to image both veins and lymphatic vessels with accumulated contrast medium by photoacoustic imaging. In this case, by setting at least one of the plurality of wavelengths to a wavelength shorter than 797 nm, veins can be imaged more clearly. Further, it is advantageous for imaging veins to set at least one of the plurality of wavelengths to a wavelength at which the Molar absorption coefficient of deoxyhemoglobin is larger than that of oxyhemoglobin. In addition, when a spectral image is generated from a photoacoustic image corresponding to two wavelengths, both of the two wavelengths are set to wavelengths in which the Molar absorption coefficient of deoxyhemoglobin is larger than the Molar absorption coefficient of oxyhemoglobin. is advantageous. By selecting these wavelengths, it is possible to accurately image both the lymphatic vessel into which the contrast medium has been introduced and the vein in the preoperative examination of the lymphatic venule anastomosis.

Claims (54)

Time-series three-dimensional image data including three-dimensional image data corresponding to each of the plurality of light irradiations, generated based on received signals of photoacoustic waves generated by the plurality of light irradiations to the subject. An image processing device that processes
An image processing apparatus, comprising flow information acquisition means for acquiring flow information of a light absorber in the subject based on the time-series three-dimensional image data. 2. The image processing apparatus according to claim 1, wherein said flow information acquisition means acquires flow information of said light absorber in a specific region within said subject. The flow information acquisition means extracts the specific region containing the light absorber from an image based on the time-series three-dimensional image data, and acquires the flow information based on a change in luminance value of the specific region. 3. The image processing apparatus according to claim 2, wherein: The flow information acquisition means selects a plurality of three-dimensional image data corresponding to a predetermined time range from the time-series three-dimensional image data, and selects between a plurality of images based on the selected plurality of three-dimensional image data. 4. The image processing apparatus according to claim 2, wherein said flow information is acquired based on a change in luminance value of said specific area in said step. 5. The image processing apparatus according to claim 4, wherein said flow information acquiring means acquires said flow information based on the number of times the luminance value of said specific region exceeds a predetermined threshold within said time range. 6. The image processing apparatus according to claim 4, wherein said flow information acquiring means acquires a flow velocity of said light absorber in said specific region as said flow information. The flow information acquiring means sets a plurality of regions including at least a part of the specific region in a plurality of images based on the time-series three-dimensional image data, and based on changes in luminance values in the plurality of regions, 7. The image processing apparatus according to claim 6, wherein the flow velocity is acquired. The flow information acquisition means obtains, for each position in the length direction, from a plurality of images based on the time-series three-dimensional image data, when the direction in which the light absorber flows in the specific region is taken as the length direction. 7. The image processing apparatus according to claim 6, wherein the flow velocity is obtained by obtaining a representative value of luminance values of the light absorbers. The flow information acquisition means calculates a function indicating a temporal change in the luminance value of the light absorber in the specific region from a plurality of images based on the time-series three-dimensional image data, and uses the function to calculate the flow velocity. 7. The image processing apparatus according to claim 6, wherein 10. The image processing apparatus according to any one of claims 1 to 9, wherein the light absorber is a contrast agent introduced into the subject. 10. The image processing apparatus according to claim 9, wherein said flow information is information relating to the flow of lymph within said subject. The image processing apparatus according to any one of claims 1 to 11, wherein the time-series three-dimensional image data is photoacoustic image data generated using the received signal. 2. The time-series three-dimensional image data is spectroscopic image data generated using photoacoustic image data corresponding to each of a plurality of wavelengths of light with which the subject is irradiated. 12. The image processing apparatus according to any one of items 11 to 11. 14. The image processing apparatus according to claim 13, wherein the light absorber is a contrast agent introduced into the subject, and the plurality of wavelengths include 797 nm and 835 nm. 15. The image processing apparatus according to any one of claims 1 to 14, further comprising display control means for displaying the flow information on a display device. The display control means displays on the display device an image obtained by coloring the image based on the time-series three-dimensional image data using a color scale based on the flow information of the light absorber. 16. The image processing apparatus according to claim 15. 17. The image processing apparatus according to claim 15, wherein said display control means repeatedly displays a moving image using a plurality of images based on said time-series three-dimensional image data on said display device. 15. The display control means associates the flow information with the specific area and displays it on the same screen by at least one of brightness display, color display, graph display, and numerical display. 18. The image processing apparatus according to any one of items 1 to 17. 19. The image processing apparatus according to any one of claims 15 to 18, wherein said display control means highlights at least one of said specific areas. 18. The image processing apparatus according to claim 17, wherein said display control means fast-forwards and displays said moving image. the flow information acquisition means generates information for supporting a user's decision based on the flow information;
21. The image processing apparatus according to claim 15, wherein said display control means displays said information on said display device. Time-series three-dimensional image data including three-dimensional image data corresponding to each of the plurality of light irradiations, generated based on received signals of photoacoustic waves generated by the plurality of light irradiations to the subject. An image processing method for processing
An image processing method, comprising: a flow information acquisition step of acquiring flow information of a light absorber in the subject based on the time-series three-dimensional image data. 23. The image processing method according to claim 22, wherein in said flow information acquisition step, flow information of said light absorber in a specific region within said subject is acquired. In the flow information obtaining step, the specific region including the light absorber is extracted from an image based on the time-series three-dimensional image data, and the flow information is obtained based on a change in luminance value of the specific region. 23. The image processing method according to claim 22, wherein: In the flow information acquiring step, a plurality of three-dimensional image data corresponding to a predetermined time range are selected from the time-series three-dimensional image data, and a plurality of images based on the selected plurality of three-dimensional image data are selected. 25. The image processing method according to claim 23, wherein said flow information is acquired based on a change in luminance value of said specific region in . 26. The image processing method according to claim 25, wherein in said flow information acquisition step, said flow information is acquired based on the number of times the luminance value of said specific region exceeds a predetermined threshold within said time range. . 27. The image processing method according to claim 25, wherein in said flow information acquiring step, a flow velocity of said light absorber in said specific region is acquired as said flow information. In the flow information obtaining step, a plurality of regions including at least a portion of the specific region are set in a plurality of images based on the time-series three-dimensional image data, and based on changes in luminance values in the plurality of regions, 28. The image processing method of claim 27, wherein the flow velocity is obtained. In the flow information acquisition step, when the direction in which the light absorber flows in the specific region is taken as the length direction, from a plurality of images based on the time-series three-dimensional image data, for each position in the length direction 28. The image processing method according to claim 27, wherein the flow velocity is obtained by obtaining a representative value of luminance values of the light absorbers. In the flow information obtaining step, a function indicating a temporal change in the luminance value of the light absorber in the specific region is calculated from a plurality of images based on the time-series three-dimensional image data, and the flow velocity is calculated using the function. 28. The image processing method of claim 27, wherein is obtained. 31. The image processing method according to any one of claims 22 to 30, wherein the light absorber is a contrast agent introduced into the subject. 31. The image processing method according to claim 30, wherein said flow information is information relating to the flow of lymph within said subject. 33. The image processing method according to any one of claims 22 to 32, wherein the time-series three-dimensional image data is photoacoustic image data generated using the received signal. 22. The time-series three-dimensional image data is spectroscopic image data generated using photoacoustic image data corresponding to each of a plurality of wavelengths of light with which the subject is irradiated. 33. The image processing method according to any one of items 1 to 32. 35. The image processing method of claim 34, wherein the light absorber is a contrast agent introduced into the subject, and the plurality of wavelengths includes 797 nm and 835 nm. 36. The image processing method according to any one of claims 22 to 35, further comprising a display control step of displaying the flow information on a display device. In the display control step, the image based on the time-series three-dimensional image data is colored using a color scale based on the flow information of the light absorber, and the image is displayed on the display device. 37. The image processing method according to claim 36, wherein 38. The image processing method according to claim 36, wherein in said display control step, a moving image using a plurality of images based on said time-series three-dimensional image data is repeatedly displayed on said display device. . In the display control step, the flow information is associated with the specific area and displayed on the same screen by at least one of brightness display, color display, graph display, and numerical display. Item 39. An image processing method according to any one of Items 36 to 38. 40. The image processing method according to any one of claims 36 to 39, wherein in said display control step, at least one said specific region is highlighted. 39. The image processing method according to claim 38, wherein, in said display control step, said moving image is displayed in fast forward. in the flow information obtaining step, information for supporting a user's decision is generated based on the flow information;
42. The image processing method according to any one of claims 36 to 41, wherein in said display control step, said information is displayed on said display device. The flow information is the light absorption between the two wavelengths in a specific region in the subject obtained by irradiating a light absorber in the subject with light pulses of at least two wavelengths having different light absorption coefficients. 43. The image processing method according to any one of claims 22 to 42, wherein the image is calculated from a temporal change of an image of the body. 44. The image processing method according to any one of claims 22 to 43, wherein said light absorber corresponds to a contrast medium administered into said subject. 45. The image processing method according to claim 44, wherein said contrast agent is indocyanine green. 44. An image processing method according to claim 43, wherein said change over time is an increasing function. 44. An image processing method according to claim 43, wherein said change over time is a decreasing function. 48. An image processing method according to any one of claims 43, 46 and 47, wherein said time change is periodic. 44. The image processing method according to claim 43, wherein said two wavelengths are in the ranges of 700-820 nm and 820-1020 nm, respectively. A method for displaying an image obtained by repeatedly irradiating light pulses and obtaining substantially continuously three-dimensional images of a light absorber distribution of a subject in a specific region of the subject, the image displaying method comprising a series of continuously obtained images. is repeatedly reproduced and displayed at a predetermined speed. 51. The image according to claim 50, characterized in that the material flow information of the specific region is displayed on the same screen in association with the specific region, at least by luminance display, color display, graph display, and/or numerical display. Display method. 52. The image display method according to claim 50 or 51, wherein at least one of said specific regions is highlighted. 53. The image display method according to any one of claims 50 to 52, wherein a fast-forward display is possible when the flow information is displayed as a moving image. Time-series three-dimensional image data including three-dimensional image data corresponding to each of the plurality of light irradiations, generated based on received signals of photoacoustic waves generated by the plurality of light irradiations to the subject. A program for causing a computer to execute an image processing method for processing
The image processing method comprises a flow information acquisition step of acquiring flow information of a light absorber in the subject based on the time-series three-dimensional image data.

Images