JP7000139B2 - Image information processing device and display method - Google Patents

Description

The present invention relates to an image information processing apparatus and a display method.

There is a phenomenon called photoacoustic effect, in which acoustic waves (photoacoustic waves) are generated when a subject is irradiated with pulsed light generated from a light source. Photoacoustic imaging is a technique for imaging the inside of a subject using photoacoustic waves and acquiring a photoacoustic image (for example, a blood vessel image) that reflects the optical characteristics of the subject.

The subject inspection apparatus described in Patent Document 1 includes a hemispherical support provided with a plurality of conversion elements that receive acoustic waves generated in the subject and convert them into electrical signals. The subject is held by a thin cup-shaped holding member, and an acoustic matching medium such as water is arranged between the holding member and the conversion element. When light is emitted from below the support, it reaches the subject via the acoustic matching medium and the holding member, and a photoacoustic wave is generated. The conversion element provided on the support receives the photoacoustic wave via the holding member and the acoustic matching medium.

International Publication No. 2010/030817

Here, when the user compares the photoacoustic image with the subject, it may be difficult to grasp the positional relationship between the photoacoustic image and the actual subject. For example, when comparing the thigh and palm, the former is wider than the latter and has less features on the body surface. Therefore, it is relatively difficult to grasp where the photoacoustic image corresponds to the actual thigh.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for facilitating grasping the correspondence between a photoacoustic image and a subject.

The present invention adopts the following configuration. That is,
A photoacoustic image acquisition unit that acquires a photoacoustic image generated based on a photoacoustic wave generated by irradiating a subject and a marker with light, and a photoacoustic image acquisition unit.
An optical image acquisition unit that acquires an optical image generated by optically imaging the subject and the marker, and an optical image acquisition unit.
An image in which a superposed image of the photoacoustic image and the optical image is generated and displayed on the display unit based on the component derived from the marker in the photoacoustic image and the marker shown in the optical image. Processing unit and
Equipped with
The image processing unit is an image information processing apparatus characterized by reducing components derived from the marker from the superimposed image or the photoacoustic image.
The present invention also adopts the following configuration. That is,
A photoacoustic image acquisition unit that acquires a photoacoustic image generated based on a photoacoustic wave generated by irradiating a subject and a marker with light, and a photoacoustic image acquisition unit.
An optical image acquisition unit that acquires an optical image generated by optically imaging the subject and the marker, and an optical image acquisition unit.
An image in which a superposed image of the photoacoustic image and the optical image is generated and displayed on the display unit based on the component derived from the marker in the photoacoustic image and the marker shown in the optical image. Processing unit and
Equipped with
The image processing unit reduces components derived from the surface of the subject, including the marker, from the superimposed image or the photoacoustic image.
It is an image information processing device characterized by this.

The present invention also adopts the following configuration. That is,
A photoacoustic image acquisition step of acquiring a photoacoustic image generated based on a photoacoustic wave generated by irradiating a subject and a marker with light,
An optical image acquisition step of acquiring an optical image generated by optically imaging the subject and the marker, and
An image in which a superposed image of the photoacoustic image and the optical image is generated and displayed on the display unit based on the component derived from the marker in the photoacoustic image and the marker shown in the optical image. Processing steps and
Have ,
The image processing step is a display method characterized by reducing components derived from the marker from the superimposed image or the photoacoustic image.
The present invention also adopts the following configuration. That is,
A photoacoustic image acquisition step of acquiring a photoacoustic image generated based on a photoacoustic wave generated by irradiating a subject and a marker with light,
An optical image acquisition step of acquiring an optical image generated by optically imaging the subject and the marker, and
An image in which a superimposed image in which the photoacoustic image and the optical image are superimposed is generated based on the component derived from the marker in the photoacoustic image and the marker shown in the optical image and displayed on the display unit. Processing steps and
Have,
In the image processing step, components derived from the surface of the subject including the marker are reduced from the superimposed image or the photoacoustic image.
It is a display method characterized by that.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide a technique for facilitating grasping the correspondence between a photoacoustic image and a subject.

Conceptual diagram of the subject information acquisition device of Example 1 Flow diagram of processing of Example 1 The figure which shows the appearance that the marker was arranged in the target area of a subject The figure which shows an example of an optical image Diagram showing an example of a photoacoustic image Diagram showing an example of a superimposed image The figure which shows the appearance which the marker was arranged on the holding member of Example 2. The figure which shows an example of the superimposed image of Example 3. Conceptual diagram of the subject information acquisition device of Example 4 Conceptual diagram of composite image generation of Example 4 The figure which shows another example of the marker of Example 4.

A preferred embodiment of the present invention will be described below with reference to the drawings. However, the dimensions, materials, shapes and relative arrangements of the components described below should be appropriately changed depending on the configuration of the apparatus to which the invention is applied and various conditions. Therefore, the scope of the present invention is not intended to be limited to the following description.

The present invention relates to a technique for detecting an acoustic wave propagating from a subject and generating and acquiring characteristic information (subject information) inside the subject. Therefore, the present invention can be regarded as an acoustic wave device or a control method thereof, a subject information acquisition device or a control method thereof. The present invention is also regarded as an image information processing apparatus or a control method thereof, or an image processing method. The present invention is also regarded as a subject information acquisition method and a signal processing method. The present invention can also be regarded as a program for executing these methods in an information processing apparatus having hardware resources such as a CPU and a memory, and a non-temporary storage medium that stores the program and can be read by a computer.

The subject information acquisition device of the present invention has a photoacoustic effect of receiving acoustic waves generated in a subject by irradiating the subject with light (electromagnetic waves) and acquiring characteristic information of the subject as image data. Includes photoacoustic devices that utilize. In this case, the characteristic information is information on characteristic values corresponding to each of a plurality of positions in the subject, which is generated by using a signal derived from the received photoacoustic wave. For example, the characteristic information includes the source distribution of acoustic waves generated by light irradiation, the initial sound pressure distribution in the subject, the light energy absorption density distribution and absorption coefficient distribution derived from the initial sound pressure distribution, and the substances that make up the tissue. Shows the concentration distribution. The substance concentration distribution includes an oxygen saturation distribution, a total hemoglobin concentration distribution, an oxidized / deoxidized hemoglobin concentration distribution, and the like.

Further, the characteristic information which is the subject information at a plurality of positions may be acquired as a two-dimensional or three-dimensional characteristic distribution. The characteristic distribution can be generated as image data showing characteristic information in the subject.
The image data is generated as, for example, three-dimensional volume data by image reconstruction.

It is also possible to image the vascular structure inside the subject by emphasizing the portion showing the absorption coefficient characteristic of blood. At that time, the distribution of each of the arteries and veins can be imaged based on the oxygen saturation.

The acoustic wave referred to in the present invention is typically an ultrasonic wave, and includes a sound wave and an elastic wave called an acoustic wave. A signal converted from an acoustic wave by a transducer or the like (for example, an electric signal) is also referred to as an acoustic signal or a received signal. However, the description of ultrasonic waves or acoustic waves in the specification is not intended to limit the wavelength of elastic waves. The acoustic wave generated by the photoacoustic effect is called a photoacoustic wave or an optical ultrasonic wave. A signal derived from a photoacoustic wave (for example, an electric signal) is also called a photoacoustic signal. An image generated from a photoacoustic signal by image reconstruction or the like is also called a photoacoustic image.

By the way, at the time of flap collection surgery, photoacoustic measurement of the flap collection site is performed to obtain a photoacoustic image (blood vessel image), and the position and running state of the blood vessel are grasped from this blood vessel image. It is being considered to be useful in determining the position and range of flaps to be operated. At this time, the operator needs to know which part of the subject the blood vessel image corresponds to. However, for flap collection, a relatively wide area with few features on the body surface shape, such as the thigh, is usually used. Therefore, it may be difficult to identify the positional relationship between the blood vessel image and the actual subject.

Therefore, in each of the following examples, in order to facilitate understanding of the correspondence between the photoacoustic image and the subject, the operator is presented with an easy-to-understand location of the photoacoustic image in the subject. This enables the operator to grasp the position of the blood vessel (for example, the penetrating branch) in the actual subject based on the blood vessel image, and to collect the flap well.

<Example 1>
Here, a basic embodiment will be described. In this example, the marker is placed directly on the subject. FIG. 1 is a conceptual diagram of the subject information acquisition device 1 of this embodiment.

(Device configuration)
The subject 2 is held by the holding unit 200. An acoustic matching medium (not shown) may be filled between the subject 2 and the holding member 200 so that acoustic waves can be easily transmitted. Examples of the acoustic matching medium include water, gel, gel and the like. As the holding member, a material having an acoustic impedance close to that of the human body and having high transparency of light and acoustic waves is preferable. For example, polymethylpentene, polyethylene terephthalate, acrylic and the like are suitable. Further, in order to prevent a decrease in the S / N ratio due to multiple reflections of acoustic waves, a mesh-shaped member may be used. As the material of the mesh member, metal, fiber, resin, etc. can be used.

Further, a holding member 200 in which a mesh member and a film member or a sheet member are combined can also be used. In that case, a film or sheet-like member formed of resin or the like is arranged on the upper part (subject side) or lower part (conversion element side) of the mesh member. As a result, the subject can be held even if the thickness of the film member is reduced, so that the transmission of acoustic waves and light is enhanced. Further, since the acoustic matching medium on the subject side and the acoustic matching medium on the conversion element side can be separated by the film member, cleanliness and ease of replacement of the acoustic matching medium are improved. The skeleton member that supports the film is not limited to the mesh, and may be shaped like the bone of an umbrella, for example.

The light emitted from the light source 100 is emitted from the irradiation end 140 below the support via the optical transmission path 120. The optical transmission line 120 is composed of optical members such as optical fibers, mirrors, prisms, and lenses. When the irradiation light is absorbed inside the subject, on the surface of the subject, or by the light absorber on the holding member 200, a photoacoustic wave is generated. A plurality of conversion elements 320 supported by the support 300
Receives photoacoustic waves and converts them into electrical signals (photoacoustic signals).

The light source 100 irradiates light having a pulse width of about 1 to 100 nanoseconds in order to efficiently generate photoacoustic waves. When the subject is a living body, the wavelength of light is preferably about 600 nm to 1100 nm. As the type of the light source 100, an Nd: YAG laser, an alexandrite laser, a Ti-sa laser, and an OPO laser can be used. Further, LEDs, semiconductor lasers, and the like can also be used. Further, by using a light source 100 capable of irradiating light having a plurality of wavelengths such as a tunable laser, it is possible to obtain a substance concentration in the subject such as oxygen saturation.

The support 300 is a hemispherical member made of metal, resin, or the like. The plurality of conversion elements 320 are arranged on the support 300 so as to form a high-sensitivity region in which the directional axes are gathered. The structure of the holding member and the support is not limited to the above. For example, a plate-shaped holding member that presses and holds the subject may be used. Further, the conversion element may be a single element, or may be arranged linearly or planarly. An acoustic matching medium (not shown) capable of propagating an acoustic wave is provided between the holding member and the conversion element. Examples of the acoustic matching medium include water, gel, and gel.

The camera 500 is an optical image pickup device arranged on the conversion element side with respect to the subject. The camera 500 optically captures a target area for photoacoustic measurement using visible light, infrared light, or the like, and sends the obtained optical image as image data to an information processing apparatus.

The drive unit 600 controls a movement mechanism (not shown) composed of a drive stage or the like for moving the support relative to the subject. By performing photoacoustic measurement at a plurality of positions where the drive unit 600 has moved the support, a photoacoustic image of a wide area of the subject can be acquired. The drive unit 600 moves the support 300 on the moving surface below the subject by a method such as a spiral scan or a raster scan. Then, light irradiation and reception of photoacoustic waves are repeated at regular intervals. The drive unit transmits the irradiation position of the pulsed light to the reconstruction unit.

The signal acquisition unit 400 amplifies the photoacoustic signal sequentially output from the conversion element that has received the photoacoustic wave, converts it into a digital signal, and transmits it to the reconstruction unit 840. The signal acquisition unit 400 is composed of an amplifier, an AD converter, and the like.

The information processing apparatus 800 includes a control unit 810, an optical image acquisition unit 820, a reconstruction unit 840, an image processing unit 850, and a storage unit 870 as functional blocks. As the information processing apparatus 800, a computer or workstation equipped with computational resources such as a processor and a memory and operating according to a program instruction can be used. The information processing apparatus 800 does not necessarily have to have a specific configuration (processing circuit or the like) corresponding to each functional block. Each functional block may be virtually realized as a program module that performs each process. It is not always necessary that all the functional blocks are included in one computer, and a plurality of computers connected so as to be able to transmit and receive information may cooperate to form the information processing apparatus 800.

The control unit 810 depends on the operation of each configuration of the subject information acquisition device 1, the light irradiation timing of the light source 100, the signal acquisition timing by the conversion element 320, the moving position and moving speed of the support 300 by the driving unit 600, and the camera 500. It controls the timing of optical imaging and the imaging method. The optical image acquisition unit 820 receives image data from the camera 500, performs processing such as correction as necessary, and stores the image data in the storage unit.

The reconstruction unit 840 performs image reconstruction based on the photoacoustic signal from the signal acquisition unit 400 and the photoacoustic signal acquisition position information from the drive unit 600, and acquires the characteristic information distribution of the target region of the subject. Any method such as phasing addition method or Fourier transform method can be used for image reconstruction. The acquired characteristic information distribution is transmitted to the image processing unit 850. The reconstruction unit 840 corresponds to the photoacoustic image acquisition unit of the present invention. The image processing unit 850 generates a display image by a superposition process described later.

The storage unit 870 is a subject information acquisition device 1 such as control information by the control unit 810, photoacoustic signals, image data of an optical image, reconstructed characteristic information distribution data, and image data processed by the image processing unit 850 described later. Temporarily or permanently saves the data generated by the operation of, and outputs it as needed.

The input unit 880 is a means for a user (operator or the like) to input instruction information, and for example, a user interface (mouse, keyboard, touch panel, etc.) of the information processing apparatus 800 can be used. As the display unit 900, a display device such as a liquid crystal display or an organic EL display can be used.

(Processing flow)
The outline of the processing flow of this Example will be described with reference to FIG. In step S101, the user places the marker 250 at a predetermined location. The marker 250 is a light absorber that absorbs light and generates an acoustic wave. The location, type, and material of the marker will be described later. Then, the user sets parameters related to the operation of the device. Parameters include the target area for imaging in the subject, the path and speed of scanning by the drive unit 600, the intensity and interval of light irradiation during photoacoustic measurement, and the image quality of photoacoustic images and optical images. The original is included. Then, the user installs the subject on the holding member 200.

In step S102, the camera 500 performs optical imaging according to the parameters. Subsequently, the optical image acquisition unit 820 acquires an optical image. Photoacoustic measurement is performed in step S103. Specifically, the light source 100 irradiates light, and the conversion element 320 receives the photoacoustic wave and outputs a photoacoustic signal. The photoacoustic signal is stored in the storage unit 870.

In step S104, the control unit 810 determines whether or not the photoacoustic measurement of the target region is completed. If completed (YES), the process proceeds to S106. If not completed (NO), the process proceeds to step S105, the support 300 is scanned by the drive unit 600, and the photoacoustic measurement is performed at the next position. In step S106, the reconstruction unit 840 generates a photoacoustic image of the target area and stores it in the storage unit 870. In step S107, the image processing unit 850 generates superimposed image data and displays it on the display unit 900.

(Overlaying images)
The method of superimposing and displaying the photoacoustic image and the optical image will be described in the order of steps (1) to (4). In this embodiment, the marker 250, which is a light absorber, is placed directly on the subject.

(Step 1) Placing a marker This step corresponds to step S101. The user arranges a plurality of markers as shown in FIG. 3 in a target area 210 which is an imaging site for performing photoacoustic measurement and optical imaging and is a target of superimposed image generation.

As the marker 250, a material having absorption characteristics at the wavelength used for photoacoustic measurement is used. For example, black ink can be used. In that case, the user writes the marker directly on the subject using a pen using black ink. However, the marker 250 is not limited to this, and may be any one that is drawn in both a photoacoustic image and an optical image. As an example, a seal containing a light absorber such as carbon can be used. Further, carbon or the like may be attached to the subject with an adhesive. It is desirable that the intensity of the photoacoustic wave generated from the marker is the same as or about an order of magnitude smaller than the intensity of the photoacoustic wave generated from the imaging site. Considering that the absorption coefficient of hemoglobin, which is a typical substance that absorbs light in a subject, is from 0.3 / mm to 0.9 / mm, the absorption coefficient of the marker is, for example, 0.05 / mm or more. The absorption coefficient is preferably 0.0 / mm or less.

The shape of the marker 250 may be a line shape, a matrix shape, a radial shape, a circular shape, or the like, in addition to the dot shape, and can be selected according to the application and the size of the target area.

Further, instead of the marker arranged by the user, or together with the marker arranged by the user, a portion having a high light absorption degree provided in advance in the subject may be used as a marker. For example, it is a place where pigments such as moles are accumulated.

(Step 2) Optical image acquisition This step corresponds to step S102. The camera 500 adjusts so that all the markers 250 arranged in the target area 210 are included in the imaging field of view, and performs optical imaging. The optical image acquisition unit 820 receives the image data, performs correction processing if necessary, and stores the image data in the storage unit 870. FIG. 4 is a schematic view of an optical image (target area optical image 213). It is desirable that the marker (marker optical image 253) is clearly reflected in the optical image to the extent that alignment is possible. Further, in order for the user to understand which part of the subject the target area is, it is necessary to show structural features such as contour lines. In the case of the image of the thigh as shown in FIG. 4, if the left and right contour lines are reflected, the target area can be easily grasped based on the direction and the thickness.

(Step 3) Photoacoustic image acquisition This step corresponds to steps S103 and S106. As described above, the photoacoustic wave generated by the light irradiation from the light source 100 includes a component derived from the marker 250 as well as a component derived from the subject. Therefore, in the photoacoustic image (target region photoacoustic image 215 in FIG. 5) generated by the reconstruction unit 840, in addition to the image derived from the blood vessel in the target region, the image derived from the marker (marker photoacoustic image in FIG. 5) is also included. Image 255) is included.

(Step 4) Superposition processing This step corresponds to step S107. The image processing unit 850 adjusts the sizes of the photoacoustic image and the optical image, aligns the image based on the marker optical image 253 and the marker photoacoustic image 255, and generates a superposed image 220 as shown in FIG. .. When the photoacoustic image and the optical image are superimposed, if the orientation or position of the subject in the photoacoustic image and the optical image is different, at least one of the images may be rotated or moved. good. Various known methods can be used for these processes.

According to this embodiment, the photoacoustic image and the optical image can be accurately superimposed based on the position of the marker. As a result, the user can accurately grasp which part of the subject the blood vessel image shown in the photoacoustic image corresponds to, based on the contour of the subject and the like.

<Example 2>
In this embodiment, the marker 250 is placed on the holding member instead of the subject. Hereinafter, the differences from the first embodiment will be mainly described.

FIG. 7 shows the holding member 200 of this embodiment. As shown, the marker 250 is arranged on the holding member 200. As the holding member 200 and the marker 250, the same materials as in the first embodiment can be used. Further, when arranging the marker 250 on the holding member 200, any method such as writing with a pen, sticking with a sticker, or sticking with an adhesive can be used as in the first embodiment. Further, in the case of this embodiment, when the holding member 200 is manufactured, the marker 250 is previously manufactured.
May be embedded. The marker may be constructed using any material as long as it can be confirmed by either an optical image or a photoacoustic image. One example is carbon black.

Photoacoustic measurement, optical imaging, and superimposed image generation are the same as in Example 1. That is, the camera 500 takes an image of the subject for each holding member 200 in which the marker 250 is arranged. Further, the photoacoustic wave received by the conversion element 320 includes a component derived from the subject and a component derived from the marker 250 on the holding member. The image processing unit 850 superimposes the photoacoustic image and the optical image using a marker as a marker to generate a superposed image 220.

In this example, the marker 250 is not placed on the subject. Therefore, it is preferable to separately provide a positioning marker for positioning the imaging site of the subject on the subject. As the positioning marker, a mark described or affixed to the imaging site of the subject may be used, or a physical feature such as a mole may be used. It is not necessary to generate a photoacoustic wave for the positioning marker. Prior to optical imaging, the user adjusts the imaging region to a desired position in the field of view of the camera using a positioning marker.

Also in this embodiment, it becomes possible to accurately grasp which part of the subject the blood vessel or the like shown in the photoacoustic image corresponds to based on the contour of the subject or the like.

<Example 3>
A marker 250 is drawn on the superimposed image 220 obtained in each of the above embodiments. While such a marker has an advantage that the position can be easily grasped, there is also a problem that the user's visibility is lowered. Therefore, in this embodiment, a method of excluding the marker from the displayed image will be described. Each of the following modifications describes a method in which the reconstruction unit 840 deletes or reduces the component derived from the marker from the photoacoustic image.

On the other hand, the optical image processing unit 820 excludes the marker from the optical image. Or at least reduce the components derived from the marker. Specifically, the optical image processing unit 820 first determines the range of markers in the optical image by image processing based on pixel values or designation by the user using the input unit 880. Subsequently, the pixel value in the determined marker range is corrected by interpolation processing from the periphery. Normally, in an optical image, it is sufficient if a rough structure such as a contour line can be grasped, and it is not necessary to depict a fine structure on the surface of the subject. Therefore, the resolution of the surface of the subject including the marker may be lowered, or the entire thigh may be painted with a uniform color.

The image processing unit 850 superimposes the photoacoustic image from which the marker has been deleted and the optical image. Even in that case, since it is already known at which position in the image the marker was present, the superposition process can be executed without any problem. The image processing unit 850 may also remove the marker from the superimposed image after superimposing the photoacoustic image and the optical image while leaving the marker.

(Modification 1) Deleting surface information When a user wants to see a blood vessel image inside a subject, the information on the surface of the subject may not be necessary. Therefore, in this modification, the data corresponding to the surface of the subject is deleted from the photoacoustic signal and the photoacoustic image. For example, consider the case where volume data up to a depth of 5 cm from the surface of the target area of the subject is obtained by image reconstruction. In this case, the marker can be prevented from being displayed on the display image by excluding the volume data for 2 mm on the side close to the surface of the subject from the display target. The volume data in the depth range may be deleted, or any method may be used such as attaching an identifier to the data in the depth range so that it is not displayed on the screen. When the marker is arranged on the holding member as in the second embodiment, the data corresponding to the thickness of the holding member may be excluded from the display target. The method of generating a two-dimensional image for display from the volume data is arbitrary. For example, image data having a certain depth may be extracted with reference to the surface of the subject. Further, a MIP image may be generated. Further, a tomographic image of a certain depth may be selected. In this case, it is not always necessary to generate all the volume data.

Further, instead of deleting the data of the surface portion after generating the volume data, the component derived from the marker may be deleted from the photoacoustic signal in advance. Which part of the photoacoustic signal stored in the storage unit is derived from the marker can be calculated based on the irradiation timing of light, the acoustic matching medium, and the speed of sound of the subject. By performing image reconstruction using such a photoacoustic signal, a photoacoustic image excluding the marker is generated.

(Modification 2) Subtracting marker information In this modification, the marker shape is stored in the storage unit in advance, and the marker is deleted and displayed by image processing. Further, when the marker is arranged on the holding member as in Example 2, only the marker is photoacousticly measured in advance without placing the subject on the holding member, and the component of the photoacoustic signal derived from the marker is stored. It is stored in the unit, and the stored component is subtracted from the photoacoustic image.

(Modification 3) Using the difference in oxygen saturation In this modification, the blood vessel image and the marker are distinguished from the oxygen saturation value in the photoacoustic image, and the marker is excluded and displayed. The light source of this modification can irradiate light having a plurality of wavelengths including the first wavelength and the second wavelength. Based on the absorption coefficient distribution of each wavelength, the reconstruction unit generates image data showing the oxygen saturation distribution by utilizing the difference in the absorption coefficient of the oxidized hemoglobin and the deoxidized hemoglobin for each wavelength. For example, arterial blood has an oxygen saturation of about 95% and venous blood has an oxygen saturation of about 75%. Therefore, the blood vessel image and the marker can be distinguished by selecting a material having an oxygen saturation calculated at the wavelength of the irradiation light, which is lower than that of venous blood, for example, 20% or less, as a marker.

(Modification 4) Using a difference in absorption spectrum In addition, when using a variable wavelength light source, image processing using a marker having an absorption spectrum different from that of hemoglobin is also possible. For example, a light source capable of irradiating light having a wavelength (first wavelength) that is easily absorbed by hemoglobin and light having a wavelength (second wavelength) that is difficult to be absorbed by hemoglobin and easily absorbed by the marker is used. That is, it can be realized by selecting a wavelength and a marker material such that the magnitudes of the absorption coefficients of hemoglobin and the marker are opposite to each other for at least two wavelengths. By using the photoacoustic image derived from the second wavelength for the alignment with the optical image by the marker, the accuracy of the alignment is improved. Further, when the marker information is excluded from the photoacoustic image derived from the first wavelength, the photoacoustic image derived from the second wavelength can be used. Also in this modification, the blood vessel image and the marker can be distinguished and the marker can be excluded from the displayed image.

When using a multi-wavelength light source as in Modifications 3 and 4, it is better to alternately irradiate light of a plurality of wavelengths than to measure at the next wavelength after finishing the measurement at the first wavelength. , Positional deviation due to body movement can be reduced.

According to this embodiment, as shown in FIG. 8, it is possible to present a superposed image in which the marker information is excluded and the blood vessel image is easy to see, so that the user's visibility is improved. On the other hand, since the contour line of the subject remains, there is no problem in grasping the positional relationship between the blood vessel image and the subject.

<Example 4>
In this embodiment, a modified example relating to the acquisition of an optical image will be described. Here, in the case of the device configuration as shown in FIG. 1, since the support 300 enters between the camera 500 and the subject during the photoacoustic measurement, optical imaging cannot be performed. Therefore, in this embodiment, in order to enable optical imaging even during photoacoustic measurement, the device configuration is as shown in FIG. In FIG. 9, the camera 500 is arranged on the support 300 together with the conversion element 320, and shooting is performed while moving by the drive unit 600.

In this embodiment, since the field of view of the camera 500 at the time of optical imaging is narrower than that of the configuration of Example 1, it is not possible to image the entire target area of the subject at one time. Therefore, the optical image acquisition unit 820 of this embodiment creates an optical image of the entire target region by synthesizing a plurality of optical images. Specifically, first, while the camera 500 moves, optical imaging is performed a plurality of times, and the obtained optical images are sequentially stored in the storage unit 870. The optical image acquisition unit 820 performs a synthesis process based on the position information of the support 300 received from the drive unit 600 via the control unit 810 and the timing at which the camera 500 acquires each of the plurality of optical images, and performs a single sheet. Generate an optical image.

As shown in FIGS. 10A to 10D, if a plurality of (two or more) markers are arranged at intervals so as to be included in the field of view of the camera, FIG. 10E can be used. The composite optical image as shown can be easily generated. Such an arrangement method can be applied to both the case where the marker is provided on the holding member side and the case where the marker is provided on the subject side.

FIG. 11 shows an example of the marker of this embodiment. With the method of FIG. 11, since the position of each marker is uniquely known, it becomes easy to generate a composite optical image. The shape and type of the marker are not limited to those in FIGS. 10 and 11.

<Example 5>
In this embodiment, a suitable example of superimposing image generation by the image processing unit 850 will be described. The image processing unit 850 of this embodiment changes the transparency of at least one of the images when the other is superimposed on one of the optical image and the photoacoustic image.

Here, an example will be described in which the photoacoustic image is superimposed on the optical image when the subject is the thigh and a blood vessel image is generated as the photoacoustic image. The image processing unit 850 changes the transparency of the photoacoustic image to 50% after resizing so that the sizes of both images match. Then, the photoacoustic image is superimposed on the optical image. By doing so, it becomes possible to clearly grasp the image of blood vessel running in the thigh, which is the subject. It should be noted that which image should be on top or the transparency of each image can be arbitrarily set according to the intended use.

<Example 6>
The present invention can be regarded as a display method executed by the information processing apparatus 800 for superimposing and displaying a photoacoustic image and an optical image. In that case, the information processing apparatus 800 acquires a photoacoustic signal (or a photoacoustic image generated from the photoacoustic signal) including a component derived from the marker and a component derived from the subject, and the same subject is captured by a camera. Acquire the captured optical image. Then, both images are superimposed on the marker as a reference to generate and display a superimposed image.

As described above, according to the present invention, it is possible to help the user's understanding of the correspondence between the photoacoustic image and the subject. As a result, the operator can easily grasp the vascular structure in the subject even if the area is large or the characteristic structure is small.

Image information processing device: 800, optical image acquisition unit: 820, reconstruction unit: 840

Claims (23)

A photoacoustic image acquisition unit that acquires a photoacoustic image generated based on a photoacoustic wave generated by irradiating a subject and a marker with light, and a photoacoustic image acquisition unit.
An optical image acquisition unit that acquires an optical image generated by optically imaging the subject and the marker, and an optical image acquisition unit.
An image in which a superposed image of the photoacoustic image and the optical image is generated and displayed on the display unit based on the component derived from the marker in the photoacoustic image and the marker shown in the optical image. Processing unit and
Equipped with
The image processing unit is an image information processing apparatus characterized by reducing components derived from the marker from the superimposed image or the photoacoustic image. A support that supports the conversion element that receives the photoacoustic wave, the camera that captures the optical image, and the support.
The image information processing apparatus according to claim 1, further comprising a driving unit for moving the support relative to the subject. The image information according to claim 2, wherein the optical image acquisition unit synthesizes a plurality of images taken at a plurality of positions where the camera is moved by the drive unit to acquire the optical image. Processing equipment. The image information processing apparatus according to claim 3, wherein when a plurality of the markers are provided, the field of view of the camera includes two or more of the markers. The image information processing apparatus according to any one of claims 1 to 4, wherein the marker is a substance having absorption characteristics at the wavelength of light. The image information processing apparatus according to any one of claims 1 to 5, wherein a marker provided on the subject is used as the marker. The image information processing apparatus according to claim 6, wherein the marker is a portion where a dye is accumulated in the subject. Further provided with a holding member for holding the subject,
The image information processing apparatus according to any one of claims 1 to 5, wherein the marker is provided on the holding member. The image information processing apparatus according to any one of claims 1 to 8, wherein the image processing unit generates the superimposed image by matching the sizes of the photoacoustic image and the optical image. The image information processing apparatus according to any one of claims 1 to 9, wherein the image processing unit reduces components derived from the surface of the subject including the marker from the photoacoustic image. .. The image according to any one of claims 1 to 9, wherein the image processing unit reduces components derived from the marker from the photoacoustic image by image processing based on the shape of the marker. Information processing device. The light contains light having a plurality of wavelengths, and the light includes light having a plurality of wavelengths.
The image processing unit is characterized in that the component derived from the marker is reduced based on the difference in oxygen saturation obtained by irradiation of the marker and the subject with light having a plurality of wavelengths. The image information processing apparatus according to any one of 1 to 9 . The light contains light having a plurality of wavelengths, and the light includes light having a plurality of wavelengths.
The image processing unit is characterized in that the component derived from the marker is reduced based on the difference in absorption spectra of the marker and the subject for each of the plurality of wavelengths of light, according to claims 1 to 9. The image information processing apparatus according to any one item . A photoacoustic image acquisition step of acquiring a photoacoustic image generated based on a photoacoustic wave generated by irradiating a subject and a marker with light,
An optical image acquisition step of acquiring an optical image generated by optically imaging the subject and the marker, and
An image in which a superposed image of the photoacoustic image and the optical image is generated and displayed on the display unit based on the component derived from the marker in the photoacoustic image and the marker shown in the optical image. Processing steps and
Have ,
The image processing step is a display method comprising reducing components derived from the marker from the superimposed image or the photoacoustic image. Further having a drive step of moving the support supporting the conversion element that receives the photoacoustic wave and the camera that captures the optical image relative to the subject.
The display method according to claim 14 , wherein in the optical image acquisition step, a plurality of images taken at a plurality of positions where the camera has moved in the drive step are combined to acquire the optical image. .. The display method according to claim 14 , wherein in the image processing step, the size of the photoacoustic image and the size of the optical image are matched to generate the superimposed image. The display method according to any one of claims 14 to 16 , wherein in the image processing step, an image in which a component derived from the marker is reduced is generated from the superimposed image and displayed on the display unit. .. The display method according to any one of claims 14 to 17 , wherein in the image processing step, components derived from the surface of the subject including the marker are reduced from the photoacoustic image. The display method according to claim 18 , wherein in the image processing step, components derived from the marker are reduced by image processing based on the shape of the marker from the photoacoustic image. The light contains light having a plurality of wavelengths, and the light includes light having a plurality of wavelengths.
The image processing step is characterized in that the component derived from the marker is reduced based on the difference in oxygen saturation obtained by irradiation of the marker and the subject with light having a plurality of wavelengths. The display method according to 18 . The light contains light having a plurality of wavelengths, and the light includes light having a plurality of wavelengths.
18. The image processing step according to claim 18 , wherein the component derived from the marker is reduced based on the difference in absorption spectra of the marker and the subject for each of the plurality of wavelengths of light. Display method. A photoacoustic image acquisition unit that acquires a photoacoustic image generated based on a photoacoustic wave generated by irradiating a subject and a marker with light, and a photoacoustic image acquisition unit.
An optical image acquisition unit that acquires an optical image generated by optically imaging the subject and the marker, and an optical image acquisition unit.
An image in which a superposed image of the photoacoustic image and the optical image is generated and displayed on the display unit based on the component derived from the marker in the photoacoustic image and the marker shown in the optical image. Processing unit and
Equipped with
The image processing unit reduces components derived from the surface of the subject, including the marker, from the superimposed image or the photoacoustic image.
An image information processing device characterized by this. A photoacoustic image acquisition step of acquiring a photoacoustic image generated based on a photoacoustic wave generated by irradiating a subject and a marker with light,
An optical image acquisition step of acquiring an optical image generated by optically imaging the subject and the marker, and
An image in which a superposed image of the photoacoustic image and the optical image is generated and displayed on the display unit based on the component derived from the marker in the photoacoustic image and the marker shown in the optical image. Processing steps and
Have,
In the image processing step, components derived from the surface of the subject including the marker are reduced from the superimposed image or the photoacoustic image.
A display method characterized by that.

Images