JP2020110362A - Information processing device, information processing method, and program - Google Patents

Abstract

To provide an information processing device capable of making it easy to discriminate whether the possibility of a target (observation object) being present at a certain position in an image is high or low.SOLUTION: An information processing device acquires a plurality of pieces of first image data corresponding to light irradiation performed a plurality of times derived from a photoacoustic wave generated from a subject by light irradiation performed a plurality of times onto the subject, and a plurality of pieces of second image data corresponding to the light irradiation performed a plurality of times; acquires a plurality of pieces of template data corresponding to the light irradiation performed a plurality of times; acquires correlation information indicating correlations between an image value string of the plurality of pieces of first image data in an attention position and an image value string of the plurality of pieces of template data in the attention position; generates composite image data by compositing the plurality of pieces of second image data; and causes display means to display an image based on the correlation information and the composite image data. An imaging range of each of the plurality of pieces of first image data and an imaging range of each of the plurality of pieces of second image data are different from each other.SELECTED DRAWING: Figure 9

Description

The present invention relates to an information processing device that generates image data derived from a photoacoustic wave generated by light irradiation.

A photoacoustic device is known as a device that generates image data based on a received signal obtained by receiving an acoustic wave. A photoacoustic apparatus irradiates a subject with pulsed light generated from a light source, and absorbs the energy of pulsed light propagated and diffused in the subject to generate acoustic waves (typically ultrasonic waves). , Also called photoacoustic wave). Then, the photoacoustic apparatus images the subject information based on the received signal.

Non-Patent Document 1 discloses Universal Back Projection (UBP), which is one of the back projection methods, as a method of imaging an initial sound pressure distribution from a received signal of a photoacoustic wave.

"Universal back-projection algorithm for photocou" "stic computed tomography", Minghua Xu and Lihong V. Wang, PHYSICAL REVIEW 70 6,200.

By the way, when the received signal of the acoustic wave is back-projected by the method described in Non-Patent Document 1 to generate the image data, the received signal is back-projected at a position other than the generation position of the acoustic wave and appears in the image as an artifact. As a result, it may be difficult to determine whether or not the image in the image is the image of the target (observation target).

Therefore, it is an object of the present invention to provide an information processing device that can easily determine whether a target (observation target) is highly likely or unlikely to exist at a position of interest in an image.

The information processing apparatus according to the present invention includes a plurality of first image data corresponding to a plurality of times of light irradiation derived from a photoacoustic wave generated from the subject by a plurality of times of light irradiation to the subject, and a plurality of times. A plurality of second image data corresponding to light irradiation is acquired, a plurality of template data corresponding to a plurality of light irradiations is acquired, and image value strings of a plurality of first image data at a target position and a plurality of image values at the target position are acquired. Correlation information indicating the correlation with the image value sequence of the template data is acquired, composite image data is generated by combining a plurality of second image data, and an image based on the correlation information and the composite image data is displayed on the display unit. , And the respective imaging ranges of the plurality of first image data are different from the respective imaging ranges of the plurality of second image data.

According to the information processing apparatus of the present invention, it is possible to easily determine whether a target (observation target) is highly likely or unlikely to exist at a position of interest in an image.

Diagram for explaining the time differentiation processing and the positive/negative inversion processing Figure for explaining the back projection process The figure which shows the fluctuation of image value and sensitivity for every light irradiation The figure which shows the image which applied the process which relates to this execution form Block diagram showing an information processing apparatus according to the present embodiment Schematic diagram showing a probe according to the present embodiment Block diagram showing the configuration of the computer and its peripherals according to the present embodiment The figure which shows the control flow of the photoacoustic apparatus which concerns on this embodiment. Flow chart of the information processing method according to the present embodiment Diagram showing the process of calculating composite image data Diagram showing the calculation process of the multiplied image data The figure which shows the calculation process of L2 norm of a 1st fluctuation characteristic. The figure which shows the calculation process of L2 norm of a 2nd fluctuation characteristic. The figure which shows the photoacoustic image data when changing the imaging position. Diagram showing relative positional relationship between sensitivity distribution and pixels Figure showing MIP display of composite image data of the subject held on a plane in three sections Cross-sectional view of the composite image data of the subject held on a plane in three cross sections Table showing the application area of the reference value for each pixel Figure showing MIP display of the composite image data of the subject held on the curved surface in three sections Cross-sectional view of the composite image data of the subject held on the curved surface in three sections Another table showing the application area of the reference value for each pixel The figure which shows the profile of a 1st correlation value and a 2nd correlation value in a depth direction.

Embodiments of the present invention will be described below with reference to the drawings. However, the dimensions, materials, shapes, and their relative arrangements of the components described below should be appropriately changed according to the configuration of the device to which the invention is applied and various conditions, and the scope of the present invention is not limited. It is not intended to be limited to the following description.

The present invention relates to generation of image data representing a two-dimensional or three-dimensional spatial distribution derived from a photoacoustic wave generated by light irradiation. The photoacoustic image data is at least one object such as generated sound pressure (initial sound pressure) of photoacoustic wave, light absorption energy density, and light absorption coefficient, concentration of substances constituting the object (such as oxygen saturation). It is image data representing the spatial distribution of information.

By the way, a living body, which is a main subject of photoacoustic imaging, has a property of scattering and absorbing light. Therefore, the light intensity exponentially attenuates as the light travels deep into the living body. As a result, typically, a photoacoustic wave with a large amplitude tends to occur near the surface of the subject, and a photoacoustic wave with a small amplitude tends to occur in the deep part of the subject. Particularly, a photoacoustic wave having a large amplitude is likely to be generated from a blood vessel existing near the surface of the subject.

In the reconstruction method called UBP (Universal Back-Projection) described in Non-Patent Document 1, a received signal is back projected onto an arc centered on a transducer. At this time, the received signal of the photoacoustic wave having a large amplitude near the surface of the subject is back-projected to the deep portion of the subject, resulting in an artifact in the deep portion of the subject. Therefore, when the living tissue existing in the deep part of the subject is imaged, the image quality (contrast or the like) may be deteriorated due to the artifact caused by the photoacoustic wave generated from the surface of the subject.

The present invention is an invention relating to image data processing for distinguishing an artifact from a target. That is, the present invention is an invention capable of facilitating determination as to whether or not there is a high possibility that a target exists at a position of interest in an image. In this specification, determining whether or not a target is present corresponds to determining whether or not a target is highly likely to be present. The processing according to the embodiment of the present invention will be described below.

It is known that the received signal of the photoacoustic wave generally has a waveform called N-Shape as shown in FIG. In UBP, time differential processing is performed on the N-Shape signal shown in FIG. 1(a) to generate the time differential signal shown in FIG. 1(b). Then, the positive/negative inversion processing which inverts the positive/negative of the signal level of a time differential signal is performed, and the positive/negative inversion signal shown in FIG.1(c) is produced|generated. A signal (also referred to as a projection signal) generated by performing time differentiation processing and positive/negative inversion processing on the N-Shape signal has a portion having a negative value as shown by arrows A and C in FIG. Then, a portion having a positive value appears as shown by an arrow B in FIG.

FIG. 2 shows an example in which UBP is applied when the photoacoustic wave generated from the target 10 which is a microsphere-shaped light absorber inside the subject is received by the transducer 21 and the transducer 22. When the target 10 is irradiated with light, a photoacoustic wave is generated, and the photoacoustic wave is sampled by the transducer 21 and the transducer 22 as an N-Shape signal. FIG. 2A is a diagram in which an N-Shape-shaped reception signal sampled by the transducer 21 is superimposed on the target 10. Although only the reception signal output from the transducer 21 is shown for convenience, the reception signal is also output from the transducer 22 in the same manner.

FIG. 2B is a diagram in which a projection signal obtained by performing time differentiation processing and positive/negative inversion processing on the N-Shape-shaped reception signal shown in FIG.

FIG. 2C shows a state where the projection signal obtained by using the transducer 21 is back projected by UBP. In UBP, a projection signal is projected on an arc centered on the transducer 21. In this case, the projection signal is back projected onto the range of the directional angle (for example, 60°) of the transducer 21. As a result, the image becomes as if the target 10 exists over the regions 31, 32, and 33. Here, the regions 31 and 33 are regions having a negative value, and the region 32 is a region having a positive value. In FIG. 2C, areas 31 and 33 having negative values are shaded.

FIG. 2D shows a case where the projection signal obtained by using the transducer 22 is back projected by UBP. As a result, the image becomes as if the target 10 exists over the regions 41, 42, and 43. Here, the regions 41 and 43 are regions having a negative value, and the region 42 is a region having a positive value. In FIG. 2D, areas 41 and 43 having negative values are shaded.

FIG. 2E shows a case where the projection signals corresponding to each of the plurality of transducers 21 and 22 are back-projected by UBP. Photoacoustic image data is generated by combining a plurality of backprojected projection signals in this manner.

As shown in FIG. 2E, at a position 51 inside the target 10, a positive value region 32 of the projection signal corresponding to the transducer 21 and a positive value region 42 of the projection signal corresponding to the transducer 22 overlap. .. That is, typically, in a region where the target 10 exists (also referred to as a target region), positive value regions are dominantly overlapped with each other. Therefore, in the region where the target 10 exists, typically, the image data for each light irradiation tends to have a positive value.

On the other hand, at the position 52 outside the target 10, the positive value region 32 of the projection signal corresponding to the transducer 21 and the negative value region 43 of the projection signal corresponding to the transducer 22 overlap. Further, at the position 53 outside the target 10, the negative value region 31 of the projection signal corresponding to the transducer 21 and the positive value region 42 of the projection signal corresponding to the transducer 22 overlap. As described above, in the area other than the target 10, the positive value area and the negative value area tend to be complicatedly overlapped. That is, in the area other than the target 10, the image data tends to have a positive value or a negative value each time the light is irradiated. The reason for this tendency is that the relative position between the transducer 22 and the target 10 changes every time the light is irradiated.

Next, the variation of the image data value (image value) for each light irradiation when the combination of the reception positions of the photoacoustic waves is changed for each light irradiation will be described. FIG. 3A shows a variation in image data value (image value) when the region of the target 10 is reconstructed by the UBP described in Non-Patent Document 1. The horizontal axis represents the light irradiation number, and the vertical axis represents the image value. On the other hand, FIG. 3B shows a variation in the image data value (image value) when the region other than the target 10 is reconstructed by the UBP described in Non-Patent Document 1. The horizontal axis represents the light irradiation number, and the vertical axis represents the image value.

According to FIG. 3A, it can be seen that the image value in the region of the target 10 is always a positive value, although it varies with each light irradiation. On the other hand, according to FIG. 3B, it is understood that the image value of the area other than the target 10 becomes a positive value or a negative value for each light irradiation.

Here, if the image data is generated by synthesizing the image data corresponding to all the light irradiations, a positive value is synthesized in the area of the target 10, so that the final image value becomes large. On the other hand, in the area other than the target 10, the positive value and the negative value of the image data cancel each other out, and the final image value becomes smaller than the area of the target 10. As a result, the presence of the target 10 can be visually recognized on the image based on the photoacoustic image data.

However, in the area other than the target 10, the image value may not be 0 even though there is no target, and the final image value may be a positive value. In this case, an artifact is generated at a position other than the target 10, which reduces the visibility of the target.

Therefore, it is desired to make it easy to determine whether the image in a certain area is the image of the target or the image other than the target.

Therefore, in order to solve the above-mentioned problems, the present inventor has noticed that the target region and the region other than the target typically have different characteristics of fluctuation in image value of image data for each light irradiation. That is, the present inventor has conceived to determine the target area and the area other than the target from the variation characteristic of the image value of the image data for each light irradiation. By this method, it is possible to accurately determine whether or not the target.

The present inventor also conceived to display an image showing the determination result of whether or not it is the target area. By displaying such an image, it is possible to easily determine whether or not the target is present at the position of interest in the image.

The present inventor also conceived to determine the target area by the above method and selectively extract the target image from the image data. That is, the present inventor has conceived that, when the target does not exist at the position of interest, the image based on the image data at the position is displayed with the brightness lower than the brightness corresponding to the image value at the position. According to such an information processing method, it is possible to provide the user with an image in which the target is emphasized. By displaying such an image, the user can easily determine whether or not the target exists at the target position.

The present inventor has found that the setting values of the imaging ranges of a plurality of image data corresponding to a plurality of times of light irradiation affect the accuracy of the variation characteristics of the image values and the image quality (contrast, resolution, etc.) of the combined image data. Found to give. Further, the present inventor has found that the suitable imaging range is different when the synthetic image data is obtained and when the variation characteristic of the image value is obtained.

Therefore, the present inventor has made a plurality of first image data corresponding to a plurality of light irradiations for obtaining a variation characteristic of an image value and a plurality of a plurality of light irradiations corresponding to a plurality of light irradiations for obtaining combined image data. The idea was to change the imaging range for the second image data of. That is, the present inventor has conceived that the imaging ranges of the plurality of first image data are different from the imaging ranges of the second image data. As a result, an appropriate imaging range is set for obtaining the variation characteristic of the image value and for obtaining the composite image.

Furthermore, the present inventor has made the imaging range of the first image data when the accuracy of the variation characteristic of the image value is optimized, and the imaging range of the second image data when the image quality of the combined image data is optimized. It was found to be larger than the range. Therefore, the present inventor has conceived to make each imaging range of the plurality of first image data larger than each imaging range of the plurality of second image data.

Details of the processing according to the present invention will be described later in the following embodiments.

In this embodiment, an example in which photoacoustic image data is generated by a photoacoustic apparatus will be described. Hereinafter, the configuration of the photoacoustic apparatus of this embodiment and the information processing method will be described.

The configuration of the photoacoustic apparatus according to this embodiment will be described with reference to FIG. FIG. 5 is a schematic block diagram of the entire photoacoustic apparatus. The photoacoustic apparatus according to this embodiment includes a probe 180 including a light irradiation unit 110 and a reception unit 120, a driving unit 130, a signal collecting unit 140, a computer 150, a display unit 160, and an input unit 170.

FIG. 6 shows a schematic diagram of the probe 180 according to the present embodiment. The measurement target is the subject 100. The driving unit 130 drives the light emitting unit 110 and the receiving unit 120 to perform mechanical scanning. The light irradiation unit 110 irradiates the subject 100 with light, and an acoustic wave is generated in the subject 100. An acoustic wave generated by the photoacoustic effect due to light is also called a photoacoustic wave. The receiving unit 120 outputs an electric signal (photoacoustic signal) as an analog signal by receiving the photoacoustic wave.

The signal collecting unit 140 converts the analog signal output from the receiving unit 120 into a digital signal and outputs the digital signal to the computer 150. The computer 150 stores the digital signal output from the signal collecting unit 140 as signal data derived from an ultrasonic wave, an acoustic wave, or a photoacoustic wave.

The computer 150 performs signal processing on the stored digital signal to generate photoacoustic image data representing a two-dimensional or three-dimensional spatial distribution of information about the subject 100 (subject information). The computer 150 also causes the display unit 160 to display an image based on the obtained image data. The doctor as the user can make a diagnosis by checking the image displayed on the display unit 160. The display image is stored in a memory in the computer 150 or a memory such as a data management system connected to the modality via a network based on a storage instruction from the user or the computer 150.

The computer 150 also controls the drive of the configuration included in the photoacoustic apparatus. Further, the display unit 160 may display a GUI or the like in addition to the image generated by the computer 150. The input unit 170 is configured so that the user can input information. The user can use the input unit 170 to perform operations such as starting and ending the measurement and giving an instruction to save the created image.

Hereinafter, details of each component of the photoacoustic apparatus according to the present embodiment will be described.

(Light irradiation unit 110)
The light irradiation unit 110 includes a light source 111 that emits light, and an optical system 112 that guides the light emitted from the light source 111 to the subject 100. The light includes pulsed light such as so-called rectangular wave and triangular wave.

The pulse width of the light emitted by the light source 111 may be 1 ns or more and 100 ns or less. The wavelength of light may be in the range of 400 nm to 1600 nm. When imaging a blood vessel with high resolution, a wavelength having a large absorption in the blood vessel (400 nm or more and 700 nm or less) may be used. When imaging a deep part of a living body, light having a wavelength (700 nm or more and 1100 nm or less) that is typically less absorbed in the background tissue (water, fat, etc.) of the living body may be used.

A laser or a light emitting diode can be used as the light source 111. Further, when the measurement is performed using light of a plurality of wavelengths, a light source that can change the wavelength may be used. In addition, when irradiating a subject with a plurality of wavelengths, it is possible to prepare a plurality of light sources that generate lights of different wavelengths and irradiate the respective light sources alternately. Even when a plurality of light sources are used, they are collectively expressed as a light source. As the laser, various lasers such as a solid laser, a gas laser, a dye laser, and a semiconductor laser can be used. For example, a pulsed laser such as an Nd:YAG laser or an alexandrite laser may be used as the light source. Further, a Ti:sa laser or an OPO (Optical Parametric Oscillators) laser that uses Nd:YAG laser light as excitation light may be used as a light source. A flash lamp or a light emitting diode may be used as the light source 111. A microwave source may be used as the light source 111.

The optical system 112 can be an optical element such as a lens, a mirror, a prism, an optical fiber, a diffusion plate, or a shutter.

The maximum permissible exposure amount (MPE: maximum permissible exposure) of the intensity of light allowed to irradiate a biological tissue is defined by the following safety standards. (IEC 60825-1: Safety of laser products, JIS C 6802: Safety standards for laser products, FDA: 21CFR Part 1040.10, ANSI Z136.1: Laser Safety Standards, etc.). The maximum allowable exposure dose defines the intensity of light that can be irradiated per unit area. Therefore, a large amount of light can be guided to the subject E by collectively irradiating the surface of the subject E with a large area, so that the photoacoustic wave can be received with a high SN ratio. When a living body tissue such as a breast is used as the subject 100, the emission part of the optical system 112 may be configured by a diffusion plate or the like for diffusing light in order to expand the beam diameter of high-energy light for irradiation. On the other hand, in the photoacoustic microscope, in order to increase the resolution, the light emitting portion of the optical system 112 may be configured by a lens or the like, and the beam may be focused and irradiated.

The light irradiation unit 110 may not include the optical system 112, and the light source 111 may directly irradiate the subject 100 with light.

(Reception unit 120)
The reception unit 120 includes a transducer 121 that outputs an electric signal by receiving an acoustic wave, and a support body 122 that supports the transducer 121. Further, the transducer 121 may be a transmission unit that transmits ultrasonic waves and acoustic waves. The transducer as the receiving means and the transducer as the transmitting means may be a single (common) transducer or may have different configurations.

A piezoelectric ceramic material typified by PZT (lead zirconate titanate), a polymer piezoelectric film material typified by PVDF (polyvinylidene fluoride), or the like can be used as a member forming the transducer 121. In addition, an element other than the piezoelectric element may be used. For example, a capacitance type transducer (CMUT:Capacitive Micro-machined Ultrasonic Transducers), a transducer using a Fabry-Perot interferometer, or the like can be used. Any transducer may be adopted as long as it can output an electric signal by receiving an ultrasonic wave or an acoustic wave. Also, the signal obtained by the transducer is a time resolved signal. That is, the amplitude of the signal obtained by the transducer represents a value based on the sound pressure received by the transducer at each time (for example, a value proportional to the sound pressure).

The frequency component that constitutes the photoacoustic wave is typically 100 KHz to 100 MHz, and as the transducer 121, one that can detect these frequencies can be adopted.

The support 122 may be made of a metal material or the like having high mechanical strength. In order to allow a large amount of irradiation light to enter the subject, the surface of the support 122 on the subject 100 side may be mirror-finished or light-scattered. In the present embodiment, the support body 122 has a hemispherical shell shape, and is configured to support a plurality of transducers 121 on the hemispherical shell. In this case, the directional axes of the transducers 121 arranged on the support 122 gather near the center of curvature of the hemisphere. Then, when an image is formed using the signals output from the plurality of transducers 121, the image quality near the center of curvature becomes high. The support 122 may have any configuration as long as it can support the transducer 121. The support 122 may have a plurality of transducers arranged side by side in a plane or a curved surface such as 1D array, 1.5D array, 1.75D array, and 2D array. The plurality of transducers 121 correspond to a plurality of receiving means.

The support 122 may also function as a container that stores the acoustic matching material 210. That is, the support 122 may be a container for disposing the acoustic matching material 210 between the transducer 121 and the subject 100.

Further, the receiver 120 may include an amplifier that amplifies the time-series analog signal output from the transducer 121. Further, the receiving unit 120 may include an A/D converter that converts a time-series analog signal output from the transducer 121 into a time-series digital signal. That is, the receiving unit 120 may include the signal collecting unit 140 described below.

In order to detect the ultrasonic wave, the acoustic wave, and the photoacoustic wave at various angles, ideally, the transducer 121 may be arranged so as to surround the subject 100 from the entire circumference. However, when the object 100 is large and the transducer cannot be arranged so as to enclose the entire circumference, the transducer may be arranged on the hemispherical support 122 so as to approximate the state of enclosing the entire circumference.

The arrangement and number of transducers and the shape of the support may be optimized according to the subject, and any receiving unit 120 can be adopted in the present invention.

The space between the receiving unit 120 and the subject 100 is filled with a medium in which ultrasonic waves, acoustic waves, and photoacoustic waves can propagate. A material that can propagate ultrasonic waves, acoustic waves, and photoacoustic waves in this medium, has matching acoustic characteristics at the interface with the subject 100 and the transducer 121, and has a high transmittance of ultrasonic waves, acoustic waves, and photoacoustic waves as much as possible. To adopt. For example, water, ultrasonic gel, or the like can be adopted as this medium.

FIG. 6A shows a side view of the probe 180, and FIG. 6B shows a top view of the probe 180 (a view seen from above the paper surface of FIG. 6A). The probe 180 according to the present embodiment shown in FIG. 6 has a receiving unit 120 in which a plurality of transducers 121 are three-dimensionally arranged on a hemispherical support 122 having an opening. Further, in the probe 180 shown in FIG. 6, the light emitting portion of the optical system 112 is arranged at the bottom of the support 122.

In the present embodiment, as shown in FIG. 6, the subject 100 is held in its shape by contacting the holder 200. In the present embodiment, when the subject 100 is a breast, an opening for inserting the breast is provided in a bed supporting the subject in the prone position, and the breast hung in the vertical direction from the opening is measured. I'm assuming.

The space between the receiving unit 120 and the holding unit 200 is filled with a medium (acoustic matching material 210) in which ultrasonic waves, acoustic waves, and photoacoustic waves can propagate. A material that can propagate ultrasonic waves, acoustic waves, and photoacoustic waves in this medium, has matching acoustic characteristics at the interface with the subject 100 and the transducer 121, and has a high transmittance of ultrasonic waves, acoustic waves, and photoacoustic waves as much as possible. To adopt. For example, this medium can be water, castor oil, ultrasonic gel, or the like.

The holding unit 200 as a holding unit is used to hold the shape of the subject 100 during measurement. By holding the subject 100 by the holding unit 200, it is possible to suppress the movement of the subject 100 and to keep the position of the subject 100 in the holding unit 200. A resin material such as polycarbonate, polyethylene, or polyethylene terephthalate can be used as the material of the holding portion 200.

The holder 200 is preferably made of a material having a hardness capable of holding the subject 100. The holder 200 may be made of a material that transmits light used for measurement. The holding unit 200 may be made of a material having an impedance similar to that of the subject 100. When the subject 100 having a curved surface such as a breast is used as the subject 100, the holding unit 200 molded in a concave shape may be used. In this case, the subject 100 can be inserted into the concave portion of the holding unit 200.

The holding unit 200 is attached to the attaching unit 201. The attachment unit 201 may be configured to be capable of exchanging a plurality of types of holding units 200 according to the size of the subject. For example, the mounting portion 201 may be configured to be replaceable with a holding portion having a different radius of curvature or center of curvature.

Further, the holding unit 200 may be provided with a tag 202 in which the information of the holding unit 200 is registered. For example, information such as the radius of curvature of the holding unit 200, the center of curvature, the speed of sound, and the identification ID can be registered in the tag 202. The information registered in the tag 202 is read by the reading unit 203 and transferred to the computer 150. The reading unit 203 may be installed in the mounting unit 201 in order to easily read the tag 202 when the holding unit 200 is mounted in the mounting unit 201. For example, the tag 202 is a bar code and the reading unit 203 is a bar code reader.

(Drive unit 130)
The drive unit 130 is a unit that changes the relative position between the subject 100 and the receiving unit 120. In the present embodiment, the drive unit 130 is a device that moves the support 122 in the XY directions, and is an electric XY stage equipped with a stepping motor. The driving unit 130 includes a motor such as a stepping motor that generates a driving force, a driving mechanism that transmits the driving force, and a position sensor that detects positional information of the receiving unit 120. A lead screw mechanism, a link mechanism, a gear mechanism, a hydraulic mechanism, or the like can be used as the drive mechanism. Further, as the position sensor, a potentiometer using an encoder, a variable resistor, a linear scale, a magnetic sensor, an infrared sensor, an ultrasonic sensor, or the like can be used.

The driving unit 130 is not limited to changing the relative position between the subject 100 and the receiving unit 120 in the XY directions (two-dimensional), but may change the relative position to one-dimensional or three-dimensional. The movement path may be planarly scanned in a spiral shape or line and space, or may be inclined three-dimensionally along the body surface. Further, the probe 180 may be moved so that the distance from the surface of the subject 100 is kept constant. At this time, the drive unit 130 may measure the movement amount of the probe by monitoring the rotation speed of the motor.

The driving unit 130 may fix the receiving unit 120 and move the subject 100 as long as the relative positions of the subject 100 and the receiving unit 120 can be changed. When moving the subject 100, a configuration in which the subject 100 is moved by moving a holding unit that holds the subject 100 may be considered. Further, both the subject 100 and the receiving section 120 may be moved.

The drive unit 130 may move the relative position continuously, or may move the relative position by step and repeat. The driving unit 130 may be an electric stage that moves along a programmed locus or a manual stage. That is, the photoacoustic apparatus may be a handheld type in which the user holds and operates the probe 180 without the drive unit 130.

Further, in the present embodiment, the driving unit 130 drives the light emitting unit 110 and the receiving unit 120 at the same time for scanning, but it may drive only the light emitting unit 110 or only the receiving unit 120. May be.

(Signal collection unit 140)
The signal collecting unit 140 includes an amplifier that amplifies an electric signal that is an analog signal output from the transducer 121, and an A/D converter that converts the analog signal output from the amplifier into a digital signal. The signal collection unit 140 may be configured by an FPGA (Field Programmable Gate Array) chip or the like. The digital signal output from the signal collection unit 140 is stored in the storage unit 152 in the computer 150. The signal collection unit 140 is also called a Data Acquisition System (DAS). In this specification, an electric signal is a concept including both an analog signal and a digital signal. A light detection sensor such as a photodiode may detect light emission from the light irradiation unit 110, and the signal collection unit 140 may start the above processing in synchronization with the detection result as a trigger. Further, the signal collection unit 140 may start the processing in synchronization with an instruction given using a freeze button or the like as a trigger.

(Computer 150)
The computer 150 as a display control device includes a calculation unit 151, a storage unit 152, and a control unit 153. The function of each component will be described when the processing flow is described.

A unit having an arithmetic function as the arithmetic unit 151 can be configured by a processor such as a CPU or a GPU (Graphics Processing Unit), or an arithmetic circuit such as an FPGA (Field Programmable Gate Array) chip. These units may be composed not only of a single processor and arithmetic circuit, but also of a plurality of processors and arithmetic circuits. The calculation unit 151 may receive various parameters such as the sound velocity of the subject and the configuration of the holding unit from the input unit 170 and process the received signal.

The storage unit 152 can be configured by a ROM (Read only memory), a non-temporary storage medium such as a magnetic disk or a flash memory. The storage unit 152 may be a volatile medium such as a RAM (Random Access Memory). The storage medium in which the program is stored is a non-temporary storage medium. The storage unit 152 is not limited to one storage medium, and may be a plurality of storage media.

The storage unit 152 can store image data indicating a photoacoustic image generated by the calculation unit 151 by a method described below.

The control unit 153 is composed of an arithmetic element such as a CPU. The control unit 153 controls the operation of each component of the photoacoustic apparatus. The control unit 153 may control each component of the photoacoustic apparatus by receiving an instruction signal from the input unit 170 for various operations such as the start of measurement. Further, the control unit 153 reads the program code stored in the storage unit 152 and controls the operation of each component of the photoacoustic apparatus. For example, the control unit 153 may control the light emission timing of the light source 111 via the control line. When the optical system 112 includes a shutter, the control unit 153 may control opening/closing of the shutter via a control line.

Computer 150 may be a dedicated workstation designed. Further, each configuration of the computer 150 may be configured by different hardware. Further, at least a part of the configuration of the computer 150 may be configured by a single hardware.

FIG. 7 shows a specific configuration example of the computer 150 according to this embodiment. The computer 150 according to this embodiment includes a CPU 154, a GPU 155, a RAM 156, a ROM 157, and an external storage device 158. A liquid crystal display 161 as a display unit 160, a mouse 171 as an input unit 170, and a keyboard 172 are connected to the computer 150.

Moreover, the computer 150 and the plurality of transducers 121 may be provided in a configuration housed in a common housing. However, a part of the signal processing may be performed by a computer housed in the housing, and the rest of the signal processing may be performed by a computer provided outside the housing. In this case, the computers provided inside and outside the housing can be collectively referred to as the computer according to the present embodiment. That is, the hardware that constitutes the computer does not have to be housed in one housing.

(Display 160)
The display unit 160 is a display such as a liquid crystal display, an organic EL (Electro Luminescence) FED, a glasses-type display, or a head mounted display. This is a device for displaying an image based on volume data obtained by the computer 150, a numerical value of a position of interest, and the like. The display unit 160 may display an image based on the volume data or a GUI for operating the device. When displaying the subject information, the display unit 160 or the computer 150 may perform image processing (such as adjustment of the brightness value) and then display the subject information. The display unit 160 may be provided separately from the photoacoustic apparatus. The computer 150 can transmit the photoacoustic image data to the display unit 160 by wire or wirelessly.

(Input unit 170)
As the input unit 170, an operation console that can be operated by a user and that is configured by a mouse, a keyboard, or the like can be adopted. Alternatively, the display unit 160 may be configured with a touch panel, and the display unit 160 may be used as the input unit 170.

The input unit 170 may be configured to be able to input information such as a position and depth to be observed. As an input method, a numerical value may be input, or the input may be performed by operating the slider bar. Further, the image displayed on the display unit 160 may be updated according to the input information. Thereby, the user can set the appropriate parameters while checking the image generated by the parameters determined by his/her operation.

In addition, the user may operate the input unit 170 provided at a remote location of the photoacoustic apparatus, and the information input using the input unit 170 may be transmitted to the photoacoustic apparatus via the network.

The components of the photoacoustic apparatus may be configured as separate apparatuses or may be configured as one integrated apparatus. Further, at least a part of the configuration of the photoacoustic apparatus may be integrated into one apparatus.

In addition, information transmitted and received between the respective components of the photoacoustic apparatus is exchanged by wire or wirelessly.

(Subject 100)
The subject 100 does not constitute a photoacoustic apparatus, but will be described below. The photoacoustic apparatus according to the present embodiment can be used for the purpose of diagnosing malignant tumors and vascular diseases of humans and animals, and observing the progress of chemotherapy. Therefore, the subject 100 is assumed to be a living body, specifically, breasts of human bodies and animals, various organs, vascular network, head, neck, abdomen, limbs including fingers and toes, etc. It For example, if the human body is the measurement target, oxyhemoglobin or deoxyhemoglobin, blood vessels containing a large amount thereof, new blood vessels formed in the vicinity of a tumor, or the like may be used as the light absorber. Also, the plaque on the carotid artery wall may be the target of the light absorber. Further, melanin, collagen, lipids, etc. contained in the skin or the like may be the target of the light absorber. Further, a dye such as methylene blue (MB) or indocyanine green (ICG), fine gold particles, or an externally introduced substance obtained by integrating or chemically modifying them may be used as the light absorber. Further, a phantom imitating a living body may be used as the subject 100.

In the present specification, the light absorber that is the object of imaging described above is called a target. Further, a light absorber that is not a target of imaging, that is, a target of observation is not a target. For example, when the breast is the subject and the blood vessel is the target light absorber, tissues such as fat and mammary glands that constitute the breast are not targets. In describing the embodiments of the present invention, blood vessels are typically targeted. However, the target may change depending on the interest of the user, and is not necessarily limited to a specific living tissue such as a blood vessel.

Next, a control flow of the photoacoustic apparatus according to the present embodiment will be described with reference to FIG. Note that each step is executed by the computer 150 controlling the operation of the configuration of the photoacoustic apparatus.

(S100: Step of setting control parameters)
The user uses the input unit 170 to specify the control parameters such as the irradiation conditions (repetition frequency, wavelength, etc.) of the light irradiation unit 110 and the position of the probe 180, which are necessary for acquiring the object information. The computer 150 sets the control parameter determined based on a user's instruction.

(S200: Step of moving probe to designated position)
The control unit 153 causes the drive unit 130 to move the probe 180 to the designated position based on the control parameter designated in step S100. When imaging at a plurality of positions is designated in step S100, the drive unit 130 first moves the probe 180 to the first designated position. The driving unit 130 may move the probe 180 to a pre-programmed position when a measurement start instruction is given. In the case of the handheld type, the user may grasp the probe 180 and move it to a desired position.

(S300: Step of irradiating light)
The light irradiation unit 110 irradiates the subject 100 with light based on the control parameter designated in S100.

The light generated from the light source 111 is applied to the subject 100 as pulsed light via the optical system 112. Then, the pulsed light is absorbed inside the subject 100, and a photoacoustic wave is generated by the photoacoustic effect. The light irradiation unit 110 transmits a synchronization signal to the signal collection unit 140 along with the transmission of pulsed light.

(S400: Step of receiving photoacoustic wave)
When the signal collection unit 140 receives the synchronization signal transmitted from the light irradiation unit 110, the signal collection unit 140 starts the signal collection operation. That is, the signal collecting unit 140 generates an amplified digital electric signal by amplifying and AD converting the analog electric signal derived from the acoustic wave, which is output from the receiving unit 120, and outputs the amplified digital electric signal to the computer 150. The computer 150 stores the signal transmitted from the signal collecting unit 140 in the storage unit 152. When the imaging at a plurality of scanning positions is designated in step S100, the steps S200 to S400 are repeatedly executed at the designated scanning positions, and the irradiation of the pulsed light and the generation of the digital signal derived from the acoustic wave are repeated. Note that the computer 150 may use the light emission as a trigger to acquire and store the position information of the receiving unit 120 at the time of light emission based on the output from the position sensor of the driving unit 130.

(S500: Step of generating photoacoustic image data)
The calculation unit 151 of the computer 150 as the image data generation unit generates photoacoustic image data based on the signal data stored in the storage unit 152 and stores the photoacoustic image data in the storage unit 152.

As reconstruction algorithms for converting signal data into volume data as spatial distribution, analytical reconstruction methods such as back projection in the time domain and back projection in the Fourier domain, and model-based methods (iteration operation method) Can be adopted. Examples of the back projection method in the time domain include Universal back-projection (UBP), Filtered back-projection (FBP), and phased addition (Delay-and-Sum).

In addition, the calculation unit 151 obtains absorption coefficient distribution information by calculating a light fluence distribution inside the subject 100 of the light emitted to the subject 100 and dividing the initial sound pressure distribution by the light fluence distribution. You may. In this case, the absorption coefficient distribution information may be acquired as photoacoustic image data. The computer 150 can calculate the spatial distribution of the light fluence inside the subject 100 by a method of numerically solving a transport equation and a diffusion equation that show the behavior of light energy in a medium that absorbs and scatters light. As a numerical method, a finite element method, a difference method, a Monte Carlo method, etc. can be adopted. For example, the computer 150 may calculate the spatial distribution of the light fluence inside the subject 100 by solving the light diffusion equation shown in Expression (1).

Here, D is the diffusion coefficient, μa is the absorption coefficient, S is the incident intensity of the irradiation light, φ is the reaching light fluence, r is the position, and t is the time.

In addition, the calculation unit 151 may acquire the absorption coefficient distribution information corresponding to each of the light of the plurality of wavelengths by performing the processes of S300 and S400 using the light of the plurality of wavelengths. Then, the calculation unit 151 acquires, as the photoacoustic image data, the spatial distribution information of the concentration of the substance forming the subject 100 as the spectral information, based on the absorption coefficient distribution information corresponding to the light of each of the plurality of wavelengths. May be. That is, the calculation unit 151 may acquire the spectral information by using the signal data corresponding to the light having a plurality of wavelengths.

Next, the information processing method according to the present embodiment in S500 will be described using the flowchart shown in FIG.

(S510: Step of performing time differentiation processing and inversion processing on the received signal)
The computer 150 as a signal processing unit performs signal processing on the received signal stored in the storage unit 152, including time differentiation processing and inversion processing for inverting the positive/negative of the signal level. The received signal subjected to these signal processings is also called a projection signal. In this step, these signal processes are performed on each reception signal stored in the storage unit 152. As a result, projection signals corresponding to each of the plurality of times of light irradiation and the plurality of transducers 121 are generated.

For example, the computer 150 performs time differentiation processing and inversion processing (giving negative to the time differentiation signal) on the reception signal p(r,t) as shown in Expression (2), and the projection signal b(r, t) is generated, and the projection signal b(r,t) is stored in the storage unit 152.

Here, r is the reception position, t is the elapsed time from the light irradiation, p(r,t) is the reception signal indicating the sound pressure of the acoustic wave received at the reception position r at the elapsed time t, and b(r,t). ) Is the projection signal. Note that other signal processing may be performed in addition to the time differentiation processing and the inversion processing. For example, the other signal processing is at least one of frequency filtering (low pass, high pass, band pass, etc.), deconvolution, envelope detection, and wavelet filtering.

In the present invention, the time differentiation process or the inversion process may not be executed as long as the image value varies among a plurality of first image data described below. Even in this case, the effect of this embodiment is not impaired.

(S520: Step of acquiring a plurality of first image data)
The computer 150 as the image data acquisition means, based on the light irradiations generated in S510 and the reception signals (projection signals) corresponding to the plurality of transducers 121, respectively, performs a plurality of light irradiations. Acquire photoacoustic image data. The plurality of photoacoustic image data generated in this step correspond to the plurality of first image data used for generating the characteristic information indicating the variation characteristic of the image value in step S540 described later.

For example, the computer 150 generates image data indicating the spatial distribution of the initial sound pressure p ₀ for each light irradiation, based on the projection signal b(r _i , t) as shown in Expression (3). As a result, image data corresponding to each of a plurality of light irradiations is generated, and a plurality of first image data can be acquired.

Here, r ₀ is a position vector indicating a position to be reconstructed (also referred to as a reconstructed position or a position of interest), p ₀ (r ₀ ) is an initial sound pressure at the position to be reconstructed, and c is a sound velocity of the propagation path. .. Further, ΔΩ _i is a solid angle that allows the i-th transducer 121 to be seen from the reconstruction position, and N is the number of transducers 121 used for reconstruction. Expression (3) indicates that the projection signal is weighted by the solid angle and the phasing addition is performed (back projection).

In this step, the computer 150 sets an imaging range (first imaging range) larger than the imaging range (second imaging range) of the second image data obtained in step S530 to be described later, and this imaging is performed. First image data of the range is generated. That is, the computer 150 generates the first image data of the first imaging range corresponding to each of the multiple times of light irradiation. By setting such an imaging range, it is possible to accurately generate the characteristic information indicating the variation characteristic of the image value in step 550 described later.

In this embodiment, the image data can be generated by the analytical reconstruction method or the model-based reconstruction method as described above. In particular, when the analytical reconstruction method is adopted, the variation of the image value for each light irradiation is relatively large, so that the present invention can be preferably applied. That is, in the analytical reconstruction method, when an acoustic wave is received under the conditions of the Limited View, the sound source cannot be accurately reproduced, and thus the region other than the target may be reconstructed with a negative value.

Alternatively, the computer 150 may acquire the plurality of first image data in the first imaging range from an image server such as a PACS (Picture Archiving and Communication Systems) or the like.

Further, the computer 150 may acquire the first image data of the first imaging range by extracting the data of the first imaging range from the image data of the imaging range larger than the first imaging range. ..

(S530: Step of acquiring a plurality of second image data)
The computer 150 as the image data acquisition unit acquires a plurality of photoacoustic image data corresponding to a plurality of light irradiations, as in the process of step S520. The photoacoustic image data obtained in this step is image data corresponding to an imaging range (second imaging range) smaller than the first imaging range of the first image data. The plurality of photoacoustic image data obtained in this step correspond to the second image data. The plurality of second image data obtained in this step are the targets of the combining process in S540 described below.

The method of acquiring the second image data is different from the method of acquiring the first image data in step S520 only in the imaging range, and thus the description thereof will be omitted. However, the computer 150 acquires the second image data of the second imaging range by extracting the data of the second imaging range from the first image data of the first imaging range obtained in step S520. May be.

(S540: Step of combining a plurality of second image data)
The computer 150 as the combined image data generation unit generates combined image data by combining the plurality of second image data obtained in step S530.

How to generate the composite image data will be described with reference to FIG. For simplicity, FIG. 10 shows a case where light irradiation is performed 9 times. FIG. 10 shows photoacoustic image data (second image data) 1001 obtained by the first light irradiation. The size of the imaging range of the photoacoustic image data obtained by one light irradiation is X1 voxels in the X direction, Y1 voxels in the Y direction, and ZA voxels in the Z direction. Further, FIG. 10 is a schematic diagram in the case where the photoacoustic image data 1001 obtained by the first light irradiation is stored in the memory space of the size of XA voxels in the X direction, YA voxels in the Y direction, and ZA voxels in the Z direction. Show. Further, FIG. 10 shows photoacoustic image data 1009 obtained by the ninth light irradiation. FIG. 10 shows photoacoustic image data 1001 to 1009 obtained by the first to ninth light irradiations. As illustrated in FIG. 10, the computer 150 generates the combined image data in the memory space by combining the second image data 1001 to 1009 obtained by the light irradiation nine times.

The positions of the photoacoustic image data 1001 to 1009 in the memory space are determined according to the position of the receiving unit 120 during light irradiation. Further, as is clear from FIG. 10, the sizes of the photoacoustic image data (second image data) and the composite image data are different, and the size of the composite image data is larger than the size of the second image data.

As described above, the nine photoacoustic image data 1001 to 1009 obtained by the nine times of light irradiation are arranged in the memory space for storing the combined image data at a position corresponding to the imaging position of the receiving unit 120, and are combined. Image data is synthesized.

When different photoacoustic image data are arranged in the memory space for storing the combined image data, the image values of the image data whose positions overlap may be cumulatively added. Alternatively, the cumulative addition value of the image values may be divided by the number of overlaps to obtain the addition average value. Note that the processing method for image data in which overlap occurs is not limited to a particular one as long as the visibility of the combined image is good.

In the following description, when the term “pixel” is used, the form of a pixel of a two-dimensional image, a voxel of a three-dimensional image, or the like is not particularly limited. A "pixel" includes any form of image element used for any purpose.

(S550: Step of Acquiring Correlation Information Showing Correlation between Image Value Sequence of First Image Data and Image Value Sequence of Template Data)
The computer 150 as the correlation information acquisition unit acquires the correlation information indicating the correlation between the image value sequence of the first image data obtained in S520 and the image value sequence of the template data corresponding to a plurality of light irradiations. In the present embodiment, the computer 150 calculates the correlation value between the variation characteristic of the image value of the first image data at the attention position and the variation characteristic of the template data at the attention position.

In the present embodiment, from the variation characteristic of the image value of the pixel at a certain attention position (hereinafter, referred to as the first variation characteristic or image value sequence), which is observed when referring to the plurality of first image data, Discriminate artifacts. At the time of determination, the variation characteristic of the image value of the image data that can be generated when the light absorber is present at a certain attention position in each of the plurality of light irradiations (hereinafter, referred to as the second variation characteristic or the template data string). Also used.

The template data is data that reflects the spatial sensitivity distribution of the photoacoustic apparatus that executes light irradiation and reception of photoacoustic waves a plurality of times. It can be said that the template data is data indicating image data that can be generated when the light absorber exists at each position in each of the plurality of times of light irradiation. That is, it can be said that the template data is a spatial distribution indicating the estimated value of the image value when the light absorber is present at each position. Further, the template data may be template data that reflects the characteristics of the subject 100 in addition to the spatial sensitivity distribution corresponding to the photoacoustic apparatus. In the present embodiment, the spatial sensitivity distribution corresponding to the space in which photoacoustic image data is generated (hereinafter referred to as image data generation space) is called “sensitivity information”.

Here, the computer 150 reads the sensitivity distribution formed by the plurality of transducers 121 arranged on the support 122 from the RAM or ROM as the storage unit 152. This sensitivity distribution shows the relative positional relationship between the absorber and the plurality of transducers 121 when the absorber is present in the region corresponding to the partial volume, and the partial volume image when the absorber at that position is imaged. It is the data of the relationship with the value. In the case of a hemispherical shell-shaped arrangement, typically, the sensitivity data tends to be higher as it is closer to the center of the spherical shell. That is, in the present embodiment, the template data adopts the spatial distribution data that reflects the sensitivity of the transducer 121 to the acoustic wave based on the relative positional relationship between the absorber (the voxel to be reconstructed) and the plurality of transducers 121. doing.

The computer 150 as the template data acquisition unit acquires the template data corresponding to each light irradiation by assigning coordinates to predetermined template data based on the relative position information between the receiving unit 120 and the position of interest in each light irradiation. You may. For example, the computer 150 can generate template data corresponding to each light irradiation by interlocking the spatial sensitivity distribution of the photoacoustic apparatus with the position of the receiving unit 120 at the time of light irradiation to define the coordinates.

The computer 150 may also acquire relative position information between the receiving unit 120 and the position of interest for a plurality of times of light irradiation. Subsequently, the computer 150 may read the template data corresponding to the relative position information from the storage unit 152 to acquire the template data corresponding to the plurality of times of light irradiation. In this case, the storage unit 152 may store a plurality of template data corresponding to a plurality of relative position information. If the storage unit 152 does not store the template data corresponding to the acquired relative position information, the template data corresponding to the relative position information near the acquired relative position information is used to interpolate new template data. May be generated.

Although the method of reading the template data has been described here, the template data may be calculated in the apparatus every time the computer 150 takes an image, and even in that case, the effect of the present invention can be obtained. When calculating the template data, in addition to the information on the relative position between the target position and the plurality of transducers 121, information such as the speed of sound in the propagation path of the photoacoustic wave and the attenuation characteristic of the acoustic wave may be considered. Further, the template data may be template data that takes into consideration the light intensity distribution of the irradiation light. Further, it may be template data in consideration of diffusion and attenuation due to light propagation in the subject. That is, it is possible to employ template data that takes into account all the effects on the image data in each light irradiation. Different template data may be used for each light irradiation, or common template data may be used between light irradiations.

When obtaining the template data, the calculation may be performed on the assumption that a uniform light amount is applied to the entire image data generation space, or the calculation is performed in consideration of the spatial distribution of the light amount depending on the light irradiation profile. You can go. Further, the calculation may be performed in consideration of the attenuation of the light amount due to the influence of the subject.

Further, the template data may be calculated such that the template data at the target position and the image value at the target position in the photoacoustic image data obtained by actually performing light irradiation are close to each other.

Two or more pixels existing in the image data generation space may be grouped, and the image values of the same template data may be assigned to the pixels existing in the same group.

Further, the computer 150 may change the template data according to the size of the observation target, for example, the blood vessel diameter. That is, the computer 150 may change the template data according to the reception frequency band or the frequency band used for processing. Specifically, the influence that the directivity of the receiving element changes according to the frequency of the received photoacoustic signal may be reflected in the template data. Further, the computer 150 may change the template data according to the shape of the target and the optical characteristics of the target. For example, consider a case where the user specifies information about the target (target size, shape, etc.). In this case, the computer 150 may acquire the template data corresponding to the instructed target information according to the information (relation table or relational expression) indicating the relationship between the target information and the template data.

The present invention can be preferably applied to a case where the spatial sensitivity distribution changes in a plurality of measurements. For example, in the photoacoustic apparatus, the present invention may be applied to a case where the spatial sensitivity distribution changes due to a change in the light irradiation position, a change in the photoacoustic wave receiving position, or the like between a plurality of times of light irradiation. it can.

For example, FIG. 3A is a first variation characteristic showing an image value sequence of the first image data when a light absorber such as a blood vessel exists at a certain pixel position. Further, FIG. 3B is a first variation characteristic showing an image value sequence at a position where a light absorber does not actually exist at a certain target position, that is, at a position corresponding to an artifact. On the other hand, FIG. 3C is a second variation characteristic showing an estimated value sequence of image values (image value sequence of template data) when a light absorber is present at a certain pixel position. The image value of the first image data and the image value of the template data correspond to a common light irradiation number. That is, it can be said that the image value sequence of the first image data and the image value sequence of the template data are associated with each other through the common light irradiation number.

In this case, the computer 150 calculates the correlation between the first variation characteristic shown in FIG. 3A or 3B and the second variation characteristic shown in FIG. 3C. Then, depending on the degree of correlation, it is possible to determine whether the target or the artifact exists at the position where the pixel corresponding to FIG. 3A or FIG. 3B exists. That is, the correlation between the first variation characteristic and the second variation characteristic corresponds to the presence information indicating the possibility of existence of the target.

As a method of calculating the correlation, the correlation value obtained from the inner product shown in Expression (4) can be used.

f is a vector of the first variation characteristic and can be expressed by f=(f ₁ , f ₁ , f ₂ ,..., F _n ). g is a vector of the second variation characteristic and can be expressed by g=(g ₁ , g ₂ , g ₃ ,... G _n ). The number of elements of the vector corresponds to the number of data samples (for example, n samples) of each variation characteristic pattern, that is, the number of times of light irradiation. f*g represents an inner product calculation of the vector f and the vector g, and |f| and |g| represent the L2 norm of the vectors f and g. It can be expressed as |f|=√(f ₁ ² +f ₂ ² +...+f _n ² ) and |g|=√(g ₁ ² +g ₂ ² +...+g _n ² ).

The image value sequence of the first image data corresponds to the vector f of the first variation characteristic. The image value sequence of the template data corresponds to the vector g of the second variation characteristic.

In the present embodiment, the inner product calculation is divided into corresponding inner product calculations for each light irradiation, and the processing of Expression (4) is executed. By doing so, it is possible to easily perform the inner product calculation. Hereinafter, a specific description will be given.

In the present embodiment, the image value at a certain pixel position in the photoacoustic image data obtained by one light irradiation is multiplied by the template data at the same pixel position in the sensitivity information. By this process, the inner product process of the first variation characteristic and the second variation characteristic is divided and carried out for each photoacoustic image data obtained by one light irradiation.

Here, the calculation processing of the correlation value in the present embodiment will be described with reference to FIGS. 11 to 13. FIG. 11 shows a process of calculating f*g in the equation (4). FIG. 12 shows a process of calculating |f| in equation (4). FIG. 13 shows a process of calculating |g| in equation (4).

As shown in FIG. 11, first, photoacoustic image data (first image data) 2001 corresponding to the first light irradiation and sensitivity information 2010 of the photoacoustic apparatus are multiplied by corresponding image values at the same position ( Hereafter, referred to as multiplication image data) 2101 is generated. “.*” means to multiply the image value of the photoacoustic image data corresponding to the same position by the image value of the template data of the sensitivity information 2010. This is because in the pixel at the position of interest, the first element f _{1 of} the first variation characteristic vector f shown in equation (4) is multiplied by the _first element g ₁ of the second variation characteristic vector g to obtain f ₁ × It is equivalent to obtaining g ₁ . This corresponds to performing the inner product calculation of the first variation characteristic vector f and the second variation characteristic vector g by dividing the portion corresponding to the photoacoustic image data obtained by the first light irradiation. The multiplication image data 2101 thus obtained is arranged in the memory space according to the image pickup position.

In FIG. 11, photoacoustic image data 2009 corresponding to the ninth light irradiation is similarly multiplied by the sensitivity information to generate multiplied image data 2109. Then, the multiplication image data 2101 to 2109 is generated from the photoacoustic image data obtained by the light irradiation of 9 times, the multiplication image data is arranged in the memory space based on the information of the imaging position, and the synthetic multiplication image data is generated. ..

At this time, the image sizes of the image data 2001 to 2009 and the sensitivity information 2010 used for calculating the correlation value are X2 voxels in the X direction, Y2 voxels in the Y direction, and ZA voxels in the Z direction. On the other hand, the image sizes of the image data 1001 to 1009 used to generate the composite image data shown in FIG. 10 are X1 voxels in the X direction, Y1 voxels in the Y direction, and ZA voxels in the Z direction. At this time, since the imaging range of the first image data is larger than the imaging range of the second image data, it is preferable to set X1>X2 and Y1>Y2. Further, also in the Z direction, the imaging range of the first image data may be larger than the imaging range of the second image data. By setting the imaging range in this way, it is possible to improve the image quality of the combined image data and the accuracy of calculating the correlation value at the same time.

In the present embodiment, the processing is performed using the values of X1 and Y1 at which the image quality such as the contrast of the combined image data is optimal, and the values of X2 and Y2 at which the correlation value is optimally calculated. X1 and X2 and Y1 and Y2 can take different values. Here, "optimal for calculating a correlation value" refers to a state in which the difference between the correlation values of a portion where blood vessels are present and a portion where blood vessels are not present is as large as possible.

In the process shown in FIG. 11, the multiplication pixel data is cumulatively added to the pixels generated by overlapping the photoacoustic image data even at different imaging positions. For example, when an overlap occurs 9 times at a certain pixel j, the multiplied pixel data is represented by cumulative addition as in Expression (5).
f _(j,1) *g _(j,1) +f _(j,2) *g _(j,2) +f _(j,3) *g _(j,3) +... +f _(j,9) *g _{( j, 9)} ... Formula (5)

Here, f _{(j,1) to} f _(j,9) represent image values of the first image data of the first to ninth light irradiations in a certain pixel j. Then, g _{(j,1) to} g _{(j, 9)} indicate sensitivity information (template data) from a first light irradiation to a ninth light irradiation in a certain pixel j. In FIG. 11, the memory space is composed of 50 [voxel]×40 [voxel]×10 [voxel]=20000 [voxel], and therefore takes a value of j=1 to 20000.

That is, the image value variation pattern of the first image data from the first to the ninth light irradiation of a certain pixel j has vectors f _j ={f _(j,1) , f _(j,2) , f _(j,3) ,..., F _{(j, 9)} }, which is a vector representation of the first variation characteristic. Further, the image value variation pattern of the template data at the pixel j is expressed by the vector g _j ={g _(j,1) , g _(j,2) , g _(j,3) ,..., G _(j,9) }. Which is a vector representation of the second variation characteristic. At this time, the equation (5) is the inner product operation itself of the vector f _j and the vector g _j .

In a plurality of light irradiations, if the image pickup positions are different, even the same pixel will be imaged at different positions of the receiving unit 120, so the values of the template image values to be multiplied may be different.

By performing the processing shown in FIG. 11, the inner product value of the image values of the first image data and the image value of the template data, that is, the inner product value of the first variation characteristic and the second variation characteristic is calculated for all the target pixels. It is possible (Process 1).

Similarly, as shown in FIGS. 12 and 13, the square value of the image value of the first image data at the target position and the square value of the image value of the template data at the target position are calculated, and cumulatively added according to the imaging position. Get the value. After that, by taking the square root of the cumulative addition value, the L2 norm of the first variation characteristic and the L2 norm of the second variation characteristic corresponding to all the target pixels j (j=1 to 20000) can be calculated. ..

The process of calculating the L2 norm of the first variation characteristic will be described with reference to FIG. The computer 150 squares the image value f _{(j, 1)} of the photoacoustic image data (first image data) 2001 corresponding to the first light irradiation at a certain pixel j and squares the image value f _{(j, 1)} corresponding to the first light irradiation. The generated first image data 1101 is generated. Then, the computer 150 stores f _(j,1) ² in the memory space for storing the L2 norm data of the first variation characteristic. Next, the image value f _{(j, 2)} of the photoacoustic image data corresponding to the second light irradiation is squared, and the value {f _{(j (j} ) is cumulatively added to the memory space for storing the L2 norm data of the first variation characteristic. _{, 1)} ² +f _{(j, 2)} ² } is stored. The same process is repeated until the ninth photoacoustic image data 2009 is obtained, and {f _(j,1) ² +f _(j,2) ² +... +f _(j,9) ² } is stored in the memory space. Finally, the computer 150 calculates the square root of {f _(j,1) ² +f _(j,2) ² +... +f _(j,9) ² } and stores it in the memory space. The L2 norm data calculation of the variation characteristic is completed (process 2). The size of the memory space needs to be at least as large as the size of the composite image data.

Next, the process of calculating the L2 norm of the second variation characteristic will be described with reference to FIG. The computer 150 squares the image value g _{(j, 1)} of the sensitivity information (template data) 2010 corresponding to the first light irradiation at a pixel j, and squares the sensitivity information corresponding to the first light irradiation. 1201 is generated. The computer 150 stores g _(j,1) ² in the memory space for storing the L2 norm data of the second variation characteristic. Next, a value {g _{(j,1 ) obtained} by squaring the image value g _(j,2) of the sensitivity information corresponding to the second light irradiation and cumulatively adding it to the memory space for storing the L2 norm data of the second variation characteristic. ₎ ² +g _(j,2) ² }. The same process is repeated until the squared sensitivity information 1209 corresponding to the ninth light irradiation is obtained, and {g _(j,1) ² +g _(j,2) ² +... +g _{(j,9) is} stored in the memory space. ² } is saved. Finally, when the computer 150 calculates the square root of {g _(j,1) ² +g _(j,2) ² +... +g _(j,9) ² } and stores it in the memory space, the second variation of the pixel j is obtained. The L2 norm data calculation of the characteristic is completed (process 3). The size of the memory space needs to be at least as large as the size of the composite image data.

The computer 150 substitutes the results of the processes 1 to 3 shown in FIGS. 11 to 13 into the equation (4) to determine the inner product of the first variation characteristic and the second variation characteristic in all the target pixels. The correlation value (takes a value of -1 to 1) can be calculated. Here, data that is a set of correlation values corresponding to all target pixels j (j=1 to 20000) is referred to as correlation value image data (correlation information). Moreover, what displayed the correlation value image data as an image is called a correlation value image.

A feature of this processing is that no matter how the number of elements of the first variation characteristic vector and the second variation characteristic vector increases, it is necessary to save the multiplied image data of the photoacoustic image data and the sensitivity information for each light irradiation. It does not require a huge amount of memory because it has only memory space. Further, since the calculation is only multiplication of the photoacoustic image data and the sensitivity information, there is an advantage that the processing is simple and can be easily applied to real-time processing.

For example, a pixel having a light absorber such as a blood vessel and corresponding to a target has a correlation value of 0.8, and a pixel having an artifact has a correlation value of 0.2. In this way, the target and the artifact can be discriminated for each pixel from the magnitude of the correlation value.

The inner product calculation operation of the first variation characteristic and the second variation characteristic will be described in more detail, focusing on a specific pixel in the combined image data. As described above, in the present embodiment, the processing is performed using the values of X1 and Y1 at which the image quality such as the contrast of the combined image data is optimal, and the values of X2 and Y2 at which the correlation value is optimally calculated. X1 and X2 and Y1 and Y2 can take different values. From here, the matters to be considered when determining the values of X1 and Y1 and the values of X2 and Y2 will be described. That is, items to be considered when determining the imaging range of the first image data and the imaging range of the second image data will be described.

First, when generating composite image data, it is desirable to set the values of X1 and Y1 so that the second image data 1001 to 1009 are included in a region where light is sufficiently strongly distributed inside the subject. Even in the region where the light is diffused inside the subject, the luminance values of the second image data 1001 to 1009 become extremely small in the portion where the light intensity is weak. This is because, in this case, if the averaging process is performed to generate the composite image data using the extremely small luminance value component, an adverse effect such as a decrease in the contrast of the composite image data occurs.

Therefore, when generating the synthetic image data, it is better to match the imaging range of the image data 1001 to 1009 to the region where the light intensity is somewhat high among the regions where the light diffuses inside the subject. Typically, the imaging range of the image data 1001 to 1009 defined by X1 and Y1 is smaller than the area where light diffuses inside the subject.

On the other hand, when calculating the correlation value, it is preferable to set the values of X2 and Y2 so that the region where the light diffuses inside the subject and the region of the first image data 2001 to 2009 are as similar as possible. That is, it is preferable to make the imaging range of the first image data larger than the imaging range of the second image data. This is because if the values of X2 and Y2 are made as large as possible, the probability that the first image data 2001 to 2009 include the target position becomes higher when the target position is focused. In other words, increasing the values of X2 and Y2 as much as possible increases the possibility of increasing the number of times the first image data 2001 to 2009 overlap each other when focusing on the position of interest.

When the number of times the first image data 2001 to 2009 overlaps increases, the sample information that can be used for calculating the correlation value increases, and therefore it is expected that the accuracy of identifying the blood vessel structure will improve. Therefore, it seems that the larger the values of X2 and Y2, the better. However, even if X2 and Y2 are set to be larger than a region where light is diffused inside the subject, the determination is performed using a portion where effective photoacoustic image data does not exist, and improvement of the determination accuracy cannot be expected. That is, it is preferable that the imaging range of the first image data is larger than the imaging range of the second image data and is less than or equal to the diffusion area of the irradiation light.

Here, description will be given with reference to the equation (4). Increasing X2 and Y2 leads to an increase in the value of n because the number of times the first image data 2001 to 2009 overlap each other increases. However, when the imaging range of the first image data 2001 to 2009 includes a portion where effective photoacoustic image data does not exist, a value close to 0 increases as an element of the first variation characteristic vector f of a certain pixel. The value of f*g does not increase, since For the same reason, the value of the L2 norm |f| of the first variation characteristic vector f also does not increase.

On the other hand, when the sensitivity information 2010 has effective sensitivity in a portion where effective photoacoustic image data does not exist in the first image data 2001 to 2009, the L2 norm |g| of the second variation characteristic vector g Values can increase. Therefore, if X2 and Y2 are redundantly increased, as a result, the correlation value derived by the equation (4) may be calculated small. It can be seen that if the correlation value of the blood vessel structure is to be made as high as possible, it is not so good that X2 and Y2 are increased, and there is an optimum value.

Here, how the imaging range of the image data generated for each light irradiation affects the calculation accuracy of the combined image data and the correlation value will be described with reference to FIG.

First, FIG. 4A is a composite image of one wire phantom when the imaging range of the second image data that can appropriately generate the composite image data is set. On the other hand, FIG. 4B is a composite image of one wire phantom when the imaging range of the second image data is larger than that in FIG. 4A. In FIG. 4B, it can be seen that the brightness value of the wire is lower than that in FIG. That is, it is understood that FIG. 4A having a smaller imaging range is more preferable than FIG. 4B in the calculation of the composite image.

Next, FIG. 4C is an image showing the spatial distribution of the correlation values of one wire phantom when the same imaging range as in FIG. 4A is set. On the other hand, FIG. 4D shows the correlation value of one wire phantom when the same imaging range as that of FIG. 4B, that is, the imaging range larger than that of FIG. 4C is set. It is an image showing the spatial distribution of. In FIG. 4D, it can be seen that the contrast of the image is improved as compared with FIG. 4C. That is, as the image showing the correlation value, FIG. 4D having a large imaging range is preferable to FIG. 4C. As is clear from this example, it can be seen that the appropriate imaging range is different between the case where the composite image data is calculated and the case where the correlation information is calculated. Further, it can be seen that it is preferable to make the imaging range for calculating the correlation information larger than the imaging range for calculating the composite image data. By setting such an imaging range, it is possible to improve both the image quality of the composite image data and the accuracy of the correlation information.

FIG. 14A shows a positional relationship when the first image data 3001 to 3004 obtained when the light irradiation unit 110 and the reception unit 120 capture images at different imaging positions are arranged in the memory space. It shows a state of being projected on the XY plane. The first image data 3001 to 3004 are image data captured at mutually different image capturing positions A to D and having a light diffusion region as an imaging range. Since the imaging positions A to D are different from each other, the first image data 3001 to 3004 corresponding to the imaging positions A to D are arranged at different positions in the memory space. Pixels 1 to 4 are defined in the memory space.

As shown in FIG. 14B, the pixel 1 is included only in the image data 3001 corresponding to the image pickup position A. The pixel 2 is included in the two image data 3001 and 3002 corresponding to the imaging position A and the imaging position B. Further, the pixel 3 is included in the three image data 3001, 3002, and 3003 corresponding to the image pickup position A, the image pickup position B, and the image pickup position C. The pixel 4 is included in the four image data 3001 to 3004 of the imaging position A, the imaging position B, the imaging position C, and the imaging position D.

In this way, image data corresponding to different image pickup positions may overlap at a certain pixel position. In this case, as the image value of a certain pixel, a process of using a cumulative addition value obtained by cumulatively adding all overlapping image values, or using an addition average value obtained by dividing the cumulative addition value by the number of times of cumulative addition is performed. What kind of calculation is performed may be any calculation according to the purpose of the user, and is not limited to a specific calculation. Further, as is clear from FIG. 14A, the same number of times of overlapping of pixel data does not necessarily occur for all pixels, and the number of times of overlapping may differ depending on the position of the pixel.

In FIG. 14B, the image values of the pixels 4 of the image data 3001 to 3004 corresponding to the imaging positions A to D are f _(4,1) , f _(4,2) , f _(4,3) , Let f _(4,4) .

Next, it is assumed that the position of the receiving unit 120 is fixed for each light irradiation and the sensitivity information is fixed. It is assumed that there is a light absorber of the same size as the pixel at the position of pixel 4. Further, it is assumed that the sensitivity of the receiving unit 120 in the image data generation space is not uniform, the sensitivity of the center of curvature of the hemisphere is high, and the sensitivity decreases as the distance from the center decreases. That is, it is assumed that the sensitivity has a spatial distribution in the image data generation space. In this case, when the light absorber existing at the target position is imaged at different positions, the image value of the imaged light absorber is changed due to the change in the relative positional relationship between the receiving unit 120 and the pixel where the light absorber exists. Different values can be taken for each imaging position.

This state is shown using FIG. FIG. 15A shows the relative positional relationship between the sensitivity distribution 125 of the receiver 120 and the pixel 4 in each light irradiation. The image values at the pixel 4 of the template data corresponding to the imaging positions A to D are g _(4,1) , g _(4,2) , g _(4,3) , and g _(4,4) , respectively. As shown in the graph of FIG. 15B, the image value of the template data may be different for each light irradiation even though the pixel 4 is the same. The variation of the image value of the template data shown in FIG. 15B corresponds to the image value sequence of the template data. This can be seen by referring to the plot data of the image values of the template data along the line segment 1310 in the X direction shown in FIG. It is the effect of having sex. As described above, the template image value when it is assumed that the light absorber having the same size as the pixel exists at the target position has a characteristic that it can vary depending on the image capturing position due to the influence of the sensitivity distribution 125 of the receiving unit 120. .. Further, if the position where the light absorber is present is different, the image value of the light absorber in the photoacoustic image data may also be different due to the influence of the sensitivity distribution 125 of the receiver 120.

From this, it is understood that the image value in the photoacoustic image data is determined by two parameters, that is, the existing position of the light absorber and the sensitivity of the photoacoustic apparatus. First, depending on the pixel position, the image value of the light absorber in the photoacoustic image data can be different. Further, due to the change in the relative positional relationship between the light irradiation unit 110, the receiving unit 120, and the pixels, which is caused by the driving unit 130 driving the light irradiation unit 110 and the receiving unit 120 and performing mechanical scanning. The image value of the light absorber in the photoacoustic image data changes.

Although the size of the assumed light absorber has been described as the same size as the pixel when the template data is obtained, the present invention is not limited to this as long as the template data can be appropriately generated. The shape of the light absorber is not limited to a cube, a square, a sphere, a circle, or the like, and is not limited to a specific shape as long as the template image value can be appropriately generated.

(S560: Step of displaying image based on combined image data and correlation information)
The computer 150 as a display control unit generates an image based on the combined image data obtained in S540 and the correlation information obtained in S550, and causes the display unit 160 to display the image.

The computer 150 may display the composite image in which the image value of the composite image data is assigned to the brightness value of the display image and the correlation image in which the value indicated by the correlation information is assigned to the brightness value of the display image. For example, when the image value or the value indicated by the correlation information is a positive value, it is assigned to the brightness, and when the image value or the value indicating the correlation information is a negative value, the display image in which the brightness is 0 may be generated. The display mode of each display image may be any display mode such as superimposed display, side-by-side display, and switching display.

Further, the computer 150 may generate an image in which the combined image data and the correlation information are combined and display the image on the display unit. For example, each of the image value of the composite image data and the value indicated by the correlation information may be assigned to any one of hue, lightness, and saturation. The image value of the composite image data may be assigned to the lightness, and the value indicated by the correlation information may be assigned to the hue and the saturation. That is, luminance may be assigned to the image value of the composite image data, and the value indicated by the correlation information may be assigned to the color map. For example, display control is performed such that the composite image data is displayed in red when the correlation value at the position of interest is 0.3 or higher, which is highly likely to be the target, and is displayed in blue when the correlation value is less than 0.3. be able to.

Further, the computer 150 determines whether the attention position corresponds to the target or is something other than the target such as an artifact based on the correlation information, and determines the display mode of the composite image data based on the determination result. Good.

For example, the value indicated by the correlation information at the position of interest is compared with the reference value, and if the value indicated by the correlation information indicates that the correlation is higher than the reference value, then the position of interest is determined to be the target. On the other hand, when the value indicated by the correlation information at the target position indicates that the correlation is lower than the reference value, it is determined that the target position is other than the target. It can be said that this determination result is presence information indicating the possibility that the target exists at the attention position. The method of setting the reference value will be described later.

For example, when it is determined that the attention position is the target, the process of validating the combined image data at the attention position may be performed. Further, when it is determined that the attention position is other than the target, a process of invalidating the combined image data at the attention position may be performed. It should be noted that only one of the validation and the invalidation may be performed, or both may be performed.

As an example of the activation processing method, the image value of the corresponding pixel is multiplied by a coefficient of 1 or more to perform amplification processing, or color-coded display is performed in a color having good visibility, or a combination of amplification and color-coded display is used.

Examples of the invalidation processing method include attenuating the image value of the corresponding pixel by multiplying it by a coefficient less than 1, or performing color-coded display in a color with poor visibility, or a combination of attenuation and color-coded display. As the attenuation processing, for example, an image value of a pixel to be invalidated may be multiplied by 0 to generate a display image in which a portion other than the target region is substantially set to 0. Alternatively, the image value of the pixel to be invalidated may be multiplied by a numerical value less than 1 to perform processing so that the boundary between the valid area and the invalidated area is smoothly connected.

In addition, an image that has been subjected to at least one of the validating and invalidating processes and an image that has not undergone the validating and invalidating processes are displayed in parallel on the display unit 160, in a superimposed display, or alternately displayed. May be.

It should be noted that any valid or invalid processing method may be adopted as long as the ability to discriminate between the target area and other areas is high.

The user may judge empirically with respect to the reference value which is the criterion for distinguishing the target from the artifact. Further, even if the correlation value of the pixel corresponding to the region where the target is clearly not present, for example, the acoustic matching material 210 where the subject 100 does not exist is analyzed and the range of the correlation value taken by the artifact is grasped in advance. Good. Then, an area corresponding to a correlation value higher than the correlation value range grasped in advance may be determined as a target. At that time, the user may specify a region where the target does not exist, and the computer 150 may calculate a correlation value as a criterion for determining an artifact from the correlation value range of the region. Further, the computer 150 may display the correlation value in the area where the target does not exist on the display unit 160, and the user may determine the correlation value as the determination criterion. The maximum value, the average value, the intermediate value, the variance value of the correlation values in the region where it is clear that the target does not exist, or a value obtained by combining these parameters may be used as the determination criterion. The method of calculating the reference correlation value is not limited to a specific calculation method as long as the correlation value that can determine the target and the artifact can be determined.

Also, the display range of the correlation value that the user enables or disables may be adjusted. That is, the user may freely change the reference value. At that time, the user may be allowed to specify the reference value from the input unit 170 or the display unit 160 composed of a touch panel. Numerical value specifying methods include, but are not limited to, a method in which a user inputs numerical values in a numerical value input field, a specifying method using a slider bar or a dial, and the like.

Further, the user may specify and display an arbitrary correlation value range. At that time, a plurality of correlation value ranges may be designated and displayed. For example, voxels having correlation values of 0.2 to 0.3, 0.5 to 0.7, and 0.85 to 0.9 are displayed. The method of designating and displaying the correlation value is not limited to a specific method as long as the visibility of the image is improved.

Hereinafter, a method of determining the reference value for the correlation value with reference to the synthetic image data will be described. FIG. 16 shows a state in which the composite image data of the subject 100 is displayed by MIP (maximum intensity projection) in three cross sections. FIG. 16 shows a state in which the surface of the subject 100 is kept parallel to the XY plane by the holding unit 200. Here, the region in which it is clear that the target does not exist refers to a region where the acoustic matching material 210 exists outside the subject 100, such as regions 4010, 4011, 4012, and 4013 in the XZ sectional image 4001. Alternatively, it indicates a region where the acoustic matching material 210 exists outside the subject 100, such as regions 4020, 4021, 4022, and 4023 in the YZ cross-sectional image 4002. In this method, the correlation value in the region where the target does not exist is referred to and used as the reference value for the discrimination between the target and the artifact. For example, all the pixels in the area where the target does not exist, or the maximum value or the average value of the correlation values in a certain fixed area can be used as the reference value for discriminating the target from the artifact. Further, the standard deviation σ of the correlation value in the region where the target does not exist may be obtained, and (average value+n*σ) may be used as the reference value for the discrimination between the target and the artifact. In this case, n can take any value. As long as the reference value for discrimination is derived from the correlation values of a plurality of pixels in the region where the target does not exist, the type of calculation to be performed on the correlation value is specific so that the target and the artifact can be accurately discriminated. Not limited.

As a reference value for discrimination, not the parameter value itself in the region where the target does not exist, but a value obtained by adding some processing to the parameter value, for example, maximum value, average value, intermediate value, variance value, or a combination of these parameters. A value may be used. However, the type of calculation for deriving the reference value from the parameter is not limited to a specific one as long as the target and the artifact can be distinguished.

Regarding the reference value of the parameter that is used as the discrimination criterion between the target and the artifact, it is not always necessary to apply the same reference value to all the imaged areas, but the imaged area is divided and different reference values are set for each area. You may apply.

Next, FIG. 17 shows a three-sectional image of the synthetic image data of the subject 100. Similar to FIG. 16, FIG. 17 shows a state in which the surface of the subject 100 is held flat. The cross-sectional image 4001 and the cross-sectional image 4002 are not MIP displays, but are a cross-sectional image at the line 4030 of the composite image data (cross-section 1) and a cross-sectional image at the line 4031 of the composite image data (cross-section 2), respectively. As shown in FIG. 17, the XY cross-sectional image 4000 may be divided into, for example, a plurality of regions, and the reference value may be calculated from the regions that are present in the respective regions and in which it is clear that the target does not exist. In this example, 16 types of reference values can be calculated at the maximum, and the targets and the artifacts can be discriminated using the individual reference values in the regions 1 to 16 of the subject 100.

Furthermore, as shown in FIG. 17, a region in which it is clear that no target is present is divided into a plurality of regions in the Z direction, such as A to F, and the reference value is calculated individually for each region in the Z direction. You can also In this case, different reference values can be applied in the Z direction of the subject 100, and can be used as the reference value of the parameter serving as the reference for discriminating between the target and the artifact.

As a result of examination by the present inventor, even in a region where it is clear that no target exists, the correlation value tends to be high near the surface of the subject 100, and the correlation value tends to decrease as the distance from the surface of the subject 100 increases. I found it. That is, when the target and the artifact are discriminated from each other in the region close to the surface of the subject 100, it is preferable to use the reference value that refers to the correlation value of the region near the surface of the subject 100 in which the target does not exist. On the other hand, in a region far from the surface of the subject 100, it is preferable to use a reference value that refers to a correlation value of a region in the region far from the surface of the subject 100 where no target exists.

In the area far from the surface of the object 100, when the target is discriminated using the reference value that refers to the correlation value near the surface of the object 100, the actual correlation value of the artifact in the area far from the surface of the object 100 is determined. Also discriminates artifacts with a high correlation value.

In this case, in a region far from the surface of the subject 100, there is a possibility that the artifact and eventually the accuracy of target discrimination may deteriorate. On the other hand, by setting the reference value based on the correlation value at the position of interest and the distance from the surface of the subject 100 to the position of interest, it is possible to improve the accuracy of artifact discrimination and thus the accuracy of target discrimination. That is, for example, when using the correlation value in the vicinity of the surface of the subject 100, it is preferable to use a reference value obtained by weighting the correlation value with a smaller value in a region farther from the surface of the subject 100.

FIG. 18 is a table showing which region corresponds to the reference value to be adopted as the reference value applied to each of the pixels 4040, 4041, and 4042. For example, FIG. 18 shows that for the pixel 4040, it is preferable to adopt the reference value corresponding to the region 6 in the XY plane and the region C in the Z direction. The computer 150 can determine the reference value to be applied to each pixel according to a table showing the relationship between each pixel and the application region of the reference value as shown in FIG.

Next, FIG. 19 shows MIP display of composite image data of the subject 100 in three sections. Unlike FIG. 16, FIG. 19 shows a state in which the surface of the subject 100 is held on a curved surface.

FIG. 20 is a three-section image of the synthetic image data of the subject 100 shown in FIG. In FIG. 20, the cross-sectional image 4101 and the cross-sectional image 4102 are not MIP display, but are a cross-sectional image (cross-section 1) on the line 4130 of the composite image data and a cross-sectional image (cross-section 2) on the line 4131 of the composite image data, respectively.

Even in such a case, as shown in FIG. 20, according to the shape of the subject surface in the cross-sectional image, division in the Z direction is performed so that the distance in the Z direction from each pixel on the subject surface is the same. Aspects can be determined. When the same processing is performed on all the cross sections of the synthetic image data of the subject 100, it is possible to form a plurality of regions having the same thickness in the Z direction with respect to the three-dimensional surface shape of the subject 100. Further, in addition to dividing the region in the Z direction, as shown in FIG. 20, the region of the XY cross-sectional image 4000 is also divided, and the reference value is set in each region to determine the target inside the subject 100. You can go.

As shown in FIG. 20, a region where no target is present is divided into regions on the XY cross section and regions in the Z direction, and a reference value is calculated for each region. Then, according to the pixel position inside the subject 100 for which it is desired to discriminate the target and the artifact and the distance between the surface of the subject 100 and the distance between the surface of the subject 100 and the target, refer to a reference value of a region where there is no target. To do.

As shown in FIG. 20, the pixel 4200 has the same distance from the surface of the subject 100 as the area B of the area where the target does not exist. Therefore, as a reference value for target discrimination, a value based on a parameter such as a correlation value of a region B in a region where no target exists is used. With respect to the pixels 4201, 4202, 4203, 4210, and 4211, the reference determination target value is set in the same way.

FIG. 21 is a table showing which region the reference value corresponds to as the reference value applied to each of the pixels 4200, 4201, 4202, 4203, 4210, and 4220 shown in FIG. For example, FIG. 21 shows that for the pixel 4200, it is preferable to adopt the reference value corresponding to the region 10 in the XY plane and the region B in the Z direction. The computer 150 can determine the reference value to be applied to each pixel according to the table showing the relationship between each pixel and the application region of the reference value as shown in FIG.

In this way, by dividing the area where the target does not exist into a plurality of two-dimensionally and three-dimensionally and deriving a reference value for discriminating the target from the parameters such as the correlation value in each divided area, the target is accurately discriminated. be able to.

However, the method of dividing the area where the target does not exist is not limited to the method described above. The method is not limited to a particular method as long as the target discrimination can be performed well.

Further, when the target and the artifact are discriminated, not only the correlation value but also the image value of the image to be discriminated such as the image value of the composite image may be taken into consideration for the discrimination.

The blood vessels on the surface of the subject may be removed to generate an image in which blood vessel structures far from the surface of the subject remain. Further, the visibility of the blood vessel structure can be improved by, for example, color-coding the blood vessels near the surface of the subject and the blood vessels existing in the region far from the surface of the subject.

The computer 150 may determine the imaging range of the image data used to calculate the correlation value based on a user's instruction. Further, the computer 150 may generate candidate image data corresponding to a plurality of imaging ranges, perform a correlation value calculation test using the candidate image data, and derive an optimal imaging range of the image data. ..

It is not always necessary to generate both the synthetic image data and the correlation value. Only the composite image data may be generated, or only the correlation value may be generated. It may be possible to select whether to generate only one or both.

It should be noted that, in the present embodiment, when setting the value of the correlation value serving as the discrimination criterion between the target and the artifact, it is also effective to analyze a region in which the target is clearly absent in the imaging region. It was

Up to this point, it has been described that the target and the artifact are discriminated using the correlation value as a parameter, but the parameter is not limited to the correlation value. The parameter is not limited to the correlation value as long as the parameter can be calculated realistically and the target and the artifact can be discriminated. That is, the computer 150 as the characteristic information acquisition unit may acquire the characteristic information indicating the characteristic of the image value group of the plurality of first image data at the attention position. Then, the computer 150 as the presence information acquisition unit may acquire the presence information indicating the possibility that the target exists at the position of interest, based on the characteristic information. For example, a variance value, an expected value, a kurtosis, an entropy value, or the like can be adopted as the characteristic information that can determine the possibility of existence of the target.

The image data may be displayed two-dimensionally or three-dimensionally, or may be displayed by performing image processing such as volume rendering or surface rendering.

Although the case has been described as an example where the pixels are validated or invalidated in the composite image based on the correlation information, the target of validation or invalidation is not limited to the composite image. For example, pixels of an oxygen saturation image obtained from a combined image obtained by using different light wavelengths or an image of an image showing a concentration distribution of glucose, cholesterol, or the like may be enabled or disabled.

Although the photoacoustic apparatus has been described as an example in the present embodiment, the information processing according to the present embodiment is not necessarily limited to application to the photoacoustic apparatus. In an information processing device that performs information processing using a plurality of image data, the present embodiment is applicable even when at least one pixel of the plurality of image data overlaps. In the information processing apparatus, the type of image data handled need not necessarily be related to medical diagnosis, and may be any type of image data.

In the embodiment of the present invention, the correlation value of the inner product has been described as an example of calculation. However, it is possible to perform a plurality of types of calculations other than the correlation value of the inner product by sequentially using the image data obtained by one imaging. Therefore, correlation value, variance value, expected value, kurtosis, histogram frequency data, entropy, average value, cumulative addition value, cumulative addition count, deviation, covariance, standard deviation, correlation coefficient, cumulative multiplication, cumulative subtraction, The calculation result realized by any combination of cumulative division, addition, subtraction, multiplication, division, etc. is included in the range of parameters used for target discrimination in the present embodiment.

Furthermore, a mode may be prepared in which the computer 150 displays images with the reference value changed at the same time or by switching, and the user or the computer 150 confirms the target determination condition from the image and determines the optimum reference value. .. That is, the computer 150 may automatically determine the optimum reference value. Note that the display mode of a plurality of images with different reference values is not limited to a particular one as long as it is effective for determining the optimum reference value.

The reference value thus determined may be displayed so that the user or a third party can confirm it. In addition, the computer 150 is provided with a display section so that the user or a third party can confirm that the user is determining the optimum reference value or that the computer 150 is set in the mode for determining the optimum reference value. It may be displayed on 160.

The computer 150 may acquire the photoacoustic image data (third image data) in the imaging range (third imaging range) smaller than the imaging range of the first image data (first imaging range). Good. Then, the computer 150 may calculate a correlation value between the image value sequence of the third image data at the attention position and the image value sequence of the template data at the attention position. The method for acquiring the third image data is the same as that for the first image data or the second image data, and thus the description thereof is omitted. The computer 150 may generate the third image data by extracting a part of the first image data. Also, the second image data may be used as the third image data. Further, the method of calculating the correlation value is the same as the method of calculating the correlation value described above, and therefore will be omitted.

When the inventor compares the correlation value (first correlation value) obtained from the first image data and the correlation value (second correlation value) obtained from the third image data, in a region where the target does not exist Found that the correlation value of the first correlation value tends to be lower than the second correlation value. It was also found that in the region where the target exists, the first correlation value tends to be the same as or higher than the second correlation value. FIG. 22A shows a profile of the first correlation value and the second correlation value in the depth direction of the region where the target exists. On the other hand, FIG. 22B shows a profile of the first correlation value and the second correlation value in the depth direction of the area where the target does not exist, for example, the artifact area. From FIG. 22 as well, it can be seen that the first correlation value tends to be suitable for grasping the region where the target does not exist, and the second correlation value tends to be suitable for grasping the region where the target exists. ..

Therefore, the computer 150 may generate an image including an image showing the spatial distribution of the first correlation value and an image showing the spatial distribution of the second correlation value, and display the image on the display unit 160.

(Other embodiments)
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or apparatus reads the program. This is the process to be executed.

150 computer 160 display

Claims (15)

A plurality of first image data corresponding to the plurality of times of light irradiation derived from a photoacoustic wave generated from the subject by a plurality of times of light irradiation to the subject, and a plurality of corresponding to the plurality of times of light irradiation. Image data acquisition means for acquiring the second image data of
Template data acquisition means for acquiring a plurality of template data corresponding to the plurality of times of light irradiation,
Correlation information acquisition means for acquiring correlation information indicating the correlation between the image value sequence of the plurality of first image data at the attention position and the image value sequence of the plurality of template data at the attention position,
Synthetic image data generation means for generating synthetic image data by synthesizing the plurality of second image data,
Display control means for displaying on the display means an image based on the correlation information and the composite image data;
Have
An information processing apparatus, wherein an imaging range of each of the plurality of first image data is different from an imaging range of each of the plurality of second image data. The information processing apparatus according to claim 1, wherein an imaging range of each of the plurality of first image data is larger than an imaging range of each of the plurality of second image data. The information processing apparatus according to claim 1, wherein the image data acquisition unit generates the plurality of second image data by cutting out a part of each of the plurality of first image data. .. The image data acquisition means, by performing reconstruction for each of a plurality of reception signals obtained by receiving the photoacoustic wave generated from the subject by the plurality of times of light irradiation to the subject The information processing apparatus according to claim 3, wherein the plurality of first image data are generated. 5. Each of the plurality of template data is template data that reflects a spatial sensitivity distribution of a photoacoustic device that executes the light irradiation and the reception of the photoacoustic wave a plurality of times. The information processing apparatus according to any one of items. Each of the plurality of template data is an estimated value of an image value of image data that can be generated when a light absorber is present at the attention position in each of the plurality of light irradiations. Item 6. The information processing device according to any one of items 1 to 5. The template data acquisition means, for the plurality of times of light irradiation, based on the relative position information of the receiving means for receiving the photoacoustic wave and the position of interest, by assigning coordinates to predetermined template data, the plurality of The information processing apparatus according to claim 1, wherein template data is acquired. The template data acquisition means is
Regarding the plurality of times of light irradiation, the relative position information between the receiving unit that receives a photoacoustic wave and the position of interest is acquired, and the template data corresponding to the relative position information is read from the storage unit to obtain the plurality of templates. The information processing apparatus according to claim 1, wherein the information processing apparatus acquires data. 9. The information processing according to claim 1, wherein the image value included in the image value sequence and the image value of the template data are associated with a common light irradiation. apparatus. The information processing apparatus according to claim 1, further comprising display control means for displaying an image based on the information indicating the correlation on a display means. The information according to claim 1, wherein the plurality of image data are image data generated such that a light irradiation position or a photoacoustic wave reception position is different for each of the plurality of light irradiations. Processing equipment. A plurality of first image data corresponding to the plurality of times of light irradiation derived from a photoacoustic wave generated from the subject by a plurality of times of light irradiation to the subject, and a plurality of corresponding to the plurality of times of light irradiation. Generate the second image data of
Acquiring the characteristic information representing the characteristic of the image value group of the plurality of first image data at a certain position,
Based on the characteristic information, obtain the presence information indicating the possibility that the target exists at the certain position,
Generating composite image data by combining the plurality of second image data,
An information processing method, wherein the respective imaging ranges of the plurality of first image data are different from the respective imaging ranges of the plurality of second image data. The information processing method according to claim 12, wherein an imaging range of each of the plurality of first image data is larger than an imaging range of each of the plurality of second image data. The information processing method according to claim 12 or 13, wherein the plurality of second image data is generated by cutting out a part of each of the plurality of first image data. A program that causes a computer to execute the information processing method according to any one of claims 12 to 14.