KR20150121872A - Apparatus and method for obtaining photoacoustic image using motion compensatiion - Google Patents

Abstract

The present invention relates to a technology of obtaining a photoacoustic image by using motion compensation. A method for obtaining a photoacoustic image comprises the following steps: receiving an ultrasonic signal generated by a photon which is absorbed into an object to be measured after the photon is applied to the object to be measured; obtaining a plurality of frame images of the object to be measured from the ultrasonic signals received according to time by repeating processes of applying the photon and receiving the ultrasonic signal; compensating for each movement generated in an image of a previous frame with respect to the most recent frame among the obtained frame images; and generating one photoacoustic image by accumulating a plurality of frame images of which movements are compensated for.

Description

BACKGROUND OF THE INVENTION Field of the Invention [0001] The present invention relates to an apparatus and method for acquiring photoacoustic images using motion compensation,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a medical imaging technique for diagnosis, analysis or treatment, and more particularly, to a photoacoustic image acquisition apparatus and method for acquiring a photoacoustic image from an object to be measured by using a light application laser and an ultrasonic probe .

An ultrasound (US) image is obtained by applying an ultrasound signal to an observation region in a human body using an ultrasonic probe and receiving an ultrasound signal reflected from the tissue to extract information contained in the signal, And imaging equipment. This has the advantage that real-time images without harm to human body can be obtained at low cost when compared with other medical imaging systems such as X-ray, CT, MRI, and PET.

Next, a photoacoustic (PA) image is obtained by applying a photon to an observation region in a human body, receiving an ultrasound signal directly generated by photons absorbed in the tissue, and extracting image information from the signal. This unique situation, where photons are absorbed into tissues and generates ultrasonic waves, is a phenomenon that occurs when tissue is heated when it absorbs photons. When a pulsed laser is used to illuminate an absorbent tissue structure, the temperature of the tissue changes, The structure expands. The pressure wave propagates out of the expanding structure, and this pressure wave can be detected by an ultrasonic transducer. The advantage of photoacoustic imaging is that it can acquire images based on the optical absorption contrast ratio, but also obtain resolution of ultrasound, which is not only less expensive than MRI but also does not expose patients to ionizing radiation. The prior art cited below provides technical means to utilize such photoacoustic images to diagnose breast cancer.

Therefore, it is required to develop a photoacoustic imaging technique that can acquire a sophisticated image of a patient's part to be measured in a real medical environment considering the implementation environment and application examples on the theoretical basis.

 Development of Photoacoustic Imaging System for Breast Cancer Diagnosis, Soonhyuk Lee, Jiyun Suh, Irena, Korean Medical Physics Society, 2013.

Embodiments of the present invention are directed to overcome the limitations of low image quality due to low signal-to-noise ratio In order to solve the problem of the error caused by the movement of the object to be measured and the deterioration of the image quality of the acquired image as a result.

According to an aspect of the present invention, there is provided a method for acquiring a photoacoustic image, the method comprising: applying a photon to an object to be measured, Receiving an ultrasonic signal; Acquiring a plurality of frame images of the object to be measured from the ultrasound signals received over time by repeating the photon-applying and ultrasonic signal receiving processes by the photoacoustic imaging system; Compensating for motions generated in the image of the previous frame based on the latest frame among the frame images obtained by the photoacoustic imaging system; And generating a photoacoustic image by accumulating the motion-compensated plurality of frame images in the photoacoustic imaging system.

In the method of acquiring a photoacoustic image according to an exemplary embodiment, the step of compensating the motion may include calculating a motion vector of a plurality of frame images acquired at a predetermined time interval from the most recent frame, Estimating image motion vectors between the frames; And compensating motion generated in the image by sequentially connecting the estimated video motion vectors.

In the method of acquiring the photoacoustic image according to an exemplary embodiment, the step of estimating the image motion vector may include: setting a reference macro block as a reference for one frame in the obtained frame image; Setting a reference macroblock at the same position of a previous frame to be compared with the reference macroblock; Calculating a difference between a reference macroblock and a reference macroblock to detect a change from a previous frame based on the most recent frame; And estimating an image motion vector between adjacent frames using the interval between the detected position of the motion and the reference point from the difference between the calculated macroblocks.

In addition, in the method of acquiring the photoacoustic image according to an embodiment, the step of estimating the image motion vector is performed through an iterative search process of sequentially reducing a step size, And a macroblock spaced apart by an initial step size from a reference macroblock corresponding to the macroblock, and updates a point where the calculated difference value is the smallest to a new macroblock, decreases the step size, The calculation and the macro block update are repeated a predetermined number of times.

According to another aspect of the present invention, there is provided a method for acquiring a photoacoustic image according to another embodiment of the present invention, the method comprising: a photoacoustic imaging system applying a photon to a measured object at predetermined wavelengths, Receiving ultrasound signals generated by the ultrasound system; Acquiring a plurality of frame images of the object to be measured from the ultrasound signals received over time by repeating the photon-applying and ultrasonic signal receiving processes by the photoacoustic imaging system; Compensating for motions generated in the image of the previous frame based on the latest frame among the frame images obtained by the photoacoustic imaging system; And generating a photoacoustic image by accumulating the motion compensated plurality of frame images based on an image of a wavelength at which the feature points are most clearly displayed among the images obtained by the photoacoustic imaging system by the wavelengths, .

In the method of acquiring a photoacoustic image according to another embodiment, the step of applying a photon for each wavelength may include obtaining a frame image by applying a photon while sequentially changing a predetermined wavelength, The time interval between acquisition of frame data per wavelength can be maintained constant by repeating the process of acquiring a frame image by applying a photon while changing sequentially.

In the method of acquiring a photoacoustic image according to another embodiment, the compensating the motion may include calculating a motion vector of a plurality of frame images acquired at a predetermined time interval from the latest frame, Estimating image motion vectors between the frames; And compensating motion generated in the image by sequentially connecting the estimated video motion vectors.

In the method of acquiring a photoacoustic image according to another embodiment, the predetermined wavelength may be selected from among a plurality of laser wavelengths which are responsive to a tissue constituting the object to be measured or a lesion tissue located in the object to be measured.

According to an aspect of the present invention, there is provided a photoacoustic imaging system including a light applying unit for applying a photon to an object to be measured; A probe for receiving an ultrasonic signal generated by photons absorbed by the object to be measured; A storage unit for storing a plurality of frame images obtained for the measured object from the ultrasound signals received over time by repeating the photon application and the ultrasonic signal reception process; And a processor for compensating motion generated in an image in a previous frame based on a most recent frame of the frame image, and accumulating the motion compensated frame images to generate one photoacoustic image.

In the photoacoustic imaging system according to an exemplary embodiment, the processing unit may be configured to calculate, for a plurality of frame images obtained at a predetermined time interval from the most recent frame, a video motion vector between adjacent frames at the predetermined time interval from the most recent frame And motion compensated motion can be compensated by successively connecting the estimated video motion vectors.

In addition, in the photoacoustic imaging system according to an exemplary embodiment, the processor sets a reference macro block as a reference for one frame in the obtained frame image, A difference between the reference macroblock and the reference macroblock is calculated in order to detect a change from the previous frame based on the latest frame, The image motion vector between adjacent frames can be estimated using the interval between the detected position of the motion and the reference point.

In addition, in the photoacoustic imaging system according to an embodiment, the processor estimates the video motion vector through an iterative search process of sequentially reducing a step size, Calculating a difference between macroblocks spaced by an initial step size around the reference macroblock, updating a point having the smallest difference value to a new macroblock, decreasing a step size, The update is repeated a predetermined number of times.

In the photoacoustic imaging system according to an embodiment, the light applying unit applies a photon to a measured object at predetermined wavelengths, and the processing unit determines that the extracted image has the most characteristic point, And accumulate the plurality of frame images compensated for the motion to generate one photoacoustic image.

In addition, in the photoacoustic imaging system according to an exemplary embodiment, the light applying unit sequentially obtains a frame image by applying a photon while changing a predetermined wavelength sequentially, sequentially changes a predetermined wavelength again after a predetermined time delay By repeating the process of obtaining the frame image by applying the photon, the time interval between acquisition of the frame data per wavelength can be kept constant.

Further, in the photoacoustic imaging system according to an embodiment, the processing unit may be configured to perform, for a plurality of frame images acquired at a predetermined time interval from the latest frame, The motion vectors are respectively estimated, and the estimated motion vectors are sequentially connected to compensate for motion generated in the image.

Embodiments of the present invention not only provide improved signal-to-noise ratio, axial resolution, and contrast in the image equalization process and the molecular photoacoustic image reconstruction process by compensating for possible motion errors in the photoacoustic image acquisition process Especially, by applying the motion compensation technique to both the photoacoustic image reconstruction and the image equalization, the quality of the image can be dramatically improved.

1 is a flowchart illustrating a method of acquiring a photoacoustic image according to an embodiment of the present invention.
2 is a block diagram illustrating the structure of a photoacoustic imaging system using leveling for the method of FIG. 1 according to an embodiment of the present invention.
3 is a flowchart illustrating a method of acquiring a photoacoustic image using a multi-wavelength according to another embodiment of the present invention.
4 is a block diagram illustrating the structure of a molecular photoacoustic imaging system for the method of FIG. 3 according to another embodiment of the present invention.
5 is a diagram illustrating a structure of a signal processing block for implementing inter-frame motion compensation adopted by embodiments of the present invention.
6 is a block diagram illustrating a photoacoustic imaging system for correcting motion according to an embodiment of the present invention.
FIGS. 7A to 7D are graphs illustrating experimental results showing improved signal-to-noise ratios (SNRs) using image equalization according to an exemplary embodiment of the present invention.
FIGS. 8A to 8C are graphs illustrating experimental results showing improved signal-to-noise ratios (SNRs) using image equalization according to an embodiment of the present invention.

Hereinafter, the features of the environment in which the embodiments of the present invention are implemented and the basic ideas adopted by the embodiments of the present invention under such circumstances are outlined, and then the specific technical means will be described in order.

The existing photoacoustic imaging system imaged this data without compensating for motion artifacts that can occur during data acquisition. In order to improve the resolution, contrast, and signal-to-noise ratio of the photoacoustic image, a method of leveling and restoring various images over time or a molecular photoacoustic imaging technique can be used. However, when the object to be measured is a living tissue, an organ, or an organism, the photoacoustic image reconstructed due to the limitation of the limited laser frequency includes errors, for example, blurring artifacts. In particular, since motion that is necessarily generated at the time of image acquisition degrades the effect of this technique, it is necessary to optimize the degree of image quality improvement by applying motion compensation technique. In addition, if more imaging acquisition time is required for convergence of various progressive ultrasound imaging techniques in fusing photoacoustic image and ultrasound image, the quality of image can be maintained while reducing the data acquisition interval of photoacoustic imaging technique. It is possible to converge the progressive ultrasound image technique and the photoacoustic image.

Embodiments of the present invention proposed below are to propose a method of improving the signal-to-noise ratio of a photoacoustic image or enhancing a contrast in a molecular photoacoustic image by minimizing a motion error in an image reconstruction process. In particular, by calculating the motion of the object on a continuous image frame and compensating the image with the proposed system structure, the object is positioned at the correct position when the image is averaged, minimizing motion artifacts , And to provide users with optimized image quality and enhanced molecular photoacoustic image contrast information for photoacoustic imaging techniques.

To this end, embodiments of the present invention compensate for the motion of previous data based on the most recent data among a plurality of image frame data used in an image processing process, and perform image leveling and molecular photo- Lt; / RTI >

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, these embodiments are for further illustrating the present invention, and the scope of the present invention is not limited thereto.

1 is a flowchart illustrating a method of acquiring a photoacoustic image according to an embodiment of the present invention.

In step S110, the photoacoustic imaging system applies a photon to an object to be measured, and receives an ultrasonic signal generated by the photon absorbed by the object to be measured.

In step S120, the photoacoustic imaging system obtains a plurality of frame images for the object to be measured from the ultrasound signals received over time by repeating the photon application and the ultrasound signal receiving step (S110).

In step S130, the photoacoustic imaging system compensates for the motion of the previous frame, which is generated in the previous frame, based on the latest frame among the frame images obtained through step S120. More specifically, for each of a plurality of frame images acquired at a predetermined time interval from the most recent frame, video motion vectors between adjacent frames at the predetermined time intervals from the most recent frame are respectively estimated. Then, motion estimation is compensated for by sequentially connecting the estimated motion vectors. The data for estimating the video motion vector is not limited to the result after the beam focusing, the result of the detection of the envelope, the log transformed result, or the image that is scattered on the screen.

At this time, the process of estimating the video motion vector may be performed according to the following procedure. First, a reference macro block as a reference is set for one frame in the obtained frame image, and a reference macroblock is set at the same position of the previous frame to be compared. Then, differences between the reference macroblock and the reference macroblock are calculated to detect a change from the previous frame based on the latest frame. From the difference between the calculated macroblocks, an image motion vector between neighboring frames can be estimated using the interval between the detected position of the motion and the reference point.

In addition, the step of estimating the video motion vector may be performed through an iterative search process of sequentially reducing the step size. For this, a photoacoustic imaging system according to an embodiment of the present invention calculates a difference between a first reference macroblock and a macroblock spaced apart from the first reference macroblock by an initial step size around the corresponding reference macroblock, It is possible to estimate the video motion vector by updating the smallest point to a new macroblock, decreasing the step size, and calculating the difference value and updating the macroblock a predetermined number of times.

The method of estimating an image motion vector using the macroblock and the iterative search process will be described in more detail with reference to FIG.

In step S140, the photoacoustic imaging system accumulates the motion compensated frame images through step S130 to generate a photoacoustic image.

2 is a block diagram illustrating the structure of a photoacoustic imaging system using leveling for the method of FIG. 1 according to an embodiment of the present invention. Referring to FIG. 2, a photoacoustic / ultrasound image system 10, And a display device 40. The display device 40 is a display device.

In FIG. 2, the image I (0) obtained from the last acquired image data is used as a reference image, and after the motion compensation for the previously obtained images I (-T) to I (-NT) We propose a visible system architecture. Where T is the time difference between image acquisitions and N is the number of time base frames used for image processing (i.e., N is a natural number).

The photoacoustic / ultrasound imaging system 10 includes a physical structure for acquiring a photoacoustic image frame, and includes a laser applying means for applying a photon to an object to be measured and an ultrasonic signal reflected from the object to be measured The probe may include a probe. 2, the photoacoustic / ultrasound imaging system 10 for acquiring an image acquires a plurality of frame data for image formation over time and stores it in a workstation 20 interlocked with the system. The concrete operation of the individual configuration of the photoacoustic / ultrasound imaging system 10 is not described here because there is a possibility that the essence of the present invention may be blurred.

First, when the leveling is used, the image of the human body region currently being photographed by the user is set as the reference image I (0), which is the most recently acquired image, and I (-T) To-I (-TN), motion compensation is performed on the reference image set in advance and then accumulated, thereby improving the signal-to-noise ratio and reducing the residual image error.

To this end, the workstation 20 for image processing includes memories 21 and 25, which are temporary spaces for storing respective frames, motion compensation means 23 for compensating or accumulating motion between them, (27). The memories 21 and 25 may be implemented as physical hardware capable of storing electronic image data and the motion compensation means 23 and the image accumulation means 27 may be electrically connected to the memories 21 and 25. [ Reads the necessary data, processes the image according to a preset rule, and stores it again in the memory 25 or outputs it to the display device 40.

3 is a flowchart illustrating a method of acquiring a photoacoustic image using a multi-wavelength according to another embodiment of the present invention. Unlike the previously described embodiments of FIGS. 1 and 2, In the case of the acoustic imaging technique, multiple wavelengths are used.

In step S310, the photoacoustic imaging system applies a photon to a measured object at predetermined wavelengths, and receives an ultrasonic signal generated by the photon absorbed by the measured object. Herein, in the process of applying photons for each wavelength, a photon is applied while sequentially changing a preset wavelength, a frame image is acquired, a photon is applied while sequentially changing a preset wavelength again after a predetermined time delay, , The time interval between acquisition of frame data for each wavelength is kept constant.

On the other hand, it is preferable that the predetermined wavelength is selected from a plurality of laser wavelengths which are responsive to a tissue constituting the object to be measured or a lesion tissue located in the object to be measured.

In step S320, the photoacoustic imaging system obtains a plurality of frame images for the object to be measured from the ultrasound signals received over time by repeating the photon application and the ultrasound signal receiving process.

In step S330, the photoacoustic imaging system compensates for motions generated in the previous frame, based on the most recent frame among the frame images obtained in step S320.

This process is based on the following series of methods. First, for a plurality of frame images obtained at a predetermined time interval from the most recent frame, video motion vectors between adjacent frames at the predetermined time intervals from the most recent frame are respectively estimated. Then, motion estimation is compensated for by sequentially connecting the estimated motion vectors.

In step S340, the photoacoustic imaging system accumulates the plurality of frame images compensated for motion on the basis of the image of the wavelength having the most significant feature point among the images obtained for each wavelength through step S330, And generates an image.

FIG. 4 is a block diagram illustrating the structure of a molecular photoacoustic imaging system for the method of FIG. 3 according to another embodiment of the present invention. Referring to FIG. 2, the photoacoustic / ultrasound image system 10, A work station 30 for processing the collected image frames, and a display device 40. [

In the molecular photoacoustic imaging technique of FIG. 4, a multi-wavelength image obtained from I (λ ₀ ) to I (λ _M ) is implemented. Here, [lambda] denotes M + 1 different wavelengths. Unlike the image equalization in FIG. 2, in the embodiment of FIG. 4, the convergence method of each wavelength image is changed according to the application of the molecular photoacoustic image used. Therefore, in using the acquired image, It is preferable to select the data of the wavelength having the minutiae point as the reference image.

This molecular photoacoustic imaging technique is a technique to acquire the contrast of a specific molecule by collecting ultrasound signals generated by transmitting light energy of multiple wavelengths to tissues, and oxygen saturation imaging technique is the representative one. This is based on the fact that the absorbance of hemoglobin with and without oxygen is different for each wavelength. In particular, a method of estimating the oxygen saturation using the ratio of the size of the ultrasound signal obtained using the light energy of 570 nm, which is the same wavelength as 561 nm, which is a wavelength with different absorbance, is used. As a result, the data acquired by the multiple wavelengths ensures the contrast of a particular molecule, and the signal-to-image signal location transformation is a direct cause of reduced contrast.

The photoacoustic / ultrasound imaging system 10 includes a physical structure for acquiring a photoacoustic image frame, and includes a laser applying means for applying a photon to an object to be measured and an ultrasonic signal reflected from the object to be measured The probe may include a probe. 4, the photoacoustic / ultrasound imaging system 10 for acquiring an image acquires a plurality of frame data for an image configuration according to predetermined wavelengths in time, and stores the acquired frame data in a workstation 30 interlocked with the system . The concrete operation of the individual configuration of the photoacoustic / ultrasound imaging system 10 is not described here because there is a possibility that the essence of the present invention may be blurred.

4, when the molecular photoacoustic imaging technique is implemented, the contrast of the image with respect to each wavelength image data is the most important factor for determining the quality of the molecular photoacoustic image to be restored. Therefore, And the quality of the image can be dramatically improved by performing the molecular photoacoustic imaging technique through such multi-wavelength image data.

In addition, data acquisition is not obtained separately for each wavelength, but can be made more efficient in molecular photoacoustic imaging, in which the data change per wavelength is estimated by making the degree of motion between the image and the image uniform by obtaining them alternately. This is because when the time interval between the wavelength data occurs, resolution for a specific molecule may be degraded. For example, data is acquired through the order of I (λ ₀ , 1 to N), I (λ ₁ , 1 to N), ..., I (λ _M , 1 to N) , The time interval between data acquisition between acquiring image data per wavelength may be NT and the quality of the molecular photoacoustic image may be deteriorated. On the other _{hand, I (λ 0, 1)} , I (λ 1, 1), ..., I (λ M, 1), I (λ 0, 2), I (λ 1, 2), ... _{, I (λ M, 2)} , ..., the image data is acquired over a sequence of _{I (λ 0, N),} I (λ 1, N), ..., I (λ M, N) The time interval between acquisition of image data per wavelength is reduced to T, and the quality of the image can be improved because the molecular absorbance characteristic can be improved. In addition, it is possible to variably determine the order of acquiring data and the order of wavelengths according to the number of selected wavelengths and the wavelength conversion time in consideration of efficiency.

To this end, the workstation 30 for image processing includes memories 31 and 35, which are temporary spaces for storing respective frames, motion compensation means 33 for compensating or restoring motion between them, and multi- And restoration means (37). These memories 31 and 35 can store electronic image data and the motion correction means 33 and the multispectral image reconstructing means 37 are connected to these memories 31 and 35 to read the necessary data, And then stores the image data in the memory 35 or outputs the image data to the display device 40 again.

5 is a diagram illustrating a structure of a signal processing block for implementing inter-frame motion compensation adopted by embodiments of the present invention. Prior to describing the signal processing block of FIG. 5, the basic techniques reflected here are separated.

(One) block matching  algorithm( Block Matching Algorithm , BMA )

BMA can be utilized for motion estimation in video compression technology. In order to proceed with motion estimation, reference macro blocks are first set in the most recently acquired image frame. Also, the macroblocks as the control group are set in the previous image frame, and the macroblocks located at the same position as the respective reference macroblocks are selected. The selected reference macroblock and the reference macroblocks are calculated using the mean absolute difference (MAD), for example, as a cost function between the reference macroblocks and the reference macroblocks, Can be calculated using Equation (1).

Where N and M are the number of pixels in the axial and lateral directions of the macroblock, respectively, and C _ij and R _ij denote the reference macroblock and the reference macroblock, respectively.

(2) Three-step search method ( Three Step Searching Technique , TSS )

The cost function is used as MAD as above, and the motion vectors between the respective macroblocks of C _ij and R _ij are sequentially obtained by using TSS. In this case, the TSS uses a method of gradually reducing the step size in the three iterative search steps for optimization (that is, a three-step search method).

In the process of applying the block matching algorithm and the 3-step search method, the value of MAD is firstly divided into 8 steps of the reference macroblock and the selected reference macroblock by an initial step size S pixel Find the MAD between the located macroblocks. Using this, the comparison macroblock is updated / replaced with a new macroblock located at the point where the smallest MAD value is shown, and the process is resumed by reducing the step size to half. Then, the step size is halved and the above process is repeated. As a result, the interval between the position where the motion of each macroblock is detected and the reference point can be expressed as a motion vector . Here, the step size and the number of MAD searches for motion prediction can be adjusted according to how much degree of prediction accuracy is desired.

(3) Frame motion compensation Frame Motion Compensation , FMC )

This technique is a movie improvement method that can be used additionally when applying the block matching algorithm in photoacoustic image. This method can use all of the various derivation techniques of the block matching algorithm (for example, TSS), and can calculate the motion vector and use it to calculate the motion error which helps to reduce motion artifacts.

As the embodiments of the present invention adopt, in order to compensate for the motion, the interference due to the motion error is reduced by comparing the most recently acquired image with the past images. Since the comparison is made for each of the past images, the intervals of the time differences between the images to be compared are sequentially increased, and for the unidirectional movement, such a varying time difference generally causes a disadvantage in the process of finding the motion. This is because, in the unidirectional direction, as the time difference increases, the size of the motion tends to increase, so that it is difficult to apply a simple block matching algorithm having a limited search range.

(4) Inter  Frame motion compensation Inter - frame Motion Compensation , IFMC )

When applying a frame motion compensation method based on a simple block matching algorithm, there is a limitation on the size of detectable motion between frames, and the maximum range of this size is given as the sum of repeated step sizes. (For example, S + S / 2 + S / 4 for TSS, assuming the step size is S). In particular, the motion between the first frame and the last frame in the image acquired for frame averaging easily exceeds this limit because the motion is accumulated between each frame, and the greater the time difference, the greater the motion. This makes it difficult to detect motion in the entire image.

Therefore, in the technique proposed by the embodiments of the present invention (referred to as 'interframe motion compensation'), a frame equalization technique combining motion compensation between frames is proposed to overcome the limitations of conventional frame equalization.

Referring to FIG. 5, it can be seen that the structure of a signal processing block that performs block matching using a macroblock by performing a multi-step search and performs frame processing to enable motion compensation between frames is shown. That is, the motion compensation is performed between the respective image frames, and the layered results are accumulated to output a normalized image. For example, if Img # N is the most recently obtained image having images obtained from Img # 1 to Img # N at a time, all # 1 to # N-1 image data are estimated to be motion relative to #N data There is a need to compensate. To achieve this, Img # 1, first obtained, compensates sequentially until the most recent data and motion are compensated through the compensated result of moving with the immediately preceding data. This process is performed for all the images Img # 2 to Img # N-1. Consequently, all the images converge to the morphological reference of Img # N and the motion can be compensated.

FIG. 6 is a block diagram illustrating a photoacoustic imaging system for correcting motion according to an embodiment of the present invention. Since the system structure of FIG. 2 and FIG. 4 is covered before, Only the outline of the operation performed by each configuration will be described.

The photoacoustic image acquisition means 10 includes a light applying unit 11 for applying a photon to the measurement subject 50 and a probe for receiving an ultrasonic signal generated by the photon absorbed by the measurement target 50 ) ≪ / RTI >

The photoacoustic image processing unit 20 includes a storage unit 21 having at least one physical storage medium and a processing unit 22 having a processor for performing substantial image processing.

The storage unit 21 stores a plurality of frame images obtained for the measured object from the ultrasound signals received over time by repeating the photon application and the ultrasound signal receiving process. In addition, temporary frame images generated in the image processing process may be stored.

The processing unit 22 reads out the image frame data stored in the storage unit 21 to compensate each motion generated in the previous frame with respect to the most recent frame of the frame image, Accumulates the images to generate one photoacoustic image, and outputs the photoacoustic image to the display means (40).

The processing unit 22 estimates video motion vectors between adjacent frames at the predetermined time intervals from the most recent frame for a plurality of frame images acquired at a predetermined time interval from the most recent frame, Motion motion vectors are compensated for by concatenating video motion vectors sequentially.

In particular, the processing unit 22 sets a reference macro block as a reference for one frame in the obtained frame image, sets a reference macroblock at the same position of the previous frame to be compared with the macro block, The difference between the reference macroblock and the reference macroblock is calculated to detect a change from the previous frame based on the most recent frame, and an interval between the detected position of the motion and the reference point is calculated from the difference between the calculated macroblocks It is possible to estimate an image motion vector between adjacent frames.

In addition, the processing unit 22 estimates the video motion vector through an iterative search process that sequentially reduces the step size, and estimates the video motion vector based on the initial step size , The difference between the calculated macroblocks is updated, and the calculated difference value is updated to a new macroblock, the step size is reduced, and the difference value calculation and the macroblock updating are repeated a predetermined number of times .

Meanwhile, in implementing a molecular photoacoustic image using multiple wavelengths, the light applying unit 11 applies a photon to each of the measurement targets 50 at predetermined wavelengths, and the processing unit 22 calculates It is preferable to generate a photoacoustic image by accumulating the motion-compensated plurality of frame images based on an image having a wavelength at which the feature points are most clearly displayed in the image.

In addition, the light applying unit 11 applies a photon while sequentially changing a preset wavelength, acquires a frame image, applies a photon while sequentially changing a predetermined wavelength again after a predetermined time delay, and obtains a frame image It is preferable to keep the time interval between acquisition of frame data per wavelength constant.

In order to realize a molecular photoacoustic image using such a multi-wavelength, the processing unit 22 is configured to perform, at a predetermined time interval from the most recent frame, a plurality of frame images acquired at a predetermined time interval from the latest frame, Estimates video motion vectors between adjacent frames, and sequentially connects the estimated video motion vectors to compensate for motion generated in the video.

FIGS. 7A to 7D are graphs illustrating experimental results showing improved signal-to-noise ratios (SNRs) using image equalization according to an exemplary embodiment of the present invention.

(FIG. 7C) by performing the motion compensation proposed by the embodiments of the present invention in comparison with the frame averaging technique (FIG. 7B) in which the conventional motion compensation is not performed for the artificial motion indicated by the white arrow in FIG. 7A, The improvement of the signal-to-noise ratio can be seen. Finally, Figure 7d shows a graph comparing the improved resolution. Referring to FIG. 7D, it is possible to confirm a superior resolution in the case of applying the motion compensation scheme (FA-IFMC method) proposed by the embodiments of the present invention to the reference image.

As shown in FIGS. 7A to 7D, it has been confirmed through experiments that the image quality can be improved by using a motion compensation algorithm for image equalization. TSS algorithm and block matching algorithm are applied to estimate the motion of the image data over time according to T interval. That is, the image motion vector between I (0) ↔I (-T), I (-T) ↔I (-2T), ..., I (-T The optimal images are obtained by compensating motion by connecting each estimated vector to each image.

In this experiment, a graphite core with a diameter of 0.9 mm was inserted into a chicken breast, and a laser pulse of 10 Hz having a wavelength of 700 nm was applied using the Nd: Yag laser system. At this time, the signals were received by a probe of the ultrasound imaging system, and they were received after moving 8 times to the right and the lower side in 0.2 mm increments in order to apply an artificial motion. The thus-obtained rf-data is averaged by 4 frames for each position, and the image is imaged through envelope detection and scan-conversion. Then, the conventional method and the method proposed by the embodiments of the present invention Frame leveling.

In the case of the reference image (FIG. 7B), the result is the image obtained by averaging the rf-data of 64 frames at the same position. As shown in the image, it can be seen that the noise signal shown in the conventional method (FIG. 7B) disappears through the proposed method (FIG. 7C), and the target signal is also located at the same position as the reference image, You can also see that it appears in the correct position without being scattered. The measured PSNR improvement was 6.3 dB and the axial resolution was improved by 0.83 mm. The results of these experiments are summarized in Table 1 below.

Conventional method Proposed method PSNR 23.4 dB 29.7 dB Axial resolution 1.49 mm 0.66 mm

FIGS. 8A to 8C illustrate experimental results illustrating improved signal-to-noise ratio (SNR) using image equalization according to an embodiment of the present invention, and show improved molecular absorption contrast.

Experiments were conducted to confirm that the proposed method adopted by the embodiments of the present invention for breast specimens including calcified tissues improves the contrast of calcified tissue compared to the conventional method.

These specimens are contained in water and tissues are less dense than water, so their shape is changed sensitively to the flow in the water. We also reconstructed the image with arbitrary movement of 0.5 mm for each wavelength image data.

The image of FIG. 8A shows an x-ray mammography image of the specimen, showing the location of calcification tissue inside the specimen. Also, the image of FIG. 8B shows a photoacoustic image obtained with a laser of 700 nm and 800 nm, respectively. Thus, the molecular photoacoustic image of FIG. 8C is obtained.

8C is an image obtained by matching a photoacoustic image without motion compensation to an ultrasound image, and a bottom image is a molecular photoacoustic image restored by the proposed method according to the embodiments of the present invention. As can be seen at the glance, if motion compensation is not performed, the signal can be detected at a portion other than the target calcified tissue, which can be a factor for lowering the sensitivity and accuracy in diagnosis. On the other hand, the proposed method can improve the sensitivity and accuracy of diagnosis by optimizing only the contrast for the desired calcified tissue.

According to embodiments of the present invention, it is possible to obtain improved signal-to-noise ratio, axial resolution, and contrast in the image equalization process or the molecular photoacoustic image restoration process by compensating for possible motion errors in the photoacoustic image acquisition process . In particular, applying the motion compensation technique to both the photoacoustic image reconstruction and the image equalization improves the image quality dramatically.

By applying the motion compensation technique, it is possible to improve the image quality of each wavelength by compensating the motion that can be generated in the process of frame equalization of the first photoacoustic image, and the second photoacoustic spectroscopy image The motion caused by the motion artifacts can be minimized by compensating the motion generated between the images of each wavelength in the acquisition. Thus, it is possible to efficiently acquire the image of the desired measurement object of the desired region.

Meanwhile, embodiments of the present invention can be embodied as computer-readable codes on a computer-readable recording medium for controlling the operation of the probe structure and performing image processing on individual images obtained through the probe structure. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device and the like, and also a carrier wave (for example, transmission via the Internet) . In addition, the computer-readable recording medium may be distributed over network-connected computer systems so that computer readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present invention can be easily deduced by programmers skilled in the art to which the present invention belongs.

The present invention has been described above with reference to various embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

10: photoacoustic image acquisition means
11: light applying part 12: probe
20: photoacoustic image processing means
21, 25, 31, 35: storage unit / memory
22:
23, 33: motion compensation means
27: Image accumulation means 37: Multispectral image reconstruction means
40: Display means
50: Target to be measured

Claims (15)

A photoacoustic imaging system applying a photon to an object to be measured and receiving an ultrasonic signal generated by the photon absorbed by the object to be measured;
Acquiring a plurality of frame images of the object to be measured from the ultrasound signals received over time by repeating the photon-applying and ultrasonic signal receiving processes by the photoacoustic imaging system;
Compensating for motions generated in the image of the previous frame based on the latest frame among the frame images obtained by the photoacoustic imaging system; And
Wherein the photoacoustic imaging system accumulates the motion compensated plurality of frame images to generate a photoacoustic image. The method according to claim 1,
The step of compensating for the motion comprises:
Estimating image motion vectors between adjacent frames at the predetermined time intervals from the most recent frame for a plurality of frame images acquired at a predetermined time interval from the most recent frame; And
And compensating motion generated in an image by sequentially connecting the estimated video motion vectors. 3. The method of claim 2,
Wherein the estimating the image motion vector comprises:
Setting a reference macro block as a reference for one frame in the obtained frame image and setting a reference macroblock at the same position of the previous frame to be compared;
Calculating a difference between a reference macroblock and a reference macroblock to detect a change from a previous frame based on the most recent frame; And
And estimating an image motion vector between adjacent frames using a distance between the detected position of the motion and the reference point from the difference between the calculated macroblocks. 3. The method of claim 2,
Wherein the step of estimating the image motion vector is performed through an iterative search process of sequentially reducing a step size,
The difference between the first reference macroblock and the macroblocks spaced apart from the corresponding reference macroblock by an initial step size is calculated, the point having the smallest calculated difference value is updated with a new macroblock, and the step size is reduced And the difference value calculation and the macro block update are repeated a predetermined number of times. The photoacoustic imaging system applying a photon to a measured object by a predetermined wavelength and receiving an ultrasonic signal generated by the photon absorbed by the measured object;
Acquiring a plurality of frame images of the object to be measured from the ultrasound signals received over time by repeating the photon-applying and ultrasonic signal receiving processes by the photoacoustic imaging system;
Compensating for motions generated in the image of the previous frame based on the latest frame among the frame images obtained by the photoacoustic imaging system; And
And generating a photoacoustic image by accumulating the motion compensated plurality of frame images based on an image of a wavelength at which the feature points are most clearly displayed among the images obtained by the photoacoustic imaging system by the wavelengths A method of acquiring a photoacoustic image. 6. The method of claim 5,
The method of claim 1,
By repeating a process of acquiring a frame image by applying a photon while sequentially changing a predetermined wavelength and acquiring a frame image by applying photons while sequentially changing a predetermined wavelength again after a predetermined time delay, And the time interval between acquisitions is kept constant. 6. The method of claim 5,
The step of compensating for the motion comprises:
Estimating image motion vectors between adjacent frames at the predetermined time intervals from the most recent frame for a plurality of frame images acquired at a predetermined time interval from the most recent frame; And
And compensating motion generated in an image by sequentially connecting the estimated video motion vectors. 6. The method of claim 5,
The predetermined wavelength may be, for example,
And a plurality of laser wavelengths that are responsive to a tissue constituting the object to be measured or a lesion tissue located in the object to be measured. A light applying unit for applying a photon to an object to be measured;
A probe for receiving an ultrasonic signal generated by photons absorbed by the object to be measured;
A storage unit for storing a plurality of frame images obtained for the measured object from the ultrasound signals received over time by repeating the photon application and the ultrasonic signal reception process; And
And a processor for compensating for motions generated in the image in the previous frame based on the most recent frame of the frame images and accumulating the plurality of frame images compensated for motion to generate one photoacoustic image, Image system. 10. The method of claim 9,
Wherein,
Estimating an image motion vector between adjacent frames at the predetermined time interval from the most recent frame for a plurality of frame images acquired at a predetermined time interval from the most recent frame,
And compensates motion generated in the image by sequentially connecting the estimated video motion vectors. 11. The method of claim 10,
Wherein,
Setting a reference macro block as a reference for one frame in the obtained frame image, setting a reference macroblock at the same position of a previous frame to be compared with the macro block,
A difference between a reference macroblock and a reference macroblock is calculated to detect a change from a previous frame based on the latest frame,
And estimates an image motion vector between adjacent frames using the interval between the detected position of the motion and the reference point from the difference between the calculated macroblocks. 11. The method of claim 10,
Wherein the processor estimates the image motion vector through an iterative search process of sequentially reducing a step size,
The difference between the first reference macroblock and the macroblocks spaced apart from the corresponding reference macroblock by an initial step size is calculated, the point having the smallest calculated difference value is updated with a new macroblock, and the step size is reduced And the difference value calculation and the macro block update are repeated a predetermined number of times. 10. The method of claim 9,
Wherein the light applying unit applies a photon to a measurement target by a predetermined wavelength,
Wherein the processor generates one photoacoustic image by accumulating the plurality of frame images compensated for motion on the basis of an image of a wavelength at which the feature points are most clearly displayed among the images obtained by the wavelengths, system. 14. The method of claim 13,
The light-
By repeating a process of acquiring a frame image by applying a photon while sequentially changing a predetermined wavelength and acquiring a frame image by applying photons while sequentially changing a predetermined wavelength again after a predetermined time delay, Wherein the time interval between acquisitions is kept constant. 14. The method of claim 13,
Wherein,
Estimating an image motion vector between adjacent frames at the predetermined time interval from the most recent frame for a plurality of frame images acquired at a predetermined time interval from the most recent frame,
And compensates motion generated in the image by sequentially connecting the estimated video motion vectors.

Images