JP4915726B2 - Method and apparatus for measuring blood flow velocity - Google Patents

Description

  The present invention relates to a method and apparatus for measuring blood flow velocity by image analysis.

  Conventional means for measuring the blood flow velocity include those using pulse waves (for example, Japanese Patent Application Laid-Open No. 2005-279056) and those using ultrasonic waves (for example, Japanese Patent Application Laid-Open No. 2005-130969). Japanese Patent Laid-Open No. 2003-180641 has been proposed for measuring by image processing.

In the invention described in this publication, pixel output fluctuations are measured based on at least three images among images stored in time series, and arithmetic processing is performed based on the output fluctuations. Is used to calculate the blood flow velocity. The invention described in this publication as the prior art measures the output fluctuations of the pixels of each image in a large number of stored images, and calculates the blood flow velocity by performing arithmetic processing based on the output fluctuations. It is.

  As described above, there is a technique for calculating the blood flow velocity by analyzing the image, but the blood flow velocity is directly measured by measuring the actual blood flow by paying attention to the change in the brightness of the blood flow. There is no technology to use.

  An object of the present invention is to make it possible to directly measure the blood flow velocity.

The first to fourth aspects of the present invention relate to a blood flow velocity measuring method, and the first aspect of the invention measures a blood flow velocity by the following procedure.
Recording step to record blood flow irradiated by microscope with video recording means such as video recorder , calculate average value of RGB color at the same position of all recorded images and obtain noise average image to remove noise A noise removing step for obtaining an image that reveals a blood vessel image, a measuring blood vessel extracting step for extracting a specific blood vessel from the edited image, a measuring step for measuring and recording the distribution of brightness in the extracted blood vessel over time, A calculation step for calculating a blood flow velocity based on a change in lightness distribution.

In the above, it is desirable to record blood flow about 250-300 frames, and the time is about 8-10 seconds. The observation site is preferably the base of the fingernail. This is because the portion of the nail epithelium, which is the boundary between the fingertip and the nail, has thin skin, and when this portion is enlarged using a microscope, the capillary blood vessels in the portion can easily be seen through. Then, it is preferable that the observed portion is enlarged and recorded by about 200 times or more, and the subsequent processing is performed. For this purpose, it is appropriate to use a camera and lens having an optical zoom of 200 times.
Further, since the position of the finger may move during recording, it is preferable to add a position information correction step for correcting a positional deviation of each image from the moving image recording means after the recording step. . In addition, if the range of 50 pixels or more of the subject is photographed using a camera capable of correcting the position, the position information correction work is unnecessary.
As specific means of the position information correction step, a specific part of an image is specified from one recorded image, and an image in a range in the vicinity of the specified image is specified for each image of all images and the specified A method of calculating a correlation function with an image and correcting position information of each image based on a correlation value obtained by the calculation is suitable, but other known methods can also be adopted.
When a specific part is specified from the one image, it is difficult to recognize a blood vessel part in a raw image. Therefore, if color separation is performed on RGB and a color separation image on which blood vessels are easy to recognize is used, the work is easy. It is. Empirically, blood vessels are easily recognized in red or green decomposition images. Note that the specific image is not limited to the blood vessel as long as the correlation function of the image position can be calculated.

The recorded raw image has a relatively small brightness difference between the blood vessel portion and the other portions, and it is difficult to clearly recognize the blood vessel portion. Therefore, noise is removed to increase the brightness difference between the blood vessel portion and the other portions so that the blood vessel portion can be clearly recognized.
As a means for removing noise, a method of applying a filter for increasing the brightness difference to each image may be considered. However, if an average value of each color of RGB at the same position of each image is calculated, an effective result can be obtained. .
When the RGB colors in a plurality of images recorded over time are averaged, it is possible to create a low noise image in which the noise in each image is canceled and the blood vessel portion is sharpened.
Therefore, by adopting the above-described method, each image has a large brightness difference between the blood vessel portion and the other portions, and the image is a blood vessel clearly represented. Hereinafter, an image from which noise has been removed is referred to as an average image.

  A blood vessel suitable for measurement is selected based on the average image obtained above. The blood vessel conditions suitable for the measurement are that a certain linear distance is provided and that the brightness difference in the blood flow is clear. A clear lightness difference in the bloodstream means that blood without blood cells in blood, ie plasma, can be clearly distinguished. Since the plasma appears white, the brightness difference from the black blood cells becomes clear.

Next, measurement is performed over time based on the image signal of the variation in the lightness distribution of the blood vessels to be measured specified in the previous step in all images. Specifically, the movement of the plasma is measured and recorded. Next, the blood flow velocity is calculated from the length of the blood vessel to be measured and the movement speed of the brightness distribution. In this process, when the extracted blood vessel image is displayed on the screen as a moving image, the blood flow can be measured while being visually recognized, but it is not necessarily displayed on the screen.
The measurement of the movement of the brightness distribution, high precision is performed in the center of the vessel (claim 4).

The invention of claim 5 relates to an apparatus for carrying out the above method, and includes moving picture recording means such as a video recorder for recording blood flow irradiated by a microscope, and RGB colors at the same position of all recorded images . A noise removing unit that obtains an average image that reveals a blood vessel image by removing noise by calculating an average value and obtaining an average image, a blood vessel extracting unit that extracts a blood vessel to be measured from the average image, and an extracted blood vessel Measuring means for measuring and recording the lightness distribution over time, and calculating means for calculating the blood flow velocity based on the change in the lightness distribution over time.
In the above, a specific partial image is designated from one recorded image, and a correlation function between the designated image and an image in the vicinity of the designated image is calculated for each image of all images, It is preferable to add position information correction means for correcting the calculated correlation value.

  The position information correcting means is configured as a program for executing the following procedure, for example.

(Extract region of interest)
This is a step of specifying a range (hereinafter referred to as “region of interest”) as a basis for calculating a correlation function in an arbitrary image (for example, the first image, hereinafter referred to as “reference image”). When extracting a region of interest, a designated image is first selected, and then a certain range including the selected image is determined as a region of interest.
Since the designated image is an image that requires correction of position information, an image having a remarkable brightness difference from surrounding images is selected. Then, an arbitrary region is divided around the designated image, and this is determined as a region of interest.

(Calculation of correlation function)
The correlation function is calculated based on the entire region of interest. Specifically, for example, the following calculation is performed between the region of interest in the reference image and all other images (hereinafter referred to as “comparison images”) as follows.
First, measure the pixel value (brightness) of each pixel of the region of interest in the reference image and the pixel value of the region of interest in the reference image and the image of the same size in the comparison image, and calculate the correlation function (the difference between the two) To do. The position of the comparison image is a position corresponding to the maximum correlation function calculated from the previous frame. Next, the pixel value of the comparison image at a position shifted by one pixel in the X-axis direction from the region of interest in the reference image is measured, and a correlation function with the reference image is calculated. (For example, a range of about +15 pixels and −15 pixels in the X-axis direction, +15 pixels and −15 pixels in the Y-axis direction) The correlation function at each position is calculated while moving the range one pixel at a time.

The correlation functions calculated for each movement of the comparison image are compared, and the position where the correlation function is the maximum is detected. That the correlation function is maximum means that the image of the region has a feature most similar to the image of the region of interest in the reference image. This means that the degree of overlapping of the designated images in the comparative image is high.
If the position of the specific image in the region of interest in the reference image and the position of the specific image in the comparison image are exactly the same (when both images have different regions of interest but have the exact same content) The correlation function is 1, and approaches 1 when the contents of the region of interest are almost the same.
Thereafter, the same operation is performed on all reference images.

(registration)
Registration is to search for a feature in one image (reference image) in another image (comparison image) and associate the feature with the position in the two images. is there.
In the correlation function calculating step, the region of interest of the reference image and the image position having the maximum correlation function are selected for each comparison image. Therefore, the position information of each image is corrected based on the image position having the maximum correlation function. For example, if the correlation function of the image at the position moved by +4 in the X-axis direction and −5 in the Y-axis direction in the second image is the maximum, the position information of the second image is − in the X-axis direction. 4. Correct +5 in the Y-axis direction.
By performing this operation for each image, the positions of the specific images in the reference image and all the comparative images are substantially the same.
After the above operations, the processed data is saved, and the following processing is performed using the data whose position information is corrected.

The noise removal is an operation for clarifying an image of a blood vessel, that is, an operation for increasing a brightness difference between a blood vessel and a portion other than the blood vessel, and the means is a program for executing the following procedure.
Addresses are assigned to all images in units of pixels, and the pixel values at each address are added together for each RGB and divided by the number of images to obtain the average value of RGB at that address. For all addresses, the pixel values of all images are aligned with the average value to obtain an average image.

The measurement blood vessel selecting means extracts blood vessels in which blood flow is easy to observe from an image, and is preferably performed using the average image.
Increase the contrast to enhance the blood vessel extraction effect. Then, the outer edge (Edge) of the blood vessel is extracted. An extreme point is found from the result of extraction of the outer edge, continuous points are made into lines, independent points are deleted, lines are smoothed, and blood vessels are extracted.

The measuring means for measuring and recording the distribution of lightness of blood vessels over time is constituted by a program that executes the following procedure.
The brightness in the continuous direction of the blood vessel is recorded in the extracted blood vessel image. Next, the brightness in the continuous direction of the blood vessel image corresponding to the extracted blood vessel image in the continuous image is recorded.
Take the pixel brightness value of each blood vessel in the image and calculate the relationship between time and position. The time is determined for each image with the time of the first image frame being zero seconds, and the position is a distance between the address numbers.
Based on this time and position, the rate of change of the lightness distribution in the bloodstream is calculated to obtain the bloodstream velocity.

The present invention records blood flow, removes noise in all recorded images, obtains an average image that reveals blood vessel images, extracts blood vessels that can be recognized from the average image, and calculates the brightness of the extracted blood vessels. Since the distribution is measured and recorded over time, and the blood flow velocity is calculated based on the change in lightness distribution over time, the blood flow can be measured directly, and it is simple, quick and accurate. Blood flow velocity can be determined.
In the method of the present invention, it is possible to visually recognize the blood flow displayed in the image. However, when each processing step is programmed, full automation is possible, and when fully automated, It can also be configured as a device that does not display a stream image.

(Recording)
An optical image from a microscope with a magnification of about 200 times is irradiated on the CCD sensor, and the image is recorded at 30 frames per second. The recording time is about 10 seconds, and the number of frames is about 280 to 300 frames.

(Position information correction)
(1) Extraction of outer edge of blood vessel In this step, the outer edge of the blood vessel is extracted from any one image (for example, the first image, hereinafter referred to as “reference image”). The outer edge of the blood vessel can be obtained by connecting pixels of high brightness in the image with lines. The U-shaped outer edge can be expected to give accurate results in later correlation function measurements.

(2) Determination of region of interest The outer edge of the blood vessel suitable for the measurement of the correlation function is U-shaped and long, and is a certain distance from the outer frame of the image (for example, X = 30 pixels, Y = 30). Pixels) This is because the comparison image may protrude from the outer frame of the screen if it is not so far away.
Since the outer edge that satisfies the above conditions generally has a large number of pixels, a suitable outer edge can be automatically selected by arranging the outer edges of a plurality of blood vessels obtained from one image in the order of the number of pixels.
The outer edge selected here is determined as the specific image 1, and a frame of about 30 pixels is set in the X and Y directions with the specific image as the center, and this is set as the region of interest 2 (FIG. 1). In the comparative image at the position corresponding to the region of interest, the position of the specific image is slightly moved for each frame due to the fine movement of the finger during recording (FIG. 2).

(Calculation of correlation function)
The correlation function is calculated based on the entire region of interest. Specifically, for example, the following calculation is performed between the region of interest in the reference image and all other images (comparison images) as follows.
First, measure the pixel brightness value of each pixel of the region of interest in the reference image and the pixel brightness value of the comparison image at a position and range that matches the region of interest in the reference image, and calculate the correlation function (the magnitude of the difference between the two) To do. Next, the pixel brightness value of the comparison image at a position shifted by one pixel in the X-axis direction from the region of interest in the reference image is measured, and a correlation function with the pixel brightness value of the region of interest in the reference image is calculated. The correlation function of the pixel brightness value at each position is calculated while moving the range by 1 pixel over a range of +15 pixels, −15 pixels, +15 pixels in the Y-axis direction, and −15 pixels (FIG. 3).

The correlation functions calculated for each movement of the comparison image are compared, and the position where the correlation function is the maximum is detected.
Thereafter, the same operation is performed on all reference images.
FIG. 4 shows the deviation of the region of interest in the image having the maximum correlation function in each image and the reference image for each X axis and Y axis.

(registration)
In the correlation function calculating step, the region of interest of the reference image and the image position having the maximum correlation function are selected for each comparison image. Therefore, based on the image position with the maximum correlation function, the position information of each image is corrected so that the image position with the maximum correlation function matches the position of the region of interest of the reference image.
By performing this operation on each image, the positions of the specific images in the reference image and all the comparative images are substantially matched.
After the above operations, the processed data is saved, and the following processing is performed using the data whose position information is corrected.

(3) Noise removal Clarify blood vessel images by noise removal.
The noise removing means is a program for executing the following procedure.
All registered images are assigned addresses in pixels, and pixel values at all addresses are measured. Next, the pixel values for each color separated into RGB of all the images at that address by summing the numerical values of each color separated into RGB at the pixels at the same address of all the images and dividing by the number of images Get the average value of.
Subsequently, the pixel values of all addresses in all the images are replaced with the average value of the pixels obtained by the above operation, thereby obtaining an average image.
As a result of this operation, the noise of each image is canceled, and as a result, an image (FIG. 6) in which the blood vessels appear clearly is obtained from the image before the processing (FIG. 5) where the blood vessels are unclear.

(Blood vessel extraction)
A blood vessel for measuring blood flow is extracted based on the average image.
The average image is read, and the outline (edge / edge) of the blood vessel is extracted based on the pixel value of G color (which may be other colors but G or R is preferable) (FIG. 7). Next, changes in pixel values are measured in four directions: horizontal direction, vertical direction, and diagonal direction (upper left to lower right, upper right to lower left) to obtain local extreme values in the respective directions. FIG. 8 is a graph showing the pixel values in the horizontal direction, and the values with circles are local extreme values.
Next, a line connecting local extreme values in all directions is obtained, and independent points are deleted (FIG. 9).
The U-shaped portion of the double line is searched, and blood vessels that continue downward from the U-shaped portion are extracted (FIG. 10).
When the blood vessel extraction process is completed, the lines that are not directly connected to the blood vessels are unnecessary and are deleted so that the screen is free from noise.

(Measurement of blood flow)
The extracted blood vessel image is scanned line by line. The middle address number in the X-axis direction of the start address number and end address number of one blood vessel row of the scan is the address number of the center point of the row of the blood vessel, and is centered over the entire length of the extracted blood vessel image. Get the address number of a point. When this center point is connected by a line, the center line of the blood vessel is obtained.
The pixel values (brightness) of all center points on the blood vessel center line are measured. When the brightness of the center line is measured, the brightness corresponding to plasma is high and a graph as shown in FIG. 11 is obtained. In FIG. 11, the vertical axis represents lightness, and the horizontal axis represents distance (pixels), indicating the 230th frame and the 231st frame.
Similarly, create a graph of the centerline of all blood vessels in this image. Similarly, if a graph of the center line of all blood vessels corresponding to the time of all images is created, the same blood vessel graph is aligned from zero seconds to the time corresponding to the last frame. FIG. 12 shows the measurement result of one blood vessel from the 230th frame to the 250th frame, but when 300 frames of images are taken, 300 such data are displayed. When recording at 30 frames per second, the time difference between frames is 1/30 seconds.

Hereinafter, a blood flow velocity calculation method will be described based on the results of the two frames shown in FIG.
First, the signal correlation function is calculated every time one pixel is moved while moving the images of two consecutive frames, and the difference between the positions of the two frames at the position where the correlation function is maximized is obtained. This difference is the moving distance of blood flow between frames. Specifically, the following procedure is performed.

The pixel values (brightness) of the address numbers of the same center line of all the images are added together and divided by the number of images to obtain the average value of the pixels of the address numbers. Then, the average value is subtracted from the graph values of all the same address numbers, and a correlation function with the graph values of the same center line at different times is calculated. Based on the result, the moving distance of the lightness peak is calculated.
By taking the average value, the change in brightness can be seen clearly.

After processing by the average value, the signal of the 231st frame in FIG. 11 is moved 30 pixels to the left and moved from this position to the position of 61 pixels to the right (with reference to the initial position, + −each 30 Each time one pixel is moved, a correlation value with the lightness of the corresponding address at the 230th frame is calculated for each address. The correlation value (C) is calculated by multiplying the lightness (xk) at the 230th frame by the signal value (yk) at the 231st frame for each address, adding up the calculation results at each address, Calculation is performed by comparing the sum of the squared values of the signal values at the addresses of the 231st frame.
The specific calculation formula is as follows.

n: Number of pixels in the image x: Lightness of a specific address in the reference image
y ・ ・ ・ Lightness of a specific address in the comparison image
a ... Average value of the brightness of all the reference addresses in the reference image
b: Average value of brightness of all addresses in the comparison image As a result, a graph as shown in FIG. 13 is obtained.
In this graph, the point having the highest correlation value is the position of -8 pixels (the position obtained by moving the signal of the 231st frame to the left by 8 pixels).
From this, it can be seen that the position of the plasma moved 8 pixels between the 230th frame and the 231st frame. Since the time difference between the two frames is 1/30 seconds, the plasma moving speed during this period is 8 (pixels) / 1/30 (seconds) = 240 pixels / second.
Since 1 mm = 82 pixels,
The blood flow velocity is 240 pixels / second × (1/82) millimeter / pixel.

In short, blood flow velocity is obtained by tracking the movement of specific plasma in blood vessels. Therefore, theoretically, it is possible to obtain the blood flow velocity based on two images taken with a predetermined time interval.
However, plasma is not necessarily present in all images (frames). In some cases, the position of plasma is shifted from the center line of the blood vessel to be measured. In such a case, the calculation of the correlation function of the pixel value is impossible or inaccurate, and the blood flow velocity cannot be obtained accurately.
Therefore, an accurate blood flow velocity is obtained by preparing a large number of images as described above and obtaining a correlation function of pixel values of adjacent frames.

The present invention provides a recording step for recording a blood flow irradiated by a microscope as a moving image, a noise removal step for removing an image of all recorded images to obtain an average image in which a blood vessel image is actualized, and a blood vessel that can be recognized from the average image A blood vessel extraction step for extracting the blood flow, a measurement step for measuring and recording the lightness distribution in the extracted blood vessel over time, and a calculation step for calculating the blood flow velocity based on the change in the lightness distribution over time. Thus, blood vessels are extracted as an image and the blood flow velocity is measured based on the change in the brightness of the blood flow, and the blood flow velocity can be measured easily and accurately.
In addition, by measuring the image processing, it is possible to automate the measurement of blood flow velocity, which has industrial applicability.

Diagram showing specific image and region of interest The figure which shows the shift | offset | difference of the position of the specific image in five continuous frames Diagram showing correlation function calculation method Graph showing the shift of the region of interest between frames Image obtained by RGB color separation of one frame image Image obtained by RGB color separation of the average image of all frames Extracted image of blood vessel edge Graph showing changes in pixel values Image with local extrema connected by lines and independent points removed An image of blood vessels extracted from the U-shaped part Graph showing the pixel value at the center of two consecutive blood vessels Graph showing the pixel value at the center of 20 consecutive blood vessels Graph showing blood flow velocity calculation method

Explanation of symbols

1 specific image 2 region of interest

Claims (6)

Recording step of recording blood flow irradiated with a microscope by a moving image recording means,
A noise removal step of obtaining an average image that reveals a blood vessel image by removing the noise by calculating an average value of RGB colors at the same position of all recorded images to obtain an average image;
A blood vessel extraction step for extracting blood vessels that can be recognized from the average image;
A measurement step for measuring and recording the distribution of lightness in the extracted blood vessel over time;
A calculation step of calculating a blood flow velocity based on a change in lightness distribution over time;
A method for measuring blood flow velocity. 2. The blood according to claim 1, further comprising a position information correction step for correcting a positional shift because there is a positional shift in the blood vessel in the recorded image due to a change in relative position between the finger and the moving image recording means. Measurement method of flow velocity. The position information correction step designates a specific partial image from one recorded image, and a correlation function between an image in a range in the vicinity of the designated image and the designated image for each image of all images And the position information of each image is corrected based on the calculated correlation value. The blood flow velocity measuring method according to claim 2. The method for measuring blood flow velocity according to claim 1, wherein the measurement of the distribution of lightness in the blood vessel is performed at the center line of the extracted blood vessel. Moving picture recording means for recording blood flow irradiated with a microscope, average of RGB values at the same position of all recorded images and obtaining an average image to remove noise and reveal the blood vessel image Noise removing means for obtaining an image, blood vessel extracting means for extracting a recognizable blood vessel from an average image, measurement means for measuring and recording a lightness distribution in the extracted blood vessel over time, based on the change in lightness distribution over time A blood flow velocity measuring device comprising a calculation means for calculating a blood flow velocity. A specific partial image is designated from one recorded image, and a correlation function between the designated image and the image in the vicinity of the designated image is calculated for every image, and the calculated correlation is calculated. 6. The blood flow velocity measuring device according to claim 5 , further comprising position information correcting means for correcting the position information of each image based on the value.