JP2021058478A - Image processing apparatus, image processing system, and image processing method - Google Patents

Abstract

To provide an image processing apparatus, an image processing system and an image processing method that are capable of actualizing a novel blood flow evaluation method.SOLUTION: An image processing apparatus according to the present invention comprises a blood flow index calculation unit, an evaluation information generation unit, and an evaluation information presentation unit. The blood flow index calculation unit calculates a blood flow index, which is an index of a blood flow, from a speckle image obtained by capturing a speckle pattern formed on an object to be evaluated. The evaluation information generation unit generates blood flow evaluation information inclusive of a blood flow direction on the basis of the blood flow index. The evaluation information presentation unit presents the evaluation information.SELECTED DRAWING: Figure 1

Description

The present technology relates to an image processing device, an image processing system, and an image processing method for evaluating blood flow.

Generally, in order to confirm the strength of blood flow and the direction of blood flow, an ultrasonic probe is directly applied to a blood vessel to hear a sound, or a laser Doppler is used to confirm the direction of blood flow. Further, a blood flow confirmation method using ICG (indocyanine green), which is a fluorescent dye, and a blood flow evaluation method using MRI (Magnetic Resonance Imaging) and SPECT (Single photon emission computed tomography) are also used.

One of the operations that requires evaluation of blood flow is brain surgery bypass surgery. In bypass surgery, after suturing a blood vessel, the direction of blood flow may be different from what was expected in advance, and sometimes the direction of blood flow may be lost. In such a case, the probe is applied directly to the blood vessel in order to confirm the blood flow direction, but it is troublesome for doctors and nurses to prepare another device probe such as an ultrasonic probe for the blood vessel.

In addition, ultrasonic probes usually use sound to determine the presence or absence of blood flow, making it difficult to identify the direction of blood flow and being affected by the angle between the blood flow and the probe, making it difficult to quantify blood flow. .. In addition, the confirmation method using ICG requires a drug, and even if the presence or absence of blood flow can be confirmed, the blood flow direction often relies on visual observation of the movement of the fluorescent interface, and it can be lost if the flow is fast. There is sex.

Furthermore, although blood flow is evaluated mainly by MRI and SPECT, medical institutions equipped with these facilities are limited and expensive, and frequent examinations impose a burden on patients. As described above, a more suitable method for evaluating blood flow is required.

On the other hand, as a method for evaluating motion, a speckle observation technique for detecting motion in a measurement target by irradiating the measurement target with a laser and imaging a speckle pattern generated by the laser is known. In the speckle observation technique, it is possible to evaluate the movement such as the direction of the movement by evaluating the correlation of the speckle pattern before and after the movement (for example, Patent Documents 1 and 2).

Japanese Unexamined Patent Publication No. 2015-152490 Japanese Unexamined Patent Publication No. 2015-031623

However, in the speckle observation technique as described in Patent Documents 1 and 2, when the moving object is a liquid, the correlation of the speckle pattern is lost even if the movement is slight, and the movement is evaluated. I couldn't.

In addition, evaluation of cerebral blood flow is important in neurosurgery, and tissue hyperperfusion and ischemic conditions are directly linked to cerebral dysfunction, so prompt evaluation and coping are required. However, there is no medical device capable of sufficiently evaluating cerebral blood flow during surgery, and evaluation methods using CBF (Cerebral blood flow) before and after surgery have been reported.

In view of the above circumstances, an object of the present technology is to provide an image processing apparatus, an image processing system, and an image processing method capable of realizing a new intraoperative blood flow evaluation method.

In order to achieve the above object, the image processing apparatus according to the present technology includes a blood flow index calculation unit, an evaluation information generation unit, and an evaluation information presentation unit.
The blood flow index calculation unit calculates a blood flow index, which is an index of blood flow, from a speckle image obtained by capturing a speckle pattern formed on an evaluation object.
The evaluation information generation unit generates evaluation information of blood flow including the blood flow direction based on the blood flow index.
The evaluation information presentation unit presents the evaluation information.

According to this configuration, the evaluation information generated based on the blood flow index calculated from the speckle image is presented to the user, and the user can grasp the blood flow direction by referring to the evaluation information.

The evaluation information generation unit may estimate the blood flow direction from the blood flow index including the blood flow velocity distribution, which is the distribution of the blood flow velocity with respect to the position of the speckle image.

The evaluation information generation unit may estimate the blood flow direction based on the blood flow velocity at the first position of the speckle image and the blood flow velocity at the second position of the speckle image. ..

The evaluation information generation unit estimates that the direction from the position where the blood flow velocity is low to the position where the blood flow velocity is high is the blood flow direction among the first position and the second position. May be good.

The evaluation information generation unit may specify a high flow velocity region, which is a region where the blood flow velocity is equal to or higher than a certain value in the blood flow velocity distribution, and estimate the blood flow direction based on the area of the high flow velocity region. Good.

The evaluation information generation unit may specify a peeling region in which the blood flow velocity is equal to or less than a certain value in the blood flow velocity distribution, and estimate the blood flow direction based on the position of the peeling region.

The evaluation information generation unit may set the first position and the second position based on the image processing result for the normal light image captured by irradiating the evaluation target with normal light.

The evaluation information generation unit may set the first position and the second position based on the blood flow index.

In order to achieve the above object, the image processing system according to the present technology includes a lighting device, an imaging device, and an image processing device.
The lighting device irradiates the evaluation object with coherent light and normal light.
The imaging device captures a speckle pattern generated by irradiating the evaluation object with the coherent light to generate a speckle image, and images the evaluation object irradiated with the normal light. Generates a normal optical image.
The image processing device has a blood flow index calculation unit that calculates a blood flow index, which is an index of blood flow, from the speckle image, and evaluates blood flow including a blood flow direction based on the blood flow index. It includes an evaluation information generation unit to be generated and an evaluation information presentation unit that presents the above evaluation information.

In order to achieve the above object, in the image processing method according to the present technology, the blood flow index calculation unit captures a speckle pattern formed on an evaluation object from a speckle image, which is an index of blood flow. Calculate the index.
The evaluation information generation unit generates evaluation information of blood flow including the blood flow direction based on the blood flow index.
The evaluation information presentation unit presents the above evaluation information.

It is a schematic diagram of the image transmission processing system which concerns on 1st Embodiment of this technique. It is a block diagram of the image processing apparatus provided in the said image processing system. This is an example of the blood flow velocity distribution obtained from the blood flow index calculated by the blood flow index calculation unit of the image processing device. This is an example of the blood flow velocity distribution obtained from the blood flow index calculated by the blood flow index calculation unit of the image processing system. It is a schematic diagram which shows the blood flow in a branching blood vessel. It is a schematic diagram which shows the blood flow in the merging blood vessel. It is a schematic diagram which shows the blood flow in the blood vessel which has the stenosis part by a thrombus. It is a blood flow velocity distribution in the blood flow direction estimation method 1 by the above-mentioned image transmission processing system. It is a histogram of the blood flow velocity which concerns on the said blood flow direction estimation method 1. This is the blood flow velocity distribution in the blood flow direction estimation method 1. It is a histogram of the blood flow velocity which concerns on the said blood flow direction estimation method 1. It is a blood flow velocity distribution in the blood flow direction estimation method 2 by the above-mentioned image transmission processing system. It is a blood flow velocity distribution in the blood flow direction estimation method 3 by the above-mentioned image transmission processing system. This is the blood flow velocity distribution in the blood flow direction estimation method 3. It is a schematic diagram which shows the analysis range set by the evaluation information generation part of the image processing apparatus. It is a flowchart which shows the operation of the said image processing apparatus. It is a flowchart which shows the blood flow evaluation method by the said image processing system. It is a schematic diagram which shows the blood flow direction in the anastomosed blood vessel. It is a block diagram which shows the hardware structure of the said image processing apparatus. It is a schematic diagram of the image transmission processing system which concerns on the 2nd Embodiment of this technique. It is a schematic diagram which shows the 1st time and 2nd time related to the said image transmission processing system. It is a block diagram of the image processing apparatus provided in the said image processing system. It is a block diagram of the motion estimation part included in the image processing apparatus. It is a schematic diagram of the motion estimation by the said motion estimation unit. It is a schematic diagram which shows the evaluation information generation method by the evaluation information generation part provided in the image processing apparatus. This is an example of a difference image generated by the evaluation information generation unit included in the image processing device. It is a flowchart which shows the operation of the said image processing apparatus. It is a flowchart which shows the blood flow evaluation method by the said image processing system. It is a schematic diagram which shows the evaluation index example 1 by the evaluation information generation part. It is a schematic diagram which shows the evaluation index example 1 by the evaluation information generation part. It is a schematic diagram which shows the evaluation index example 1 by the evaluation information generation part. It is a schematic diagram which shows the evaluation index example 1 by the evaluation information generation part. It is a schematic diagram which shows the evaluation index example 1 by the evaluation information generation part.

(First Embodiment)
The image processing system according to the first embodiment of the present technology will be described.

[Image processing system configuration]
FIG. 1 is a schematic diagram showing a configuration of an image processing system 100 according to the present embodiment. As shown in the figure, the image processing system 100 includes a lighting device 101, an image pickup device 102, an image processing device 103, and a display device 104. Further, in FIG. 1, the object to be evaluated by the image processing system 100 is shown as the evaluation object S. The evaluation object S is a biological tissue such as the brain or other organs, and is not particularly limited.

The lighting device 101 irradiates the evaluation object S with coherent light and white light for speckle observation. The coherent light can be, for example, laser light in the near infrared wavelength band. The illuminating device 101 may irradiate the evaluation object S with coherent light and normal light at the same time, or may irradiate the evaluation object S sequentially.

The image pickup apparatus 102 images the evaluation object S. The image pickup device 102 images a speckle pattern formed on the evaluation object S when coherent light is irradiated from the illumination device 101, and generates a speckle image. Further, the image pickup device 102 images the evaluation target object S in a state where the evaluation target object S is irradiated with normal light from the lighting device 101, and generates a normal light image.

The image pickup apparatus 102 captures a speckle image and a normal light image simultaneously or sequentially. The image pickup apparatus 102 can capture a speckle image and a normal light image in the same field of view, and it is preferable that the speckle image and the normal light image are captured via one optical system. .. The image pickup device 102 may include one image pickup element, and the image pickup element may generate a speckle image and a normal light image. The image pickup device 102 includes two image pickup elements, and each image pickup element has a speckle image and a normal light image. It may generate an image.

In the following description, the range on the evaluation object S on which the speckle image and the normal light image are captured by the image pickup device 102 (that is, the field view range of the image pickup device 102) is referred to as an “imaging range”. The image pickup apparatus 102 supplies the generated speckle image and the normal light image to the image processing apparatus 103.

The image processing device 103 acquires a speckle image and a normal light image from the image pickup device 102 as described later, and generates "evaluation information" which is an evaluation result of blood flow. The image processing device 103 generates a presentation image for presenting the generated evaluation information to the user, and outputs the presented image to the display device 104.

The display device 104 displays the presented image output from the image processing device 103. The display device 104 can be a display device having a general configuration.

[Configuration of image processing device]
FIG. 2 is a block diagram showing the configuration of the image processing device 103. As shown in the figure, the image processing device 103 includes a blood flow index calculation unit 111, an evaluation information generation unit 112, and an evaluation information presentation unit 113.

The blood flow index calculation unit 111 acquires a speckle image from the imaging device 102 and calculates a “blood flow index” which is an index of blood flow from the speckle image. The blood flow index is at least one of SC (speckle contrast), SFI (speckle flow index) and BFI (blood flow index). The blood flow index calculation unit 111 outputs the calculated blood flow index to the evaluation information generation unit 112.

The evaluation information generation unit 112 generates "evaluation information" based on the blood flow index output from the blood flow index calculation unit 111. The evaluation information includes at least the direction of blood flow (hereinafter, blood flow direction), and the details will be described later. The evaluation information generation unit 112 outputs the generated evaluation information to the evaluation information presentation unit 113.

The evaluation information presentation unit 113 acquires a speckle image, a normal light image, and evaluation information, and integrates these information to generate a presentation image. The evaluation information presentation unit 113 can generate a presentation image by superimposing the visualized evaluation information on, for example, a normal optical image. Further, the evaluation information presentation unit 113 may select either a speckle image or a normal light image as a presentation image, or both images may be arranged in parallel or superimposed to form a presentation image. The evaluation information presentation unit 113 outputs the generated presentation image to the display device 104.

[About the evaluation information generator]
The evaluation information generation unit 112 generates evaluation information from the blood flow index as described above. The evaluation information generation unit 112 can calculate the evaluation information by using the blood flow velocity distribution, which is the distribution of the blood flow velocity with respect to the position of the speckle image, which is represented by the blood flow index. 3 and 4 are examples of blood flow velocity distributions, and show blood flow velocity distributions flowing in the blood vessel 301. The blood vessel 301 is constricted by the clip 302.

As shown by the arrow in FIG. 3, blood flows from the right direction in the figure to the left direction in the figure, and in FIG. 4, blood flows from the left direction in the figure to the right direction in the figure as shown by the arrow. Flowing. As shown in these figures, narrowing the blood vessel 301 with the clip 302 creates an asymmetric flow with respect to the clip 302.

The evaluation information generation unit 112 can estimate the blood flow direction based on such a blood flow velocity distribution. Further, the evaluation information generation unit 112 can estimate the blood flow direction from the blood flow velocity distribution not only for the stenosis of the blood vessels but also for the connection relationship of the blood vessels.

5 to 7 are schematic views showing the state of blood flow. Each figure (a) shows the direction in which the blood flow flows, and each figure (b) shows the blood flow velocity. As shown in FIG. 5, when the blood vessel 301 branches, the flow is separated at the branch position as shown by the arrow E. Further, as shown in FIG. 6, when the blood vessels 301 merge, the flow is separated at the branch position as shown by the arrow E. Further, when the thrombus 303 is present in the blood vessel 301 as shown in FIG. 7, the flow is separated immediately after the thrombus 303 as shown by the arrow E.

The evaluation information generation unit 112 can detect the characteristics of blood flow as shown in FIGS. 3 to 7 based on the blood flow velocity distribution and estimate the blood flow direction. Specifically, the evaluation information generation unit 112 can estimate the blood flow direction from the blood flow velocity distribution by any of the following three methods.

<Blood flow direction estimation method 1>
FIG. 8 is a schematic diagram of the blood flow direction estimation method 1 and shows the blood flow velocity distribution. As shown in the figure, the evaluation information generation unit 112 sets the analysis windows W1 and W2 in the blood flow velocity distribution. The "analysis window" is a fixed area indicating the position of the analysis target. The evaluation information generation unit 112 can set the analysis windows W1 and W2 according to the designation by the user. Further, the evaluation information generation unit 112 may specify the blood vessel stenosis portion due to a blood vessel, a clip, a thrombus, or the like from the image processing result of the normal optical image, and set the analysis windows W1 and W2. Further, the evaluation information generation unit 112 may set the analysis windows W1 and W2 based on the blood flow index, such as a region where the blood flow index is equal to or higher than a certain value.

The evaluation information generation unit 112 compares the blood flow velocity in the analysis window W1 (hereinafter, first blood flow velocity) with the blood flow velocity in the analysis window W2 (hereinafter, second blood flow velocity). FIG. 9 is a histogram showing the first blood flow velocity V _W1 and the second blood flow velocity V _W2. As shown in the figure, when the analysis window W1 is downstream of the stenosis and the analysis window W2 is upstream of the stenosis, the blood flow velocity in the analysis window W1 downstream of the stenosis due to the asymmetric flow of blood flow. Becomes larger.

Further, FIG. 10 shows a blood flow velocity distribution when the blood flow direction is opposite to that of FIG. 8, and FIG. 11 is a histogram showing a _{first blood flow velocity V W1} and a second blood flow velocity V _{W2 in this case.} .. As shown in these figures, when the analysis window W1 is upstream of the stenosis and the analysis window W2 is downstream of the stenosis, the blood flow velocity is increased in the analysis window W2 downstream of the stenosis. Therefore, the evaluation information generation unit 112 can estimate that the direction from the analysis window having a low blood flow velocity to the analysis window having a high blood flow velocity is the blood flow direction.

The blood flow velocity of each analysis window can be calculated by any method such as the sum or average value of the blood flow velocity distribution in the analysis window. Further, the evaluation information generation unit 112 can estimate the blood flow direction by comparing the blood flow velocities between the analysis windows even when the blood vessels 301 merge or branch as shown in FIGS. 5 to 7. Is. The number of analysis windows is not limited to two, and may be three or more.

The evaluation information generation unit 112 can supply the estimated blood flow direction as evaluation information to the evaluation information presentation unit 113. The evaluation information presentation unit 113 can generate a presentation image in which the blood flow direction is indicated by an arrow or the like. Further, the evaluation information generation unit 112 may supply a graph such as a histogram as shown in FIGS. 9 and 11 to the evaluation information presentation unit 113 as evaluation information.

<Blood flow direction estimation method 2>
FIG. 12 is a schematic diagram of the blood flow direction estimation method 2 and shows the blood flow velocity distribution. As shown in the figure, the evaluation information generation unit 112 can specify the "high flow velocity region" in the blood flow velocity distribution. The high flow velocity region is a region where the blood flow velocity is above a certain level. FIG. 11 shows the high flow velocity regions H1 and H2.

The evaluation information generation unit 112 calculates and compares the areas of the high flow velocity region H1 and the high flow velocity region H2. As shown in FIG. 12, the high-velocity region H1 located downstream of the constriction has a larger area than the high-velocity region H2 located upstream of the constriction. Therefore, the evaluation information generation unit 112 can estimate that the direction from the high flow velocity region having a small area to the high flow velocity region having a large area is the blood flow direction. Further, the evaluation information generation unit 112 can estimate the blood flow direction by comparing the areas of the high flow velocity region even when the blood vessels 301 merge or branch as shown in FIGS. 5 to 7. ..

Further, the evaluation information generation unit 112 may compare not only the area of the high flow velocity region but also the total product of the area of the high flow velocity region and the blood flow velocity between the high flow velocity region H1 and the high flow velocity region H2. In this case, it can be estimated that the direction from the high flow velocity region with a small sum to the high flow velocity region with a small sum is the blood flow phrase. Further, the evaluation information generation unit 112 may estimate the blood flow direction by comparing the area and the like among three or more high flow velocity regions, and may form the shape of the high flow velocity region such as when there is one high flow velocity region. The blood flow direction may be estimated based on this.

The evaluation information generation unit 112 can supply the estimated blood flow direction as evaluation information to the evaluation information presentation unit 113. The evaluation information presentation unit 113 can generate a presentation image in which the blood flow direction is indicated by an arrow or the like. Further, as shown in FIG. 12, the evaluation information generation unit 112 may supply the evaluation information presentation unit 113 with a marker or the like indicating a high flow velocity basin as evaluation information.

<Blood flow direction estimation method 3>
FIG. 13 is a schematic diagram of the blood flow direction estimation method 3 and shows the blood flow velocity distribution. As shown in the figure, the evaluation information generation unit 112 sets the analysis windows R3 and R4 in the blood flow velocity distribution. The evaluation information generation unit 112 can set the analysis windows R3 and R4 according to the designation by the user. Further, the evaluation information generation unit 112 may specify the blood vessel stenosis portion due to a blood vessel, a clip, a thrombus, or the like from the image processing result of the normal optical image, and set the analysis windows R3 and R4. Further, the evaluation information generation unit 112 may set the analysis windows R3 and R4 based on the blood flow index, such as a region where the blood flow index is equal to or higher than a certain value.

The evaluation information generation unit 112 determines whether or not the analysis window R3 and the analysis window 4 correspond to the peeling region. FIG. 14 is a schematic view showing the peeling region F. As shown in the figure, the peeled region F is a region formed downstream of the constricted portion by the clip 302 and having almost no blood flow velocity. The peeled region is also formed downstream of the confluence or bifurcation of blood vessels, as shown in FIGS. 5 to 7. The evaluation information generation unit 112 can determine that the analysis window corresponds to the peeled region when the blood flow velocity in the analysis window is equal to or less than the threshold value. In FIG. 13, it is determined that the analysis window W3 corresponds to the peeling region and the analysis window W4 does not correspond to the peeling region.

Since the peeling region is formed on the downstream side as described above, the evaluation information generation unit 112 can estimate the blood flow direction based on the position of the peeling region, and specifically, the analysis window that does not correspond to the peeling region. It is possible to estimate that the direction toward the analysis window corresponding to the peeled region is the blood flow direction. Further, the evaluation information generation unit 112 can estimate the blood flow direction based on the position of the detached region even when the blood vessels 301 merge or branch as shown in FIGS. 5 to 7. Further, the evaluation information generation unit 112 can estimate the blood flow direction based on the positional relationship between the narrowed portion detected by image processing or the like and the analysis window that does not correspond to the peeled region.

The evaluation information generation unit 112 can supply the estimated blood flow direction as evaluation information to the evaluation information presentation unit 113. The evaluation information presentation unit 113 can generate a presentation image in which the blood flow direction is indicated by an arrow or the like. Further, the evaluation information generation unit 112 may generate a presentation image using a marker or the like indicating a peeling region as evaluation information as shown in FIG.

[About setting the analysis range]
The evaluation information generation unit 112 may use the entire imaging range as the analysis target, or may use only a part of the imaging range as the analysis range. FIG. 15 is a schematic diagram showing the analysis range. The evaluation information generation unit 112 can perform image processing on a normal optical image as shown in FIG. 15 (a) and detect a blood vessel as shown by an arrow in FIG. 15 (b).

As shown in FIG. 15C, the evaluation information generation unit 112 analyzes the analysis range A based on the detected blood vessel thickness, color (artery or vein), presence / absence of anastomosis, presence / absence of increased blood flow, and the like. Can be set. Further, the evaluation information generation unit 112 may determine the presence / absence of blood flow and the blood flow velocity from the blood flow index, and set the analysis range A. Further, the evaluation information generation unit 112 may set the central portion of the imaging range or the like as the analysis range A.

Further, the evaluation information generation unit 112 may set the analysis range A according to the designation by the user. The evaluation information presentation unit 113 can adjust the display parameters in the analysis range A so as to maximize the dynamic range in the analysis range A. The evaluation information generation unit 112 can execute the estimation of the blood flow direction as described above within the range of the analysis range A.

[Operation of image processing device]
The operation of the image processing device 103 will be described. FIG. 16 is a flowchart showing a processing flow of the image processing apparatus 103.

When the imaging range of the evaluation object is imaged while the evaluation object S is irradiated with normal light from the lighting device 101 and the normal light image is imaged, the image processing device 103 acquires the normal light image (St101). ). When the evaluation object S is irradiated with coherent light from the illumination device 101 at the same time as or at the same time as the normal light image is captured and the speckle image in the imaging range is captured, the image processing device 103 acquires the speckle image. (St102).

Subsequently, the blood flow index calculation unit 111 calculates the blood flow index (St103), and calculates the blood flow velocity distribution. The evaluation information generation unit 112 sets an analysis window in the blood flow velocity distribution (St104). The evaluation information generation unit 112 can set the analysis window based on a normal optical image or a blood flow index.

The evaluation information generation unit 112 compares the blood flow velocity in the analysis window (St105) and determines the blood flow direction (St106). The evaluation information generation unit 112 supplies the determined blood flow direction to the evaluation information presentation unit 113, and the evaluation information presentation unit 113 generates a presentation image showing the blood flow direction (St107), and causes the display device 104 to display.

The processing flow shown here is an example, and the evaluation information generation unit 112 does not set the analysis window, and determines the blood flow direction using the shape of the high flow velocity region and the position of the peeling region as described above. May be good. In addition, other processing such as noise processing may be added to this processing flow, and motion correction processing may be executed using the image processing result or the like for a normal optical image.

[Blood flow evaluation method using image processing system]
FIG. 17 is a flowchart showing an example of a blood flow evaluation method using the image processing system 100.

The imaging range is imaged by the imaging device 102 in a state where the evaluation object S is irradiated with the normal light from the lighting device 101, and the normal light image is imaged (St121). The imaging range is imaged by the imaging device 102 in a state where the evaluation object S is irradiated with coherent light from the lighting device 101 at the same time as or at the same time as the imaging of the normal light image, and the speckle image is captured (St122).

Subsequently, the blood flow index calculation unit 111 calculates the blood flow index (St123). The evaluation information presentation unit 113 can generate a presentation image showing the blood flow velocity distribution display and present it to the user.

The user selects an analysis method by referring to the blood flow velocity distribution (St124). The user can instruct the evaluation information generation unit 112 to analyze the entire region of the imaging range (St125).

Further, the evaluation information generation unit 112 can set the analysis range from the normal optical image (St126). The evaluation information generation unit 112 detects an evaluation target such as a blood vessel or a blood vessel bifurcation by image processing in a normal optical image, and the range including the detected evaluation target can be set as an analysis range (see FIG. 15). Further, the evaluation information generation unit 112 can set the analysis range by using the existing segmentation method.

In addition, the evaluation information generation unit 112 can set the analysis range from the blood flow index (St127). The evaluation information generation unit 112 can set a region in which the presence / absence of blood flow and the blood flow velocity are above a certain level as the analysis range. Further, the evaluation information generation unit 112 may set the analysis range by combining the blood flow index with the image processing result for the normal optical image.

In addition, the user may set an arbitrary analysis range in the imaging range (St128). The user can set an arbitrary analysis range including, for example, the presence / absence of anastomosis, the presence / absence of increased blood flow, a blood vessel having a certain level or higher, a blood vessel having a flow rate of a certain level or higher, a blood vessel in the center of the image, and the like.

When analyzing the entire region of the imaging range (St125), the evaluation information generation unit 112 detects a region where the flow velocity / flow velocity distribution is asymmetric and the flow is separated (St129). When a region with asymmetry of flow velocity / flow velocity distribution or flow separation is detected (St129; Yes), or when the analysis range is set by a normal optical image or blood flow index (St126, 127), between the analysis windows. Detects a region with asymmetry of flow velocity and flow velocity distribution and flow separation (St130).

When the evaluation information generation unit 112 detects a region where the flow velocity / flow velocity distribution is asymmetric or has a flow separation between the analysis windows (St130; Yes), or when an arbitrary analysis range is set (St128), blood The flow direction is determined (St131). When the evaluation information generation unit 112 cannot detect the asymmetry of the flow velocity / flow velocity distribution (St129, 130; No) or cannot determine the blood flow direction (St131: No), another analysis method is used. Recommended (St132).

When the blood flow direction is supplied from the evaluation information generation unit 112, the evaluation information presentation unit 113 generates a presentation image that presents the blood flow direction and displays it on the display device 104 (St133).

[Effect of image processing system]
As described above, in the image processing system 100, it is possible to determine the blood flow direction from the blood flow index calculated from the speckle image. In addition, by using the blood flow index, it is possible to observe a delicate blood flow velocity distribution, and it is also possible to observe a blood flow asymmetrical with respect to the flow in the narrowed portion.

The following points can be mentioned as the medical value of the image processing system 100. In neurosurgery, there is a desire to detect blood clots that occur during surgery. For example, in bypass surgery, blood clots may form in the donor blood vessels and blood flow may not be improved as intended. Here, by using the image processing system 100, the unsteady flow of blood is detected, and information on the thrombus generated during the operation is obtained to easily and safely judge the success or failure of the procedure and predict the possibility of complications. It is possible to do. In the image processing system 100, it is not necessary to use a special fluorescent agent such as ICG or an ultrasonic probe for observing blood flow, and the risk of complications can be reduced without imposing a burden on the operator or the patient. ..

Furthermore, in neurosurgery, there is a desire to evaluate the direction of blood flow. For example, bypass surgery is used to confirm that the donor vessel is nourishing the recipient vessel as intended. As a method of confirming the resumption of blood flow, there is also a method of fluorescence observation using ICG or the like, but in this method as well, blood perfusion by bypass can be confirmed, but the blood flow direction is unknown.

FIG. 18 is a schematic view showing the blood flow direction in the bypass operation, showing a state in which the blood vessel 305 is anastomosed to the blood vessel 304. When the user predicts that blood flow will flow in the direction indicated by the arrow in FIG. 18 (a) and anastomoses the blood vessels, as shown by the arrow in FIG. 18 (b), blood flow may flow in an unintended direction. appear.

Since the image processing system 100 can determine the blood flow direction as shown in FIG. 18, it is possible to easily and safely determine the success or failure of the procedure and predict the possibility of complications. The risk of complications can be reduced without burdening the patient or the patient. Furthermore, the blood vessels of the brain run in a complicated manner, and it is not easy to grasp which blood vessels are connected to where and which part is nourished. Regarding this point as well, by using the image processing system 100, the nutritional range of blood vessels can be estimated, and surgery can be performed safely.

[Hardware configuration of image processing device]
A hardware configuration capable of realizing a functional configuration of the image processing device 103 will be described. FIG. 19 is a schematic diagram showing a hardware configuration of the image processing device 103.

As shown in the figure, the image processing device 103 has a built-in CPU (Central Processing Unit) 1001. The input / output interface 1005 is connected to the CPU 1001 via the bus 1004. A ROM (Read Only Memory) 1002 and a RAM (Random Access Memory) 1003 are connected to the bus 1004.

The input / output interface 1005 includes an input unit 1006 including an input device such as a keyboard and a mouse for the user to input operation commands, an output unit 1007 for outputting a processing operation screen and an image of the processing result to a display device, a program, and various data. A storage unit 1008 consisting of a hard disk drive or the like for storing, a LAN (Local Area Network) adapter or the like, and a communication unit 1009 for executing communication processing via a network represented by the Internet are connected. Further, a drive 1010 for reading and writing data is connected to a removable storage medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

The CPU 1001 is read from a program stored in the ROM 1002 or a removable storage medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, installed in the storage unit 1008, and loaded from the storage unit 1008 into the RAM 1003. Various processes are executed according to the program. The RAM 1003 also appropriately stores data and the like necessary for the CPU 1001 to execute various processes.

In the image processing device 103 configured as described above, the CPU 1001 loads and executes the program stored in the storage unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004, for example. The series of processes described above is performed.

The program executed by the image processing device 103 can be recorded and provided on the removable storage medium 1011 as a package medium or the like, for example. Programs can also be provided via wired or wireless transmission media such as local area networks, the Internet, and digital satellite broadcasting.

In the image processing device 103, the program can be installed in the storage unit 1008 via the input / output interface 1005 by mounting the removable storage medium 1011 in the drive 1010. Further, the program can be received by the communication unit 1009 and installed in the storage unit 1008 via a wired or wireless transmission medium. In addition, the program can be pre-installed in the ROM 1002 or the storage unit 1008.

The program executed by the image processing apparatus 103 may be a program in which processing is performed in chronological order in the order described in the present disclosure, and is necessary in parallel or when calls are made. It may be a program in which processing is performed at the timing.

Further, the hardware configuration of the image processing device 103 does not have to be all mounted on one device, and the image processing device 103 may be configured by a plurality of devices. Further, it may be mounted on a part of the hardware configuration of the image processing device 103 or a plurality of devices connected via a network. Further, the circuit that realizes the function of the image processing device 103 may be a logic circuit such as an FPGA (field-programmable gate array) or a GPU (Graphics Processing Unit).

(Second embodiment)
The image processing system according to the second embodiment of the present technology will be described.

[Image processing system configuration]
FIG. 20 is a schematic view showing the configuration of the image processing system 200 according to the present embodiment. As shown in the figure, the image processing system 200 includes a lighting device 201, an image pickup device 202, an image processing device 203, and a display device 204. Further, in FIG. 1, the object to be evaluated by the image processing system 200 is shown as the evaluation object S. The evaluation object S is a biological tissue such as the brain or other organs, and is not particularly limited.

The lighting device 201, the image pickup device 202, and the display device 204 each have the same configuration as that of the first embodiment. That is, the illumination device 201 irradiates the evaluation object S with coherent light and normal light, and the image pickup device 202 captures the speckle image and the normal light image and outputs the speckle image and the normal light image to the image processing device 203. In the following description, the range on the evaluation object S on which the speckle image and the normal light image are captured by the image pickup device 202 (that is, the field view range of the image pickup device 202) is referred to as an “imaging range”. The display device 204 displays the presented image output from the image processing device 203.

[About imaging time]
The image pickup apparatus 202 captures a normal optical image and a speckle image for a certain period of time (hereinafter referred to as an imaging time) and outputs the images to the image processing apparatus 203. Hereinafter, a part of the imaging time will be referred to as a first time and a second time. FIG. 21 is a schematic view showing the first time J1 and the second time J2. As shown in the figure, the first time J1 is, for example, the time before the blood vessel is sutured in the blood vessel bypass operation, and the second time J2 is, for example, the time after the blood vessel is sutured in the same operation. In addition to this, the first time J1 and the second time J2 can be set to each timing during the operation, and the details will be described later.

The illuminating device 201 irradiates the evaluation object S with normal light in the first time J1, and the imaging device 202 captures a normal light image in the first time J1. Hereinafter, this normal light image will be referred to as a first normal light image. Further, the lighting device 201 irradiates the evaluation object S with coherent light in the first time J1, and the imaging device 202 captures a speckle image in the first time J1. Hereinafter, this speckle image will be referred to as a first speckle image.

The imaging timings of the first normal light image and the first speckle image may be the same, and may be slightly different within the first time J1. The length of the first time J1 is not particularly limited, but is, for example, about several seconds.

Similarly, the lighting device 201 irradiates the evaluation object S with normal light in the second time J2, and the image pickup device 202 captures the normal light image in the second time J2. Hereinafter, this normal light image will be referred to as a second normal light image. Further, the lighting device 201 irradiates the evaluation object S with coherent light in the second time J2, and the imaging device 202 images a speckle image in the second time J2. Hereinafter, this speckle image will be referred to as a second speckle image.

The second normal light time and the imaging timing of the second speckle image may be at the same time, and may be slightly different as long as they are within the second time J2. The length of the second time J2 is not particularly limited, but is, for example, about several seconds.

[Configuration of image processing device]
FIG. 22 is a block diagram showing the configuration of the image processing device 203. As shown in the figure, the image processing device 203 includes a blood flow index calculation unit 211, a motion estimation unit 212, an evaluation information generation unit 213, and an evaluation information presentation unit 214.

The blood flow index calculation unit 211 acquires the first speckle image and the second speckle image from the imaging device 202, and the “blood flow index” which is an index of blood flow from the first speckle image and the second speckle image. Is calculated. The blood flow index is at least one of SC (speckle contrast), SFI (speckle flow index) and BFI (blood flow index). Hereinafter, the blood flow index calculated from the first speckle image will be referred to as a "first blood flow index", and the blood flow index calculated from the second speckle image will be referred to as a "second blood flow index". The blood flow index calculation unit 211 outputs the calculated first blood flow index and second blood flow index to the motion estimation unit 212 and the evaluation information generation unit 213.

The motion estimation unit 212 specifies the motion from the calculation position of the first blood flow index to the calculation position of the second blood flow index between the first time J1 and the second time J2. FIG. 23 is a block diagram showing the configuration of the motion estimation unit 212. As shown in the figure, the motion estimation unit 212 includes a first motion calculation unit 251, a second motion calculation unit 252, a third motion calculation unit 253, a correlation calculation unit 254, and a motion vector determination unit 255.

The first motion calculation unit 251 compares the first ordinary light image with the second ordinary light image, and estimates the motion vector from the first ordinary light image to the second ordinary light image. FIG. 24 is a schematic view showing the movement from the first normal light image G1 to the second normal light image G2. In the example shown in the figure, the imaging range for the evaluation object S is shifted between the first time J1 and the second time J2.

As shown in the figure, when the first normal light image G1 and the second normal light image G2 are divided into a large number of small sections, the pixel group in the section K1 in the first normal light image G1 is the second normal light image G2. It corresponds to the pixel group in the compartment K2 in, that is, the movement from the compartment K1 to the compartment K2 occurs.

The first motion calculation unit 251 compares both images by image processing for the first ordinary light image G1 and the second ordinary light image G2, detects a corresponding pixel group, and detects the corresponding pixel group, and the first ordinary light image G1 to the second ordinary light. The motion vector to the image G2 can be estimated.

Although the movement due to the movement of the imaging range is shown in FIG. 24, even if the imaging range is the same, the living tissue may be moved or deformed due to the operation, and the first motion calculation unit 251 may also perform these cases. The motion vector from the first normal light image G1 to the second normal light image G2 can be estimated. The first motion calculation unit 251 supplies the estimated motion vector to the correlation calculation unit 254 and the motion vector determination unit 255.

The second motion calculation unit 252 compares the first speckle image with the second speckle image, and estimates the motion vector from the first speckle image to the second speckle image. Similar to the first motion calculation unit 251, the second motion calculation unit 252 compares both images by image processing on the first speckle image and the second speckle image, detects the corresponding pixel group, and detects the corresponding pixel group, and detects the corresponding pixel group, and detects the corresponding pixel group. The motion vector from the image to the second speckle image can be estimated. The second motion calculation unit 252 supplies the estimated motion vector to the correlation calculation unit 254 and the motion vector determination unit 255.

The third motion calculation unit 253 compares the first blood flow index and the second blood flow index, and estimates the motion vector from the calculation position of the first blood flow index to the calculation position of the second blood flow index. The third motion calculation unit 253, for example, binarizes the first blood flow index and the second blood flow index, extracts the blood vessel structure in which blood flow exists from each, and matches the extracted blood vessel structures. Estimate the motion vector required for. The third motion calculation unit 253 can estimate this motion vector as a motion vector from the calculation position of the first blood flow index to the calculation position of the second blood flow index. The third motion calculation unit 253 supplies the estimated motion vector to the correlation calculation unit 254 and the motion vector determination unit 255.

The correlation calculation unit 254 acquires motion vectors from the first motion calculation unit 251, the second motion calculation unit 252, and the third motion calculation unit 253, respectively, and calculates the motion correlation (hereinafter, motion correlation) from each motion vector. To do. The correlation calculation unit 254 can calculate the motion correlation by calculating an index capable of evaluating the sum of the absolute values of the motion vectors, the sum of the differences squared of the motion vectors, the correlation index, or other correlation (similarity). The correlation calculation unit 254 outputs the calculated motion correlation to the motion vector determination unit 255.

The motion vector determination unit 255 determines the motion vector by using the motion correlation from the motion vectors acquired from the first motion calculation unit 251 and the second motion calculation unit 252 and the third motion calculation unit 253, respectively. The motion vector determination unit 255 can select the motion vector having the highest correlation from each motion vector and determine the motion vector to be output.

The motion estimation unit 212 determines the motion vector as described above, and specifies the motion from the calculation position of the first blood flow index to the calculation position of the second blood flow index. The motion estimation unit 212 supplies the determined motion vector to the evaluation information generation unit 213.

The evaluation information generation unit 213 generates evaluation information of blood flow including the blood flow direction. Specifically, the evaluation information generation unit 213 generates "increase / decrease information" indicating an increase / decrease in the first blood flow index and the second blood flow index as evaluation information. As will be described later, the evaluation information generation unit 213 calculates the evaluation index calculated from the first blood flow index and the second blood flow index, and the difference between the first blood flow index and the second blood flow index as increase / decrease information. Can be done.

At this time, the evaluation information generation unit 213 may compensate the movement from the calculation position of the first blood flow index to the calculation position of the second blood flow index by using the motion vector supplied from the motion estimation unit 212. it can.

The evaluation information presentation unit 214 acquires a speckle image, a normal optical image, and evaluation information, and integrates these information to generate a presentation image. The evaluation information presentation unit 214 can generate a presentation image by superimposing the visualized evaluation information on, for example, a normal optical image. Further, the evaluation information presentation unit 214 may select either a speckle image or a normal light image as a presentation image, or both images may be arranged in parallel or superimposed to form a presentation image. Further, the evaluation information presentation unit 214 can also generate a difference image as a presentation image, which is an image of the difference between the first blood flow index and the second blood flow index, and superimposes the normal optical image and the difference image. It may be a presentation image. The evaluation information presentation unit 214 outputs the generated presentation image to the display device 204.

[About the evaluation information generator]
The evaluation information generation unit 213 generates the increase / decrease information indicating the increase / decrease of the first blood flow index and the second blood flow index as the evaluation information as described above.

FIG. 25 is a schematic diagram showing a method of generating increase / decrease information by the evaluation information generation unit 213. Bypass surgery in which the donor blood vessel D is anastomosed to the recipient blood vessel R shown in FIG. 25 (a) as shown in FIG. 25 (b) will be described as an example. The time before the anastomosis shown in FIG. 25 (a) is set as the first time J1, and the first normal optical image and the first speckle image are imaged. Further, the time before the anastomosis shown in FIG. 25 (b) is set as the second time J2, and the second normal optical image and the second speckle image are captured.

_{The position P 1} in FIG. 25 (a) _{and the position P 2} in FIG. 25 (b) are located at the same position with respect to the recipient blood vessel R, but the position in the image is changed due to the movement of the imaging range with respect to the recipient blood vessel R. It is different. Evaluation information generation unit 213 can grasp that the position P ₁ is moved to the position P ₂ by the motion vector supplied from the motion estimation unit 212.

Evaluation information generation unit 213 Accordingly, it is possible to generate an increase or decrease information from the second blood flow indicator is a blood flow index of the first blood flow index and the position P2 is a blood flow indicator position P _1. The evaluation information generation unit 213 can calculate an evaluation index indicating an increase / decrease (difference, ratio, change rate, etc.) between the first blood flow index and the second blood flow index as increase / decrease information.

For example the blood flow of the cortical region first blood flow index at position P ₁ (hereinafter, cortical blood flow) is V _c1 is a blood flow velocity of the blood flow in cortical blood flow second blood flow index at the position P ₂ In _{the case of V c2} which is the velocity, the evaluation information generation unit 213 can generate the ratio (V _c1 / V _{c2 ) of the first blood flow index V c1} and the second blood flow index V _c2 as the evaluation index. In addition to this, the evaluation information generation unit 213 can generate various evaluation indexes as increase / decrease information at each timing of surgery. The details will be described later.

Further, the evaluation information generation unit 213 can calculate the difference between the first blood flow index and the second blood flow index, and generate a difference image in which the difference can be imaged as increase / decrease information. FIG. 26 is an example of a difference image. As shown in the figure, the evaluation information generation unit 213 generates a difference image by displaying a region where the difference between the first blood flow index and the second blood flow index is large in white and a region where the difference is small in black. Is possible.

In addition to the increase / decrease information, the evaluation information generation unit 213 can also generate the blood flow direction as the evaluation information as described in the first embodiment. Further, the evaluation information generation unit 213 may use the entire imaging range as the analysis target, or may use only a part of the imaging range as the analysis range. The evaluation information generation unit 213 may set the analysis range based on the image processing result for the normal optical image and the blood flow index as in the first embodiment, or may set the analysis range according to the user's designation. ..

[Operation of image processing device]
The operation of the image processing device 203 will be described. FIG. 27 is a flowchart showing a processing flow of the image processing device 203.

When the imaging range of the evaluation object is imaged while the evaluation object S is irradiated with normal light from the lighting device 201 and the normal light image is imaged, the image processing device 203 acquires the normal light image (St201). ). When the evaluation target S is irradiated with coherent light from the lighting device 201 at the same time as or at the same time as the normal light image is captured and the speckle image in the imaging range is captured, the image processing device 203 acquires the speckle image. (St202).

The blood flow index calculation unit 211 selects a normal light image to be used as a first normal light image and a second normal light image, and selects a first blood flow index and a second from the corresponding first speckle image and second speckle image. Calculate the blood flow index (St203). The motion estimation unit 212 estimates the movement from the calculation position of the first blood flow index to the calculation position of the second blood flow index (St204), and supplies the motion to the evaluation information generation unit 213.

The evaluation information generation unit 213 compensates for the movement from the calculation position of the first blood flow index to the calculation position of the second blood flow index (St205), and increases / decreases the increase / decrease information of the first blood flow index and the second blood flow index. It is calculated as (St206). This increase / decrease information includes the evaluation index and the difference image as described above. The evaluation information generation unit 213 supplies the calculated increase / decrease information to the evaluation information presentation unit 214, and the evaluation information presentation unit 214 generates a presentation image showing the increase / decrease information (St207) and displays it on the display device 204.

The processing flow shown here is an example, and other processing such as noise processing may be added. The blood flow index calculation unit 211 may specify the first normal light image and the second normal light image by the user, or may automatically select the first normal light image and the second normal light image by image processing or the like. In addition to the evaluation information, the evaluation information presentation unit 214 can also select arbitrary information from a normal optical image, a speckle image, a blood flow index, and the like to serve as a presentation image.

[Blood flow evaluation method using image processing system]
FIG. 28 is a flowchart showing an example of a blood flow evaluation method using the image processing system 200. The following flowchart shows a blood flow evaluation method for obtaining the blood flow direction described in the first embodiment as evaluation information in addition to the increase / decrease information.

The imaging range is imaged by the imaging device 202 in a state where the evaluation object S is irradiated with the normal light from the lighting device 201, and the normal light image is imaged (St221). At the same time as or at the same time as the normal light image is captured, the imaging range is imaged by the image pickup device 202 in a state where the coherent light is irradiated to the evaluation object S from the illumination device 201, and the speckle image is imaged (St222).

Subsequently, the blood flow index calculation unit 211 calculates the blood flow index (St223). The blood flow index calculation unit 211 selects a first normal light image and a second normal light image from the normal light image, and the first blood flow index and the second from the corresponding first speckle image and second speckle image, respectively. The blood flow index can be calculated.

In addition, the motion estimation unit 212 estimates the motion of the calculation positions of the first blood flow index and the second blood flow index from at least one of the normal light image, the speckle image, and the blood flow index (St224). The evaluation information generation unit 213 compensates for the movement from the calculation position of the first blood flow index to the calculation position of the second blood flow index using the movement estimated by the motion estimation unit 212 (St225), and the first blood flow. Increase / decrease information indicating the increase / decrease of the index and the second blood flow index is generated (St226).

In addition, the user selects an analysis method by referring to the blood flow velocity distribution (St227). The user can instruct the evaluation information generation unit 213 to analyze the entire region of the imaging range (St228).

Further, the evaluation information generation unit 213 can set the analysis range from the normal optical image (St229). The evaluation information generation unit 213 can detect an evaluation target such as a blood vessel or a blood vessel bifurcation in a normal optical image by image processing, and set the range including the detected evaluation target as the analysis range (see FIG. 15). Further, the evaluation information generation unit 213 can set the analysis range by using the existing segmentation method.

In addition, the evaluation information generation unit 213 can set the analysis range from the blood flow index (St230). The evaluation information generation unit 213 can set a region in which the presence / absence of blood flow and the blood flow velocity are above a certain level as the analysis range. Further, the evaluation information generation unit 213 may set the analysis range by combining the blood flow index with the image processing result for the normal optical image.

In addition, the user may set an arbitrary analysis range in the imaging range (St231). The user can set an arbitrary analysis range including, for example, the presence / absence of anastomosis, the presence / absence of increased blood flow, a blood vessel having a certain level or higher, a blood vessel having a flow rate of a certain level or higher, a blood vessel in the center of the image, and the like.

When analyzing the entire region of the imaging range (St228), the evaluation information generation unit 213 detects a region where the flow velocity / flow velocity distribution is asymmetric and the flow is separated (St232). When a region with asymmetry of flow velocity / flow velocity distribution or flow separation is detected (St232; Yes), or when the analysis range is set by a normal optical image or blood flow index (St229, 230), between analysis windows. Detects a region with asymmetry of flow velocity and flow velocity distribution and flow separation (St233).

When the evaluation information generation unit 213 detects a region where the flow velocity / flow velocity distribution is asymmetric or has a flow separation between the analysis windows (St233; Yes), or when an arbitrary analysis range is set (St231), blood The flow direction is determined (St234). When the evaluation information generation unit 112 cannot detect the asymmetry of the flow velocity / flow velocity distribution (St232, 233; No) or cannot determine the blood flow direction (St234: No), another analysis method is used. Recommended (St235).

When the blood flow direction is supplied from the evaluation information generation unit 213, the evaluation information presentation unit 214 generates a presentation image including the increase / decrease information and the blood flow direction and displays it on the display device 204 (St236).

[Example of procedure flow and evaluation information]
Hereinafter, an example of the flow of the procedure by the image processing system 200 and the evaluation index generated by the evaluation information generation unit 213 as increase / decrease information will be described.

<Evaluation index example 1>
FIG. 29 is a schematic diagram showing a recipient blood vessel R and a donor blood vessel D. V _R1 and V _R2 are blood flow indexes calculated from the speckle image by the blood flow index calculation unit 211, and V _R1 is the notation position of the recipient blood vessel R (in the figure, it is written as "_VR1". The blood flow index indicating the blood flow velocity at the position (the same applies hereinafter), _VR2 is a blood flow index indicating the blood flow velocity at the position indicated by the recipient blood vessel R.

Evaluation information generation unit _{213, V} R2 _/ V _R1 or _{(V R1 -V R2) / (} V R1 + V R2) can be calculated as the evaluation index.

<Evaluation index example 2>
FIG. 30 is a schematic view showing the flow of the procedure in the operation of anastomosing the donor blood vessel D with the recipient blood vessel R. The lower part of the figure shows the blood flow (cortical blood flow) in the cortical region located near the recipient blood vessel R and the donor blood vessel D and nourished by these blood vessels.

At time T0, the anastomotic position is selected in the recipient blood vessel. At time T1, the donor blood vessel D is prepared, and the blood flow of the donor blood vessel D is blocked by the clip C. At time T2, the donor vessel D is anastomosed to the recipient vessel R, and the donor vessel D and the recipient vessel R are blocked from blood flow by the clip C.

At time T3, the clip C of the recipient vessel R is removed and the recipient vessel R is reperfused. At time T4, clip C of donor vessel D is removed and donor vessel D is perfused.

Blood flow index calculating unit 211, the time T0 as the first hour J1, it is possible to calculate the blood flow index _{V Cb0, V Cb1} and _{V Cb2} from the first speckle image. Blood flow indicator V _Cb0 indicates the blood flow velocity of the cortical blood flow at the location of the representation, the blood flow index V _Cb1 is the blood flow rate of cortical blood flow at the location of the representation, the blood flow index V _Cb2 is the notation Shows the blood flow velocity of cortical blood flow at the position.

_{The blood flow index calculation unit 211 can calculate the blood flow indexes V Ca0} , V _Ca1 and V _Ca2 from the second speckle image, with the time T4 as the second time J2. Blood flow indicator _{V Ca0} indicates the blood flow velocity of the cortical blood flow at the position _{C 0,} blood flow indicator _{V Ca1} is the blood flow rate of cortical blood flow at position _{C 1,} blood flow indicator _{V Ca @ 2,} the position C The blood flow velocity of the cortical blood flow in No. _{2 is shown.}

_{The evaluation information generation unit 213 can specify} the blood flow indexes V Ca0, V _Ca1 and V _Ca2 by using the motion vector supplied from the motion estimation unit 212. Evaluation information generation unit 213 can calculate, based on the time _T0, the metrics _{V Ca0 / V Cb0, V Ca1} / V Cb1 or _V Ca2 _{/ V Cb2} as increase or decrease information.

_{Incidentally,} V _Ca0, the value of V _{Ca1, V Ca @ 2} is (in the figure, an arrow E1) blood flow rate of cortical blood flow at time T4 _represent, V _Cb0, the value of V _{Cb1, V Cb2} is cortex at time T0 It represents the blood flow velocity of blood flow (arrow E2 in the figure).

<Evaluation index example 3>
FIG. 31 is a schematic view showing the flow of the procedure in the operation of anastomosing the donor blood vessel D with the recipient blood vessel R, and the flow of the procedure is the same as that of the evaluation index Example 2.

The blood flow index calculation unit 211 can calculate the _{blood flow indexes V R0} and V _D0 from the first speckle image, with the time T2 as the first time J1. The blood flow index V _R0 indicates the blood flow velocity of the blood flow flowing through the recipient blood vessel R at time T2 (hereinafter referred to as the recipient blood flow), and the blood flow index V _D0 is the blood flow flowing through the donor blood vessel D at time T2. (Hereinafter, the blood flow velocity of the donor blood flow) is shown.

Further, the blood flow index calculation unit 211 can calculate the _{blood flow indexes V R1} and V _{D 1} from the second speckle image, with the time T3 as the second time J2. The blood flow index V _R1 indicates the blood flow velocity of the recipient blood flow at time T3, and the blood flow index V _D1 indicates the blood flow velocity of the donor blood flow at time T3.

Further, the blood flow index calculation unit 211 can calculate the _{blood flow indexes V R2} and V _D 2 from the second speckle image, with the time T4 as the second time J2. The blood flow index V _R2 indicates the blood flow velocity of the recipient blood flow at time T4, and the blood flow index V _D2 indicates the blood flow velocity of the donor blood flow at time T4.

The evaluation information generation unit 213 can identify the blood flow indexes V _R1 , V _D1 , VR ₂ and V _{D 2} by using the motion vector supplied from the motion estimation unit 212. Evaluation information generation unit 213 can calculate the evaluation index as the reference time _{T2 (V R2 -V R0) /} (V R1 -V R0) as the increase or decrease information.

_{Incidentally,} (V R2 _{-V R0)} values, (in the figure, an arrow E3) blood flow velocity of the recipient blood flow at time T4 represents _the value of (V R1 _{-V R0)} is the recipient blood at time T3 It represents the blood flow velocity of the flow (arrow E4 in the figure).

<Evaluation index example 4>
FIG. 32 is a schematic view showing the flow of the procedure in the operation of anastomosing the donor blood vessel D with the recipient blood vessel R, and the flow of the procedure is the same as that of the evaluation index Example 2.

The blood flow index calculation unit 211 can calculate the _{blood flow index VRb0} from the first speckle image with the time T0 as the first time J1. The blood flow index _VRb0 indicates the blood flow velocity of the recipient blood flow at time T0.

Further, the blood flow index calculation unit 211 can calculate the _{blood flow indexes V R0} and V _D0 from the second speckle image, with the time T2 as the second time J2. The blood flow index V _R0 indicates the blood flow velocity of the recipient blood flow at time T2, and the blood flow index V _D0 indicates the blood flow velocity of the donor blood flow at time T2.

Further, the blood flow index calculation unit 211 can calculate the _{blood flow indexes V R2} and V _D 2 from the second speckle image, with the time T4 as the second time J2. The blood flow index V _R2 indicates the blood flow velocity of the recipient blood flow at time T4, and the blood flow index V _D2 indicates the blood flow velocity of the donor blood flow at time T4.

The evaluation information generation unit 213 can specify the _{blood flow indexes V R0} , V _D0 , _VR2 , _VR2 and V _D2 by using the motion vector supplied from the motion estimation unit 212. Evaluation information generation unit 213, based on the time _T2, can be calculated as decrease information metrics _{(V R2 -V R0) / (} V Rb0 -V R0).

_The value of (V R2 _{-V R0)} is (in the figure, an arrow E5) blood flow velocity of the recipient blood flow at time T4 represents _the value of _(V Rb0 _{-V R0)} is the recipient at time T0 It represents the blood flow velocity of blood flow (arrow E6 in the figure).

<Evaluation index example 5>
FIG. 33 is a schematic view showing the flow of the procedure in the operation of anastomosing the donor blood vessel D with the recipient blood vessel R, and the flow of the procedure is the same as that of the evaluation index Example 2.

The blood flow index calculation unit 211 can calculate the _{blood flow index VRb0} from the first speckle image with the time T0 as the first time J1. The blood flow index _VRb0 indicates the blood flow velocity of the recipient blood flow at time T0.

Further, the blood flow index calculation unit 211 can calculate the _{blood flow indexes V R0} and V _D0 from the second speckle image, with the time T2 as the second time J2. The blood flow index V _R0 indicates the blood flow velocity of the recipient blood flow at time T2, and the blood flow index V _D0 indicates the blood flow velocity of the donor blood flow at time T2.

Further, the blood flow index calculation unit 211 can calculate the _{blood flow indexes V R2} and V _D 2 from the second speckle image, with the time T4 as the second time J2. The blood flow index V _R2 indicates the blood flow velocity of the recipient blood flow at time T4, and the blood flow index V _D2 indicates the blood flow velocity of the donor blood flow at time T4.

The evaluation information generation unit 213 can specify the _{blood flow indexes V R0} , V _D0 , V _D2, and VR ₂ by using the motion vector supplied from the motion estimation unit 212. Evaluation information generation unit 213, based on the time _T2, can be calculated as the evaluation information evaluation index _{(V R2 -V Rb0) / (} V D2 -V D0).

_The value of (V _{R2 -V Rb0)} represents the blood flow velocity difference of the recipient blood flow between times T0 and the time T4 (in the figure, an arrow _E7), the value of (V D2 _{-V D0)} Represents the blood flow velocity of the donor blood flow at time T4 (arrow E8 in the figure).

[Effect of image processing system]
As described above, in the image processing system 200, the difference between the first blood flow index and the second blood flow index can be presented to the user as evaluation information. By using the image processing system 200, before and after the bypass operation. It is possible to judge the change in blood flow in the above, and it is possible to easily and safely judge the success or failure of the procedure and predict the possibility of complications. Further, in the image processing system 200, it is not necessary to use a special fluorescent agent such as ICG or an ultrasonic probe for observing the blood flow, and the blood flow can be evaluated without imposing a burden on the operator or the patient.

[Hardware configuration of image processing device]
The functional configuration of the image processing device 203 can be realized by the same hardware configuration (see FIG. 19) as the image processing device 103 according to the first embodiment.

The present technology can have the following configurations.
(1)
A blood flow index calculation unit that calculates a blood flow index, which is an index of blood flow, from a speckle image that captures a speckle pattern formed on an evaluation object.
An evaluation information generation unit that generates evaluation information of blood flow including the blood flow direction based on the above blood flow index,
An image processing device including an evaluation information presenting unit that presents the above evaluation information.
(2)
The image processing apparatus according to (1) above.
The evaluation information generation unit is an image processing device that estimates the blood flow direction from the blood flow index including the blood flow velocity distribution, which is the distribution of the blood flow velocity with respect to the position of the speckle image.
(3)
The image processing apparatus according to (2) above.
The evaluation information generation unit is an image processing device that estimates the blood flow direction based on the blood flow velocity at the first position of the speckle image and the blood flow velocity at the second position of the speckle image. ..
(4)
The image processing apparatus according to (3) above.
The evaluation information generation unit estimates that the direction from the position where the blood flow velocity is low to the position where the blood flow velocity is high is the blood flow direction among the first position and the second position. Processing equipment.
(5)
The image processing apparatus according to (2) above.
The evaluation information generation unit identifies a high flow velocity region, which is a region where the blood flow velocity exceeds a certain value in the blood flow velocity distribution, and estimates the blood flow direction based on the area of the high flow velocity region. apparatus.
(6)
The image processing apparatus according to (2) above.
The evaluation information generation unit is an image processing device that identifies a peeling region in which the blood flow velocity is equal to or less than a certain value in the blood flow velocity distribution, and estimates the blood flow direction based on the position of the peeling region.
(7)
The image processing apparatus according to (3) or (4) above.
The evaluation information generation unit is an image processing device that sets the first position and the second position based on the image processing result of the normal light image captured by irradiating the evaluation target with normal light.
(8)
The image processing apparatus according to (3) or (4) above.
The evaluation information generation unit is an image processing device that sets the first position and the second position based on the blood flow index.
(9)
A lighting device that irradiates the evaluation object with coherent light and normal light,
The speckle pattern generated by irradiating the evaluation object with the coherent light is imaged to generate a speckle image, and the evaluation object irradiated with the normal light is imaged to obtain a normal light image. The image pickup device to generate and
A blood flow index calculation unit that calculates a blood flow index, which is an index of blood flow, from the speckle image, and an evaluation information generation unit that generates evaluation information of blood flow including the blood flow direction based on the blood flow index. An image processing system including an image processing device including an evaluation information presenting unit for presenting the evaluation information.
(10)
The blood flow index calculation unit calculates the blood flow index, which is an index of blood flow, from the speckle image obtained by capturing the speckle pattern formed on the evaluation object.
The evaluation information generation unit generates evaluation information of blood flow including the blood flow direction based on the above blood flow index.
An image processing method in which the evaluation information presenting unit presents the above evaluation information.

100, 200 ... Image processing system 101, 201 ... Lighting device 102, 202 ... Imaging device 103, 203 ... Image processing device 104, 204 ... Display device 111, 211 ... Blood flow index calculation unit 112, 213 ... Evaluation information generation unit 113 , 214 ... Evaluation information presentation unit 212 ... Motion estimation unit

Claims (10)

A blood flow index calculation unit that calculates a blood flow index, which is an index of blood flow, from a speckle image that captures a speckle pattern formed on an evaluation object.
An evaluation information generation unit that generates evaluation information of blood flow including the blood flow direction based on the blood flow index,
An image processing device including an evaluation information presenting unit that presents the evaluation information. The image processing apparatus according to claim 1.
The evaluation information generation unit is an image processing device that estimates the blood flow direction from the blood flow index including the blood flow velocity distribution, which is the distribution of the blood flow velocity with respect to the position of the speckle image. The image processing apparatus according to claim 2.
The evaluation information generation unit is an image processing device that estimates the blood flow direction based on the blood flow velocity at the first position of the speckle image and the blood flow velocity at the second position of the speckle image. .. The image processing apparatus according to claim 3.
The evaluation information generation unit estimates that the direction from the position where the blood flow velocity is low to the position where the blood flow velocity is high is the blood flow direction among the first position and the second position. Processing equipment. The image processing apparatus according to claim 2.
The evaluation information generation unit identifies a high flow velocity region, which is a region where the blood flow velocity exceeds a certain value in the blood flow velocity distribution, and estimates the blood flow direction based on the area of the high flow velocity region. apparatus. The image processing apparatus according to claim 2.
The evaluation information generation unit is an image processing device that identifies a peeling region in which the blood flow velocity is equal to or less than a certain value in the blood flow velocity distribution, and estimates the blood flow direction based on the position of the peeling region. The image processing apparatus according to claim 3.
The evaluation information generation unit is an image processing device that sets the first position and the second position based on the image processing result of the normal light image captured by irradiating the evaluation target with normal light. The image processing apparatus according to claim 3.
The evaluation information generation unit is an image processing device that sets the first position and the second position based on the blood flow index. A lighting device that irradiates the evaluation object with coherent light and normal light,
A speckle pattern generated by irradiating the evaluation object with the coherent light is imaged to generate a speckle image, and the evaluation object irradiated with the normal light is imaged to obtain a normal light image. The image pickup device to generate and
A blood flow index calculation unit that calculates a blood flow index, which is an index of blood flow, from the speckle image, and an evaluation information generation unit that generates evaluation information of blood flow including the blood flow direction based on the blood flow index. An image processing system including an image processing device including an evaluation information presenting unit for presenting the evaluation information. The blood flow index calculation unit calculates the blood flow index, which is an index of blood flow, from the speckle image obtained by capturing the speckle pattern formed on the evaluation object.
The evaluation information generation unit generates evaluation information of blood flow including the blood flow direction based on the blood flow index.
An image processing method in which the evaluation information presenting unit presents the evaluation information.

Images