JP4670054B2 - Ultrasonic diagnostic equipment - Google Patents

Description

The present invention assumes that tomography is performed using a robot for diagnosis support, instructs the robot to operate an ultrasonic probe, and uses an image processing technique to detect an ultrasonic tomogram. The present invention relates to an ultrasonic diagnostic apparatus including means for automatically extracting a tomographic plane and extracting a short-axis or long-axis tomographic plane image of the heart, which is useful for immediate diagnosis support.

The probe of an ultrasonic diagnostic device is a diagnostic device that can be used as easily as a stethoscope, observes the dynamic state of the region of interest by transmitting and receiving ultrasonic waves, and enables real-time diagnosis as well as simple diagnosis. The higher the ultrasonic frequency (the shorter the wavelength), the better the resolution and the better the distance resolution. However, in the diagnosis of deep organs, high-frequency ultrasound is greatly absorbed by the living body and its permeability decreases, so it is used to obtain tomographic images that require dynamic information on deep organs such as cardiac diagnosis. Since the frequency of ultrasonic waves is low, the resolution is low, and specialized knowledge and skilled techniques are indispensable for diagnosis, and a great deal of time and labor is spent on acquiring tomographic imaging techniques.

Therefore, as an ultrasonic diagnostic image extraction method, for example, the operator indicates the contour points on the contour of the target organ other than the both ends of the mitral valve portion and the mitral valve portion, finds the center point of the image, A method of setting a scanning line in a radial pattern across the contour of the target organ (Patent Document 1), a method in which an operator draws a separation line that separates a region of interest from an adjacent region, and accurately sets an extraction range of the region of interest; (Patent document 2), an operator adopts an optical flow method or a cross-correlation method from a plurality of designated lines that are obtained manually from the center of the contour of the target organ or that are radially extended from the center point. An extraction method is known, such as obtaining the contour of the image (Patent Document 3).

JP 7-255703 A 2002-330967 2005-137936

With the development of communication technology, it becomes an environment where a telemedicine system can be constructed, and a non-skilled operator operates the probe to obtain an organ tomographic image necessary for diagnosis in a medical institution where a specialist waits, Alternatively, there is a need for a method of rapidly extracting a long axis or short axis tomographic image of a heart, which is a reference image for diagnosis by scanning the body surface with a robot.

The present inventors have reduced the degree of dependence on human expertise through image processing and divided a heart image obtained by chronologically capturing an ultrasound image for automatically recognizing a target area, We considered a system in which the device automatically recognizes the part to which a region belongs from the time-series pattern of luminance average values.

The ultrasonic diagnostic apparatus of the present invention is at least the following steps (1) to (11):
(1) Scanning the probe and displaying on the display a moving image obtained by measuring the image of the left ventricle or left atrium including the mitral valve for a certain period of time at a frame rate selected from 5 to 30 frames per second. Subdividing the image into small regions,
(2) measuring a time-series change in luminance average value in each of the subdivided subregions, and measuring a rate of change in interframe difference;
(3) calculating and calculating a cross-correlation coefficient for a cardiac cycle between regions adjacent to each other above and below each region where the luminance average value is measured;
(4) providing a threshold value for the measured change rate, and extracting a region exceeding the threshold value in the measurement region;
(5) providing a threshold for the measured cross-correlation coefficient, and extracting a region exceeding the threshold in the measurement region;
(6) determining a region exceeding both of the two threshold values as a mitral valve, and displaying a coordinate position (x, y) on a display screen;
(7) drawing a polygon centered on the coordinates displayed on the display, and dividing the center coordinates into a plurality of triangular regions connecting each vertex of the polygon;
(8) In the triangular region, calculating a dot product of components in which a motion vector drawn in time series is directed toward the center of the region, and measuring a time-series change in contraction speed;
(9) measuring a time-series change of the contraction speed for a plurality of triangular regions selected from the triangles created by dividing the polygon, and comparing and storing the tendency;
(10) Rotate the probe at a fixed angle, draw a polygon identical to the polygon on the tomographic image for each angle, and divide it into triangular regions connecting the vertices, Measuring a time-series change of the contraction speed in a plurality of selected triangular regions, and comparing and storing the tendency;
(11) Rotating the probe at a constant angle, and measuring the rotation angle, the trend of the time-series change of the contraction speed in the selected triangular regions from the comparison / stored data is the most. Recognizing a tomographic image of an approximate rotation angle as a short axis tomographic image of a cardiac diagnostic reference, and setting the position of the probe as a reference point of the probe;
An ultrasonic diagnostic apparatus having a function of automatically recognizing a diagnostic reference tomographic plane of a heart including

The polygon described in the above (7) is a polygon selected from a triangle to a 32, and the rotation angle of the probe that rotates the probe at a constant rate is 10 ° to 90 °. The rotation angle selected from the range.

The plurality of triangular regions described in (10) are two or more regions.

As is apparent from the above description, the ultrasonic diagnostic apparatus of the present invention moves the patient's body surface by a technique or a robot using an ultrasonic diagnostic probe, inputs an image signal, and displays a display screen. When displayed above, the device checks the position of the mitral valve, operates the probe, moves the position of the mitral valve to the center of the display screen, and then rotates the probe at a fixed angle. By measuring motion vectors in a plurality of divided regions, a short-axis tomographic image serving as a heart diagnostic reference can be automatically selected, so that an early diagnosis can be supported by a specialist.

The sector type probe is used to obtain a diagnostic image of the heart because the size and position of the heart varies from person to person and the position of the ultrasound probe is limited by the sternum, ribs and lungs. In order to determine the position and angle, it requires special skill compared to other organs.

To make a diagnosis using an ultrasonic tomogram, the reference cross-section is determined for each organ. In the heart, the mitral valve is placed at the center of the tomogram, and a tomogram of a long axis image or a short axis image. If both are obtained, both become the diagnostic reference images of the heart. When the probe is rotated 90 ° from the position where the long-axis tomogram was obtained, a short-axis tomogram is obtained.

FIG. 1 shows a long-axis image (a) and a short-axis image (b), which are reference screens for ultrasonic tomographic image scanning scanned with a probe in cardiac diagnosis. The left is a schematic diagram of the heart, the position of the probe and the tomographic plane to be scanned, and the right is the corresponding ultrasonic tomographic image. In FIG. 1, the part surrounded by a dotted line is called the mitral valve (MV) and is the part that moves most rapidly in the heart. These tomographic planes pressed the probe perpendicular to the body surface. The mitral valve is observed as a vertical motion on the screen.

The present inventors used NTSC video signals from a diagnostic apparatus (Hitachi Medico EUB-565 and GE Healthcare Vivid 7) for a moving image of a heart as a coaxial cable and an analog image board (manufactured by Photoron Co., Ltd., image system, FLX-PCl ) On a PC (DELL precision670 Windows (registered trademark) XP, Intel XEON, CPU 3.60 GHz, 2.00 GB RAM) and saved in AVI format. Software that recognizes the position of the mitral valve in moving images and software that measures motion vectors makes use of the high-speed transfer capability of the PCI bus to transfer the ultrasonic moving image input image to the main memory of the PC in real time. Programming interface Using Microsoft DirectX, Visual C ++ was used to create an image processing program. Smoothing processing was performed in the 3 × 3 pixel region as a prefilter for noise removal.

When displayed on a normal VGA-compatible display, the image size is a display image of 640 × 480 pixels. On this screen, the size of each long-axis tomographic image of the average adult male is expressed in pixels (actual Table 1 shows the size expressed in mm).

Since the mitral valve performs the fastest contraction movement, it is presumed that the rate of change in luminance is large. First, as a method for determining the position of the mitral valve, a long-axis tomogram as shown in FIG. Is divided into regions of 40 × 20 pixels, and first, the change rate of the average luminance of the region is analyzed. The luminance change rate with respect to time in the divided small area is defined as in Expression 1. n represents the frame number, and Gx, y (n) represents the rate of change of the average luminance value Ix, y (n) in the y-th region with the horizontal axis x and the vertical axis. Gx, y (n) increases as the value Ix, y (n-1) immediately before Ix, y (n) is closer to 0.

From this point, by focusing on the maximum value of the luminance change rate G _{x, y} (n) for each region, it is possible to identify a region where the luminance average value I _{x, y} (n) increases rapidly. In other words, the region including the mitral valve (MV) is effective as a feature amount of the mitral valve because the high-luminance mitral valve crosses the region of the ventricular cavity (IVS) having low luminance.

FIG. 3 shows temporal changes in luminance average value in each region of the heart of a 23-year-old healthy person. Since the movement speed of the mitral valve region (MV) is faster than that of the posterior ventricular wall region (PW) and the ventricular cavity region (IVS), the time-dependent change in luminance average value of this region shows a sharp peak with its opening and closing. I understand that

Although the mitral valve region can be clearly distinguished by the rate of change in luminance, compared to the posterior ventricular region where the luminance value always changes, the difference from the ventricular septum region was not significant. Therefore, we introduced the cross-correlation coefficient described below to identify them. The cross-correlation coefficient represents the similarity in temporal change between two regions that are adjacent to each other in terms of position, and is defined as a function of the phase τ of both.

This value is 1 _if I _{x, y + 1} (n) is shifted in phase τ, 1 if it completely matches I _{x, y} (n), -1 if it matches by sign inversion, and similarity is observed. It is standardized to take a value of 0 when there is not. The correlation coefficient between the upper and lower adjacent regions within the range of motion of the mitral valve (MV) and the ventricular septum (IVS) is shown in FIG. 4 in the range of 1 ≦ τ ≦ 45 frames. The mitral valve region showed high values for each heartbeat period, but no high correlation was found in the ventricular septum. In the ventricular septum, the size of the small region is larger or similar to the range of motion, and the mitral valve moves across multiple regions, so it shows a high correlation between multiple regions. ing.

Moreover, as a result of analyzing the waveform between each region, as shown in FIG. 5, it was revealed that the region where the mitral valve is drawn shows a high correlation in the adjacent region. This is because the amplitude of the mitral valve is larger than the size of the region. Calculate the average luminance change rate of the region and the cross-correlation coefficient of the luminance above and below the region by time series, and set the threshold value for the calculated value from the subdivided region, and set the heart mitral valve The algorithm of the procedure of FIG.

As a method to confirm the validity of the algorithm, a white rod on the black background diagram is regarded as a mitral valve or ventricular wall, etc., and the state of moving in a fan shape with varying range and speed is recorded with a video camera and programmed The recognition rate was calculated from the ratio of the number of frames that correctly recognized the mitral valve coordinates. The results are shown in Table 2. It was found that it can be automatically recognized when the motion range and period of a simple vibrating object match that of the mitral valve.

Using this algorithm and applying it to a long-axis image of 10 healthy subjects, 1544 regions (of which mitral valve is 100 regions) for 2 seconds corresponding to about twice the cardiac cycle (of which mitral valve is 100 regions) Among them, we developed a program to superimpose markers on the coordinates that were recognized as mitral valve regions from the discriminated result.

FIG. 7 shows a time-series image for 8 seconds of coordinates recognized as a mitral valve when inputting a long-axis image of a 23-year-old healthy person. As is clear from this, the position of the probe was fixed, and it was confirmed that the place where the mark was drawn was almost included in the region where the mitral valve moved. In the long axis image, the mitral valve moves in a fan shape, so the region to be divided is horizontally long. However, in the short axis image, the mitral valve moves up and down, so the same algorithm was applied. FIG. 8 shows the distribution of the mitral valve coordinates recognized when inputting a short-axis image of a 22-year-old healthy person for 8 seconds. In this subject, markers were drawn in all frames in 80 frames, of which 78 frames correctly recognized the range of movement of the mitral valve, and the recognition level was 97.5%. Was a very high recognition rate.

FIG. 9 shows an example of erroneous recognition. This shows that, in 25 frames, the fast movement of the posterior ventricular wall was erroneously recognized by 3 frames. From the above, if the mitral valve is not included in the image, it can be taught to scan the nearby body surface where the heart is likely to be found until a cross-section including the mitral valve is found.

The method of obtaining organ motion vectors using an optical flow image processing method for ultrasonic tomograms of organs has been used in quantitative measurement of organ motions until now. It was entrusted to.

The optical flow is the velocity field at each point on the screen. In other words, this is a method of calculating a vector indicating in what direction a certain point in the image moves in what direction and how much distance. The block matching method and the gradient method are common, but compared to the block matching method, the gradient method is used, which does not require the corresponding point search and has a small calculation load. The gradient method is a method of estimating the motion parameters of a target from a moving image by spatiotemporal differentiation, inferring the motion of the target on the assumption that “the brightness of the point on the object does not change after movement”, and the point at time t ( If the luminance value of x, y) is E (x, y, t) and the object has moved by (Δx, Δy) after a minute time Δt, E (x, y, t) = E (x + Δx, y + Δy, t + Δt).

Here, if the right side is Taylor-expanded and the second and later are ignored,
Ex · u + Ey · v + Et = 0
Is established.
However, Ex = δE / δx, Ey = δE / δy, Et = δE / δT, u = dx / dt, v = dy / dt.
This is a constraint formula in the gradient method. While there are two parameters u and v, it is difficult to obtain a unique value with a solution using only one constraint equation. “The pixels of moving objects have similar velocities, and the entire image is slippery. Therefore, the optical flow is determined so that E is minimized by adding the condition that the image is spatially smooth, that is, the condition that the spatial change is minimized.

FIG. 10 is a diagram showing the optical flow distribution between adjacent frames for a long-axis image of the heart of a 23-year-old healthy person. The time phase was processed at 10 frames per second in the expansion phase. Compared with FIG. 1, it can be seen that the movement of the wall of the left ventricle (LV) is captured, and it can be seen that the motion vector is drawn in a direction perpendicular to the posterior ventricular wall (PW). The flow of the mitral valve (MV), which has a larger motion vector compared to the wall (PW), is not drawn a long vector even though its motion speed is faster than that of the posterior ventricular wall. , Drawn shorter than its base. This is because the speed of the mitral valve is faster and the movement speed is faster than that, and the mitral valve itself is a thin tissue, and its surroundings are filled with blood that is not reflected as brightness, so the brightness value suddenly increases. This is the same tendency in the short-axis image.

The luminance distribution on A-P in FIG. 10 is shown in FIG. It represents that the line moved from a broken line to a solid line during 100 milliseconds. Usually, the frame rate of a tomographic image by video signal input is 30 frames per second. However, in order to secure the time for performing the above-described image processing operation in real time using a general-purpose PC, it is necessary to provide an acquisition frame interval. . The maximum velocity of the left ventricular posterior wall and mitral valve is said to be 100 to 200 mm per second. When the frame rate is increased to about 10 frames per second, the mitral valve is completely separated between the frames in this way, and it can be seen that there is a limit to the detection of the movement amount by the optical flow.

The reason why the flow cannot be obtained in the vicinity of the mitral valve that performs intense exercise is that the frame time interval is as long as 100 ms compared to the movement speed. Therefore, the brightness of the point on the object is necessary even after the movement. This is because the condition “does not change” is not satisfied. Although the frame time interval can be shortened to 33 ms, the thickness of the mitral valve may be only a few pixels in the tomographic image, and this condition may not be satisfied even at the upper limit of the frame rate. As a result, the number of frames to be imaged per second is required to be at least about 5 frames, the upper limit is about 30 frames, and considering the speed of the heart, about 5 to 15 frames is considered preferable.

Extracting the mitral valve for effective diagnosis support that can shorten the diagnosis time by acquiring the heart image by combining the motion vector detection of the ventricular wall by optical flow and the position recognition algorithm of the mitral valve. Gives the probe position, angle, and direction instructions to move the valve to the middle of the display. This instruction is used when the probe is operated manually, and the direction of movement, tilt, and rotation of the probe is instructed by voice, or if the robot is scanning the probe, The robot is instructed to scan the probe so that the mitral valve coordinate position is located at the center of the display.

Since the long-axis tomogram and short-axis tomogram of the heart are the reference tomograms at the time of cardiac diagnosis, the device recognizes the short-axis tomogram or long-axis tomogram in a short time and supports the display. It only needs to be displayed on the screen. The present inventors pay attention to the motion direction of the posterior ventricular wall, and in the case of a short-axis image, the motion of the posterior ventricular wall moves concentrically around the mitral valve. If a tomographic image that performs this movement can be extracted from the image, this cross-section is a step surface of a short-axis image that becomes the reference image of the heart, and it is thought that it can be useful for early diagnosis.

Therefore, draw a polygon as large as possible in the display screen, centering on the mitral valve, divide the polygon by the dividing line connecting the center and each vertex of the polygon, and rotate the probe at a certain angle Then, in the triangular area obtained by dividing each tomogram, first, in all the coordinates where the motion vector is detected, the unit vector is calculated with the direction toward the mitral valve being positive, and the inner product with the motion vector is calculated. As a result, a velocity component having a positive contraction motion and a negative expansion motion is derived. This was averaged within each triangular region and defined as the contraction rate, and the change with time was examined. The measurement is performed by rotating the probe at a constant angle, measuring the contraction speed in a plurality of tomographic images, at least two or more triangular regions, and measuring each tomographic image rotated in order at a constant angle after the measurement. The same measurement was performed, and the time-series trends of the contraction speeds of multiple triangular regions in the measured tomograms were compared. The tomogram showing the most similar trend was that the contraction movement of the ventricular wall centered on the mitral valve. Since this is a short-axis tomographic image to be performed, if the position is determined as the reference position, the tomographic image at the position where the 90 ° probe is rotated from the angle can be determined as the long-axis image.

As the polygon, even in the case of the smallest triangle, the rotation angle is decreased and the probe is rotated at a constant angle, and the display image is measured in at least two areas set by the triangular area. It is possible by comparing the time-series tendency of motion vectors in a plurality of set areas at each angle, and in the case of a polygon with a large number of corners, if at least three triangular areas at each angle are measured and compared It is considered possible to determine the short axis image.

On the screen of the tomographic image centered on the mitral valve, an octagon is drawn with the mitral valve as the center, the apex and the center of the corner are connected, and the octagon is divided into eight triangular regions. Calculate The sampling rate was 10 frames per second, and the size of the region was variable with a maximum of 120 pixels on one side of the octagon, taking into account the observation of the short axis image on a 640 × 480 pixel screen. The names of the areas are N, NE, E, SE, S, SW, W, and NW in the clockwise direction from the top of the screen. FIG. 12 shows motion vectors when a 23-year-old healthy person inputs (a) a long-axis image and (b) a short-axis image. Here, in order to detect a concentric motion in the short-axis image, By using the inner product with the vector, only the component of the motion vector drawn in the region toward the center of the region was extracted. This leads to a velocity component with positive ventricular wall contraction and negative expansion. This was averaged in each region and defined as the contraction speed, and each time change was examined.

First, the long axis image is defined as the standard 0 °, and the cross section is changed while rotating the probe axis clockwise by 30 ° in the range of 0 to 150 ° on the body surface, and the flow in each region Was measured by 12 healthy subjects. As a result, the flow of the regions S, SE, SW included in the ventricular posterior wall perpendicular to the beam scanning direction in the short-axis image was similar in both amplitude and phase.

FIG. 13 shows the time change of the average contraction speed in the regions SE, S, and SW in the long axis image and the short axis image of the same subject. In the short-axis image, the region SE is almost synchronized, but in the long-axis image, the region SE is hardly depicted or the movement is small. Therefore, when the correlation of the contraction speed between SE-SW in each cross section was obtained, only the short axis showed a high value.

That is, in the short-axis image, the time change of the average contraction speed in the plurality of regions obtained by dividing the polygon is almost synchronized, so that the plurality of ultrasonic images obtained by rotating the probe and dividing the tomographic image The tomographic image in which the direction and velocity of the time series of the motion vector in the region are most synchronized may be determined as the short axis image, and if the tomographic image at this dividing line passing through the mitral valve is captured, cardiac diagnosis A short-axis tomographic image of the heart, which is a reference screen for the above, is obtained. The polygon is a triangle or more and up to 32, and considering the balance between speed and accuracy, it is about 6 to 12 squares, and the plurality of setting areas for measuring the motion vector are 2 areas or more and about 6 areas or less. It is suitable in terms of operability.

FIG. 14 is a control block diagram of the robot. The operator can control the position and posture of the probe while watching the display, and can control the patient so that excessive force is not applied to the patient. By performing this control, it is possible to automatically extract a reference diagnostic image of the heart for supporting the examination.

It is the figure which showed the external shape and cross section of the long-axis image (a) and short-axis image (b) used as a reference | standard image in the diagnosis of the heart. This is a screen obtained by dividing a long-axis image in ultrasonic diagnosis into an area of 40 × 20 pixels. It is the graph which displayed the time-sequential change of the brightness | luminance average value of each site | part of the heart in ultrasonic diagnosis. It is the figure which displayed the correlation coefficient between the adjacent area | regions in the upper and lower sides within the movement range of a mitral valve (MV) and a ventricular septum (IVS) by the graph. It is the figure which plotted the maximum value distribution of the brightness | luminance change rate of each site | part of the heart in a diagnostic ultrasound, and a cross correlation coefficient. It is a figure which shows the algorithm which judges the position of a mitral valve from the ultrasonic diagnostic image of a heart. Distribution of coordinates recognized as a mitral valve when inputting a long-axis image of a heart for ultrasonic diagnosis in 8 seconds. The position of the probe is fixed, and a marker is superimposed on the coordinates. The location where the marker is drawn indicates the area where the mitral valve has moved. Distribution of coordinates recognized as a mitral valve at the time of inputting a short axis image of a heart for ultrasonic diagnosis in 8 seconds. The position of the probe is fixed, and a marker is superimposed on the coordinates. The location where the marker is drawn indicates the area where the mitral valve has moved. It is a figure which shows the example which recognized the ventricular posterior wall as the mitral valve in the tomogram which does not include a mitral valve. It is the figure which calculated and displayed on the image the optical flow between adjacent frames about the long-axis image of the heart of a 23-year-old healthy person. It is a figure which shows the luminance distribution on the straight line AP described in FIG. A motion vector is drawn in each region divided into 8 parts by connecting the vertices of the octagon with the coordinates of the mitral valve at the center of (a) long-axis image and (b) short-axis image input of a 23-year-old healthy person It is a figure. It is the figure which showed the time change of the average contraction speed in the area | region S, SE, SW in (a) long-axis image of a 23-year-old healthy person, and (b) short-axis image. It is the figure which shows the control block of the robot.

Claims (3)

An ultrasound diagnostic apparatus, at least,
The probe is scanned, and a moving image obtained by measuring a captured image of the left ventricle or the left atrium including the mitral valve at a frame number selected from 5 to 30 frames per second is displayed on the display. Subdividing the image into small areas,
Measuring a time-series change in luminance average value in each of the subdivided subregions, and measuring a change rate of interframe difference;
Calculating and calculating a cross-correlation coefficient for a cardiac cycle between regions adjacent to each other above and below each region where the average brightness value was measured;
Providing a threshold value for the measured change rate, and extracting a region exceeding the threshold value in the measurement region;
Providing a threshold value for the measured cross-correlation coefficient, and extracting a region exceeding the threshold value in the measurement region;
Determining a region exceeding both of the two thresholds as a mitral valve, and displaying a coordinate position (x, y) on a display screen;
Drawing a polygon centered on the coordinates displayed on the display, and dividing the coordinates into a plurality of triangular regions connecting the coordinates and vertices of the polygon;
In the triangular region, calculating a dot product of components in which a motion vector drawn in time series is directed toward the center of the region, and measuring a time series change in contraction speed;
From the triangle created by dividing the polygon, for a plurality of selected triangle regions, measuring the time-series change of the contraction speed, comparing and storing the trend,
The probe is rotated at a constant angle, and the same polygon as the polygon is drawn on the tomographic image for each angle, divided into triangular regions connecting the vertices, and the selected plurality Measuring the time-series change of the contraction speed in the triangular region of, and comparing and storing the tendency,
Rotate the probe at a certain angle and measure the rotation angle at the same angle. Recognizing a tomographic image of an angle as a short-axis tomographic image of a cardiac diagnostic reference, and setting the position of the probe as a reference point of the probe;
An ultrasonic diagnostic apparatus having a function of automatically recognizing a diagnostic reference tomographic plane of a heart including The polygon is a polygon selected from a triangle to a 32-angle, and a rotation angle of a probe that rotates the probe at a certain rate is selected from a range of 10 ° to 120 °. The ultrasonic diagnostic apparatus according to claim 1, which is a rotation angle. The ultrasonic diagnostic apparatus according to claim 1, wherein the set plurality of triangular regions is two or more regions.