JPH0435718B2 - - Google Patents

Description

[Detailed Description of the Invention] (Industrial Application Field) This invention relates to an ultrasonic image processing method, in particular, a method for collating an ultrasonic image with a model of a target object even if the ultrasonic image is of low quality. The present invention relates to a comparison method that enables reliable comparison.

(Prior Art) In the technical field for obtaining optical images, X-ray CT images, and ultrasound images, various comparison and verification techniques have been proposed and developed.

The comparison and matching method that is commonly used in the past is, for example, by cutting out a portion that best expresses the characteristics of the video from a reference video (video serving as a model) (this clipped partial video is called a template video). ), a correlation function (or some similarity measure, etc.) with the input video is calculated, and the point where the value is maximum is matched.

FIG. 6 is a block diagram schematically showing a known device for explaining the conventional comparison and verification method. In the figure, reference numeral 10 denotes an ultrasonic image sensor that receives an ultrasonic scattering signal (input signal) from a target object, converts it into an electric signal, and sends it to an ultrasonic image data forming means 12. After the signal is digitally processed in the ultrasonic image data forming means 12, the obtained ultrasonic image data is sent to the preprocessing means 14, where image quality improvement processing such as noise removal and speckle smoothing is performed. . Thereafter, the ultrasound image data that has been subjected to image quality improvement processing is sent to the binarization processing circuit 16, where it is extracted into, for example, a contour image (line image) or a black and white pattern, and is supplied to the comparison and verification means 18. The comparison and verification means 18 is supplied with the other information to be compared and verified with the ultrasound image data, that is, model information regarding the target object, such as information on a cross-sectional line image.

This cross-sectional line image is formed as follows. First, what is the target object, what is its position,
Necessary information such as the posture and other conditions is input to the cross-sectional image data generating means 22 using, for example, a keyboard, a camera, or other suitable external information inputting means 20. Cross-sectional image data generation means 22
Based on these necessary information, cross-sectional image data corresponding to the ultrasound image data is formed and sent to the cross-sectional line image forming means 24. As this cross-sectional image data, for example, the shape, position, inclination, drawing range, and other data of the cross-sectional line image can be used. The cross-sectional line image forming means 24 generates a cross-sectional line image, for example, information on all or part of the outline of the cross-section, as line image information of the target object, and supplies it to the comparison and verification means 18.

As described above, this comparison and verification means 18 performs comparison and verification by matching the line image image with a simplified pattern of the ultrasound image, for example, as a line image, and displays the results on an appropriate display means 26, for example. The information is displayed on a CRT or other display means, a printout means, or the like, or it is sent to a required subsequent processing means (not shown) without such display. These ultrasonic image data forming means 12, preprocessing means 14, binarization processing circuit 16, comparison and verification means 18,
The respective processes of the cross-sectional image data generating means 22 and the cross-sectional line image forming means 24 are executed by computer processing, a dedicated signal processor, or any other suitable processing means.

Incidentally, the ultrasound image data used for such comparison and verification is basically formed based on the principle that will be briefly explained below.

These image data are formed by convolving the object to be measured with the beam characteristics of the ultrasonic image sensor 10. Now, suppose that the position of the object to be measured is expressed as u(r, θ) in a cylindrical coordinate system (r, θ), and the ultrasonic image sensor 10 is disposed at the origin of this coordinate. When the beam characteristic (generally power sensitivity characteristic) which is the receiving characteristic of the ultrasonic image sensor 10 is w (r, θ), the two-dimensional image v (r, θ) which is the image data to be observed is It is known that it is given by the following equation (1).

v(r, θ)=∫ ^+∞ _-∞ ∫ ^+∞ _-∞ w(rs, θ−ψ
)・u(s, ψ)dsdψ (1) Here, w(r, θ) can be considered a kind of point spread function.

When obtaining the image given by equation (1) using optical means such as a laser range finder, the spread function w(r, θ) can be almost regarded as a delta function, so equation (1) is The result of the integration is v
(r, θ) u(r, θ), and an image relatively close to the model u(r, θ) of the target object is obtained. Therefore, it is relatively easy to compare and match the obtained image v(r, θ) with the model u(r, θ) of the target object.

(Problem to be solved by the invention) However, the wavelength of ultrasonic waves is greater than the wavelength of light.
10 It is about ⁴ times longer and the beam width of ultrasound in the receiving area is usually wider than the beam width of light, so it is necessary to use ultrasound to obtain an image of the target object according to equation (1) above. When attempting to do so, the beam characteristics do not become a delta function with respect to r and θ, but become a spread function, and as a result, the two-dimensional image v(r, θ) generally becomes a blurred image. The resulting image is different from model u(r, θ). Therefore,
A problem with ultrasonic images is that it is difficult to compare and match the obtained image v(r, θ) with the target object model u(r, θ).

Furthermore, as mentioned above, ultrasound has a long wavelength;
Because the frequency band of the ultrasound signal is narrow, the resolution is poor, making it difficult to obtain a clear image, speckle noise appears, and smooth parts of the target object appear mirror-like. For this reason, there is a considerable difference in how the images appear between audio and optical images, and even if the comparison and matching technology for optical images is applied directly to ultrasound images, the correlation processing for matching will be difficult. Since the peak value of the correlation value is low, the matching position cannot be obtained with sufficient accuracy, resulting in a problem in that the incorrect position is matched.

This invention was made to solve these conventional problems, and therefore, the purpose of this invention is to solve problems such as low-resolution and unclear images such as audio images, speckle noise, and mirror effects. Therefore, it is an object of the present invention to provide a comparison and matching method that can reliably and accurately compare and match this video with a target object model even in the case of a video whose image quality has deteriorated.

(Means for Solving the Problem) In order to achieve this objective, according to the present invention, the cross-sectional line image is brought closer to the input image to perform comparison and verification, and the following measures are taken. . That is, a reference image is generated by performing spatial filtering processing on the cross-sectional line image using an audio visualization filter for reference image generation appropriate for this image.

Using this reference image as a template image for comparison and verification, comparison and verification are performed with an ultrasound image or with an image obtained by image processing the ultrasound image. This acoustic imaging filter is a spatial filter that simulates the ultrasonic image generation process, and has the role of creating an approximate image of what an object will look like when viewed using ultrasound from a cross-sectional line image.

In carrying out the present invention, the filter preferably has filter characteristics that approximate the spreading effect due to the beam direction characteristics of the ultrasonic image sensor and the differential characteristic effect in the distance direction to the cross-sectional line image. is preferable. and,
A large number of these filters may be formed in advance and stored in a readable memory, or they may be formed and prepared each time.

Furthermore, in one embodiment of the present invention, it is preferable to use a B-mode distance cross-sectional image as the ultrasound image described above, and to generate the reference image described above from the B-mode distance cross-sectional image.

Furthermore, in another embodiment of the present invention, a C-mode normal cross-sectional image is used as the ultrasound image, and a plurality of reference images of B-mode distance cross-sectional images are used to obtain the C-mode normal cross-sectional image.
It is preferable to generate a reference image corresponding to a mode normal cross-sectional image.

Furthermore, in another embodiment of the present invention, a B-mode distance cross-sectional image and a C-mode normal cross-sectional image are used as ultrasound images, and reference images corresponding to these B and C-mode cross-sectional images are combined for comparison and verification. is preferable.

(Operation) As described above, according to the comparison and matching method of the present invention, a spatial filtering process is performed on a cross-sectional line image representing contour information of a target object using an audio-visualization filter that simulates the audio-visual generation process. A reference image is generated as an audio image, and this reference image is compared with an ultrasound image.

Therefore, during matching, even if the peak value of similarity is high and the image is unclear or has degraded quality, it is possible to stably and accurately match the positions of the reference image and the ultrasound image.

(Example) Hereinafter, an example of the comparison and verification method of the present invention will be described with reference to the drawings.

It should be noted that the embodiments described below are merely preferred embodiments, and it should be understood that the present invention is not limited only to the embodiments described below.

FIG. 1 is a block diagram showing an example of the configuration of an ultrasonic image comparison and verification apparatus for explaining an embodiment of the comparison and verification method of the present invention. Furthermore, in this figure,
Components having the same functions as those of the conventional device shown in FIG. 6 are denoted by the same reference numerals, and detailed explanation thereof will be omitted. In FIG. 1, 30 is a reference image forming means provided between the cross-sectional line image forming means 24 and the comparing and matching means 18, and this reference image forming means 30 includes, for example, a spatial filter creating means 32, a spatial filter The ring means 34 is provided with necessary means such as a memory (not shown), and each processing of these means is also carried out by a computer,
It can be implemented by a dedicated signal processor or any other suitable processing means.

On the other hand, in the embodiment of the present invention, the ultrasound image data obtained by the ultrasound image data forming means 12 is sent to the preprocessing means 14, and the ultrasound image data subjected to image quality improvement processing is supplied to the comparing means 18. Alternatively, although not shown, the ultrasonic image data is directly supplied from the ultrasonic image data forming means 12 to the comparison and verification means 18.

First, the operation of an embodiment of the present invention will be explained.

According to this invention, information on the cross-sectional line image is supplied from the cross-sectional line image forming means 24 to the reference image forming means 30. This information is, for example, contour information regarding all or part of the cross-sectional contour, as in the conventional case.

This reference image forming means 30 performs spatial filtering processing on this contour information using an audio visualizing filter, that is, a spatial filter that simulates an appropriate audio image generation process of this information.
A reference video is generated by this spatial filtering process, and this reference video is output to the comparison/verification means 18.

Next, an example of the operation of this reference image forming means will be explained with reference to FIGS. 2 to 4.

First, one embodiment of the formation of this audiovisual filter will be described.

In forming this audiovisual filter,
Consider realizing a spreading effect and a differential effect in the distance direction based on the beam characteristics of the ultrasonic image sensor 10. Now, the number of elements of the wave receiving sensor is n = 64.
and the pitch interval d between elements is d = 5.4 mm,
Assume that the ultrasonic transmission frequency f is 384 KHz.
Assuming that the sound speed c is c=1500 m/s, if there is no shedding, the power sensitivity distribution (power sensitivity azimuth characteristic) of the beam of this ultrasonic image sensor 10 w
As is well known, (r, θ) is given by the following equation (2).
Note that the power sensitivity distribution w(r, θ) in equation (2) is given by an approximate expression w(θ) assuming that the power sensitivity distribution is constant with respect to the distance r in the cylindrical coordinate system.

w(θ)= |sin{nπd・sinθ/(c/f)}/n・sin{πd・si
nθ/(c/f)} | ² (2) From this equation (2), at a position distant from the ultrasonic image sensor 10 by, for example, a distance of 7.68 m, this equation (2)
In the power sensitivity distribution given by , the total width at the position where the power sensitivity is halved (half the peak value) is approximately 9.4 cm when converted to length. Therefore, for example, in the case of an image with one pixel of 3 cm, by simulating the pixel spread effect with a 3×3 spatial filter, it is possible to approximately incorporate the spread effect and the differential characteristics in the distance direction into the cross-sectional line image.

Next, a general example of the procedure for configuring this audio-visualization (spatial) filter when the power sensitivity distribution is given by equation (2) will be explained.

FIG. 2 is a flowchart of operations for explaining an embodiment of the present invention. Incidentally, in FIG. 2, a processing step is indicated by S. FIGS. 3A and 3B are explanatory diagrams for explaining the creation of a spatial filter according to the embodiment of this invention,
The power sensitivity distribution is shown by plotting the converted distance of θ on the horizontal axis and the power sensitivity distribution w(θ) on the vertical axis.

First, find the power sensitivity distribution of the ultrasonic image sensor (S1). In this embodiment, this power sensitivity distribution is based on the above-mentioned information n, d,
Determine from c, f, and θ.

Next, assuming that up to 1/K of the power sensitivity peak value H affects the spread of the image, the total width that takes a value of 1/K or more of the power sensitivity peak value H is determined (S2).

Next, this total width is equally divided into N segments (S3).

Next, for each segment, the average height from the zero level, that is, the N segment power sensitivity P ₁ ~
Find P _N for each (S4). All or part of these segment power sensitivities P ₁ to P _N are discrete values.

Appropriate power sensitivity ratios are calculated from the values of these segment power sensitivities P ₁ to P _N to determine weighting coefficients W ₁ to W _N for the spread effect and differential effect (S5). In this case, the weighting coefficient may be obtained by multiplying the aforementioned ratio value by an appropriate coefficient, or the ratio value itself may be used as the weighting coefficient.

Next, the obtained weighting coefficients W ₁ to _{W N} are subjected to matrix arrangement processing to form an audiovisual filter (S6).

Next, for example, when simulating the pixel spread effect with a 3×3 spatial filter as described above,
After determining the power sensitivity distribution (FIG. 3B) in step S1 described above, the full width at half reduction is determined in step S2. For this purpose, for example, the power sensitivity peak value H and its half value (1/2)H are calculated, and then the full width that takes a value equal to or greater than this value (1/2)H is calculated.

Next, in step S3, this total width is equally divided into three segments, and then, in step S4, the segment power sensitivity is calculated for each segment.
Calculate P ₁ , P ₂ and P ₃ respectively.

Next, in step S5, the power ratio values of these segment power sensitivities P ₁ , P ₂ , and P ₃ are calculated, and from these values, a weighting coefficient representing the spread effect of the beam characteristics of the ultrasonic image sensor 10, i.e. , calculate the horizontal weighting coefficients W ₁ , W ₂ , W ₃ of the spatial filter, and then calculate the vertical weighting coefficients R ₁ , R ₂ , R ₃ of the filter representing the differential effect in the distance direction. . In this example, W ₁ :W ₂ :W ₃ =1:2:
1, and R ₁ :R ₂ :R ₃ =-1:2:-1, but they are not limited to these values in any way.

Then, in step S6, the weighting coefficients are arranged in a 3.times.3 matrix to form a spatial filter.

FIG. 4 shows an example of the configuration of the audio-visualization filter configured in this manner. In Figure 4, the row (horizontal) direction of the filters shows spread in the azimuth direction,
The row (vertical) direction shows differential characteristics in the distance direction. The spatial filter creation means 32 includes the above-mentioned arithmetic processing means and other necessary means, such as a memory. Moreover, instead of incorporating such a spatial filter creation means 32 and creating a spatial filter each time, a large number of spatial filters can be created in advance and stored in a readable memory, and this memory can be used to create a spatial filter according to external information. The configuration may be such that an appropriate spatial filter can be called.

Next, in the spatial filtering means 34,
The audiovisual filter obtained in this way is
The cross-sectional line image sent from the cross-sectional line image forming means 24 is convolved in the image area to form and output a reference image. Note that this convolution process can be performed using conventional techniques.

Next, the comparison/verification means 18 uses this reference image to compare and verify the position with the ultrasonic image. In this comparison and matching, a portion that sufficiently expresses the characteristics of the target object is cut out from the reference image and used as a template, the cross-correlation between this template image and the ultrasound image is determined, and the position where this value is maximized is set as the matching position. or any other suitable conventional method.

In the above-mentioned embodiments, the ultrasonic image mode was not particularly explained, but as is well known,
There are B-mode and C-mode cross-sectional images.

As schematically shown in FIG. 5A, this B mode is a distance cross-sectional image (horizontal cross-sectional image) obtained by transmitting an ultrasonic beam in a fan shape in a horizontal plane from an ultrasonic image sensor to a target object 40. 44, and this fan-shaped beam is called a transmission fan beam. In the illustrated example, the target object is shown as having a cylindrical shape, but this shape may be any other suitable shape.

As shown in Figure 5B, the C mode is an equal distance △c (△ is time, c is the speed of sound) from the center obtained by dividing this transmitting fan beam vertically into many parts and swinging it. This is a cross-sectional image on the distance plane E. Therefore, in one embodiment of the present invention, the ultrasound image is
It is preferable to use the B-mode distance cross-sectional image as a B-mode distance cross-sectional image, and to form the reference image described above from this B-mode distance cross-sectional image. By comparing and matching in B mode, the direction and position of the object on the distance cross section (for example, the two-dimensional position in the horizontal plane) can be determined.

Alternatively, it is preferable to use a C-mode normal cross-sectional image as the ultrasound image and use a plurality of reference images of B-mode distance cross-sectional images to form a reference image corresponding to this C-mode normal cross-sectional image. In order to generate a reference image for this C-mode image, for example, it is necessary to generate a reference image corresponding to a B-mode cross-sectional line image (B-mode ) Reference images (representatively shown as 46 in FIG. 5C) are stored in an appropriate memory, and these (B-mode) reference images 46 are arranged three-dimensionally and a cross section is cut along an equidistant plane. The image is used as a C-mode reference image. According to this comparison and verification in the C mode, the position within a plane with a fixed distance (for example, in which direction, left or right, and how far up or down) can be determined.

Alternatively, using a B-mode distance cross-sectional image and a C-mode normal cross-sectional image as ultrasound images, these B and C
It is preferable to use a combination of a reference image corresponding to a cross-sectional image of the mode for comparison and verification. As mentioned above, when a reference image corresponding to B-mode and C-mode cross-sectional images is required (that is, when a model image for comparison has been obtained), only B-mode or C-mode only cross-sectional images are required. Highly accurate matching can be expected by comparing and matching by using reference videos of both modes simultaneously or time-sequentially and alternately, rather than performing matching using only videos. When used simultaneously, for example, when the similarity between B mode and C mode is R _b and R _c during comparison and matching, α is a real number such that 0<α<1, and R=αR _b + A new degree of similarity called (1-α)R _c is defined. The position of the B-mode image and C-mode image where this value is maximum indicates the position of the object to be dealt with. As already explained, B mode is 2 when the top and bottom of the beam are fixed.
Since the dimensional image and the C mode are two-dimensional images when the distance is fixed, by using both, it is possible to perform highly stable matching that combines various features of the target object. Furthermore, the three-dimensional position of the object can also be verified.

The mode for creating these reference images is set by the external information input means 20, and the corresponding B-mode distance cross-sectional image is generated from the cross-sectional image data generating means 22. When using C mode, or when using B mode and C mode, the cross sectional image data generating means 22 should be able to process a large number of B mode cross sectional images at the same time, and the reference image forming means 30 should be able to process a large number of B mode cross sectional images at the same time. It can also be configured to allow parallel processing. In this case, of course, the cross-sectional image data generating means 22 and the reference image forming means 30 process the B and C mode cross-sectional images one by one, and each reference image obtained through processing is temporarily stored in the memory, and then It is also possible to configure the data to be simultaneously read from the memory and sent to the comparison/verification means 18.

It is clear that the invention is not limited only to the embodiments described above, but can be subjected to many variations and modifications. For example, the spatial filter creation means can be created using any suitable procedure other than the method described above. Also, as a spatial filter, 3×3
Although a matrix filter has been described, the invention is not limited to this, and if necessary, it can generally be formed as a matrix filter with N rows and M columns (N=M included).

(Effects of the Invention) As is clear from the above description, according to the comparison method of the present invention, a reference image obtained using an audio-visualization filter is used as an audio-visual model, and a comparison with an ultrasound image is performed. Since this method performs matching, it is possible to accurately compare and match ultrasound images that look different from optical images, are unclear, or have degraded image quality.

Therefore, the present invention is suitable for application to a position identification system, a recognition system, or any other suitable ultrasound image processing device using ultrasound images.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing an example of the configuration of an ultrasonic image comparison and verification device to explain an embodiment of the comparison and verification method of the present invention, and FIG. 2 is a flowchart of an operation to explain the embodiment of the invention. FIG. 3 is an explanatory diagram for explaining the creation of a spatial filter according to an embodiment of the present invention, FIG. 4 is a diagram showing an example of an audiovisual filter used in the comparison and matching method of the present invention, and FIG. 5 is an explanatory diagram for explaining the present invention. An explanatory diagram of the ultrasound image mode used for
FIG. 6 is a block diagram showing an example of the configuration of an apparatus for explaining a conventional comparison and verification method. 10... Ultrasonic image sensor, 12... Ultrasonic image data forming means, 14... Pre-processing means, 16... Binarization processing circuit, 18... Comparison/verification means, 20... External information input means, 22... Cross-sectional image data generation means , 24...
Cross-sectional line image forming means, 26... Display means, 30... Reference image forming means, 32... Spatial filter creating means,
34...Spatial filtering means, 40...Target object, 42...Transmission fan beam, 44...Horizontal sectional image, 46...(B mode) reference image.

Claims (1)

[Claims] 1. Ultrasonic image data obtained by receiving an ultrasonic scattering signal from a target object with an ultrasonic image sensor;
In order to compare and match the position of the ultrasound image and the target object based on the cross-sectional line image obtained from the position, posture data, etc. regarding the target object, an ultrasound image generation process is performed on the cross-sectional line image. A reference image is generated by performing spatial filtering processing using an audio imaging filter that simulates A comparison and verification method characterized by performing comparison and verification with the reference video. 2. A patent characterized in that the acoustic imaging filter has filter characteristics that approximate the spread effect due to the beam azimuth characteristics of the ultrasonic image sensor and the differential characteristic effect in the distance direction to the cross-sectional line image. A comparative verification method according to claim 1. 3. The comparison and verification method according to claim 1 or 2, characterized in that a B-mode distance cross-sectional image is used as the ultrasound image, and the reference image is generated from the B-mode distance cross-sectional line image. . 4. A C-mode normal cross-sectional image is used as the ultrasound image, and a plurality of reference images of B-mode distance cross-sectional images are used to generate a reference image corresponding to the C-mode normal cross-sectional image. Comparison and verification method described in scope 1 or 2. 5. Claim 5, characterized in that a B-mode range cross-sectional image and a C-mode normal cross-sectional image are used as the ultrasound images, and reference images corresponding to these B and C-mode cross-sectional images are combined to perform comparison and verification. 1
or the comparative verification method described in paragraph 2.