JPH03251713A - Ultrasonic imaging system for three-dimensional object - Google Patents

Abstract

PURPOSE:To achieve a high precision in the reproduction of even an image of an unknown object by allowing a neural network to learn a general rule of the reproduction of images when an object with a three-dimensional shape is irradiated with an ultrasonic wave to perform an identification of the object, an estimation of a position and an angle of rotation and further, a reproduction of an image from scattered waves thereof. CONSTITUTION:An object recognition equipment is made up of an object 1, an ultrasonic transmitter 2, an A/D converter 3, an ultrasonic receiver 4 comprising a receiver array of nXn and an amplification/noise removing filter and a computer 5 and the object 1 is irradiated with a burst wave from the transmitter 2. Then, a sound pressure of an ultrasonic wave measured with the receiver 4 is brought into the computer 5 through the converter 3 and a waveform thereof is cut in a time phase to obtain information pertaining to the depth of the object 1 while XY direction information is determined by an ordinary acoustic hologram method. Thereafter, a neural network is made to learn the information to obtain an outline of the object of (n)X(n) or (n)X(n)X(m).

Description

[Detailed Description of the Invention] The present invention relates to an ultrasonic three-dimensional object imaging method, for example,
It is applied to automatic recognition devices for three-dimensional objects, robot eyes, FA (factory automation), three-dimensional copiers, three-dimensional facsimiles, and the like.

1. Many studies have been conducted on automatic recognition technology for objects having three-dimensional shapes, mainly using TV cameras, for the purpose of use in FA (factory automation). However, methods that handle three-dimensional information about objects have not yet been fully put into practical use, and the method using the TV camera described above requires a huge amount of input data, and objects that specularly reflect light, such as metals, cannot be used. The disadvantage is that it cannot handle transparent objects. Furthermore, as one of the essential conditions for realizing intelligent robots, it is desired to have robot eyes that can identify and measure target objects and recognize the external world. As methods for recognizing the external world, various methods using wave media such as light waves, X-rays, electromagnetic waves, and sound waves have been proposed, but no definitive method has yet emerged.

On the other hand, with methods that use ultrasound, although the outline of the target object can be measured, it is difficult to identify the target object, estimate its position and rotation angle, or construct a high-definition image of the target object from that information. I couldn't.

For example, there are methods that target flat objects, but when the target object has a three-dimensional shape, conventional technology can identify the target object, estimate the position and rotation angle, and perform high-definition processing of the target object. It was not possible to construct a perfect image.

In addition, the advantages of using ultrasound for recognizing three-dimensional objects include:
The following three points can be mentioned. First of all, since the propagation speed of sound waves is slow, it is possible to measure temporal changes in the phase and sound pressure of scattered waves from an object, and as a result, it is possible to directly obtain three-dimensional information about the target object. I can do it. With methods that use light waves as a medium, such as methods that use television cameras, it is relatively easy to obtain a two-dimensional image of the target object, but accurate estimation of depth information requires image understanding skills. It is difficult to do so. For this reason, even robots made with a television camera as their main sensor are often equipped with ultrasonic sensors to measure the presence or absence of a target object and the distance to the target object. One example of research that applies this advantage of ultrasound is research into a system for guiding blind people.

Second, using ultrasound, you can clean metals and transparent objects.

It becomes possible to recognize black objects, fluids, etc. It is almost impossible to recognize these objects using a television camera due to the strength and weakness of light reflection, transparency, etc. In applications such as FA, there are cases where product parts are made of metal, and in such cases, ultrasonic methods are useful for recognizing parts.This is an application of the above-mentioned advantages.

There is research on using ultrasound images for non-destructive testing.

In addition, a system for identifying 150 types of empty bottles in supermarkets and the like, and an automatic system for checking the water level of soft drinks in bottles, the presence of straws on the side of paper cartons, etc., have been put into practical use.

Third, ultrasonic methods can be used in turbid water or in a dark room.

Object recognition becomes possible even in situations filled with smoke. There is research that takes advantage of this point by using ultrasonic waves in the eyes of robots that perform extreme work on the ocean floor or rescue lives at fire scenes.

On the other hand, object recognition methods using ultrasonic waves are not as good as methods using television cameras in terms of resolution due to the long wavelength of the ultrasonic waves and limitations on the size of the receiving array and the number of receivers. Because of this, it has not been fully put into practical use except for purposes such as bioinstrumentation and underwater use.

In order to recognize a three-dimensional object, there are five methods for measuring the three-dimensional shape of the object by irradiating the object with ultrasound and measuring the scattered waves.
LTRASONICROBOT EYE FOROBJ
ECT IDENTIFICATION USING
NEURAL NETIIIORKJ (S, Wata
nabe, M., Yoneyama, 1989 UL
TRASONIC5SYMPOO5IUM P, 108
3 to 1086), there is a method in which the obtained sound pressure data is separated in the X and Y directions using an acoustic holography method, and the Z direction is separated in time and then learned by a neural network. However, in this method, although it is possible to accurately image an object having a learned shape, it is difficult to image an object image based on unlearned data.

The present invention was made to solve the above-mentioned drawbacks, and involves irradiating an object having a three-dimensional shape with ultrasonic waves and identifying the object from the scattered waves.

To provide an ultrasonic three-dimensional object imaging method for estimating the position, rotation angle, and image reconstruction, and to provide not only the shape of the target object itself but also the local By also using features, the neural network learns general rules for image reproduction.

Ultrasound 3 that reproduces images of unknown objects in high definition
This method was developed with the aim of providing an original object imaging method.

Toyo-Kazui In order to achieve the above-mentioned object, the present invention provides (1) identification of a target object and estimation of its position and rotation angle by irradiating the target object with ultrasonic waves and measuring its scattered waves; In the ultrasonic three-dimensional object detection method that performs image reconstruction, burst waves are irradiated onto the target object from one or more transmitters, and the scattered waves are measured by multiple delivery devices, and acoustic holography is used to determine the XY force direction. (2) Reproducing the image by using the method and temporal separation in the Z direction, or (2) reproducing the image by irradiating the target object with ultrasound and measuring the scattered waves. In an ultrasonic three-dimensional object recognition method that identifies objects, estimates the position and rotation angle, and performs image reconstruction, one or more transmitters irradiate a target object with burst waves, and the scattered waves are transmitted to multiple Measured with a receiver, XY
The image of the force direction is reproduced using the acoustic holography method, and the image of the two directions is reproduced by temporal separation.By having a neural network learn the outline of the object obtained in this way, the target object can be reconstructed. (3) irradiate the target object with ultrasonic waves, measure the scattered waves with multiple receivers in an array, and calculate the The two directions are separated using acoustic holography, and the two directions are temporally separated to obtain a three-dimensional image of the target object using ultrasound, and then multiple image processing filters (convolution processing) are used to create a local image. The present invention is characterized in that a neural network is trained to calculate a plurality of feature quantities of the image and reconstruct the accurate shape of the object using the feature quantities of the image as input. Let's explain based on an example.

First, the present invention will be explained in detail.

Ultrasonic wave (burst wave) with a waveform as shown in Figure 2 is applied to the target object.
When identifying a target object with a three-dimensional shape, estimating its position and rotation angle, and constructing a high-definition image of the target object from the measured values of the scattered waves, first, the target object is identified. Reconstruct the outline. The method is to use the acoustic holography method described below for the XY force directions, and to use a temporal separation method for the Z direction, and then to have a neural network learn the outline of the object obtained in this way. This enables identification of a target object having a three-dimensional shape, estimation of the position and rotation angle, construction of a high-definition image of the target object, etc. FIG. 3 illustrates this.

Next, an embodiment (claim 1) of the ultrasonic three-dimensional object imaging method according to the present invention will be described based on FIG. 1 is a target object, 2 is an ultrasonic transmitter, 3 is an A/D converter, 4 is an ultrasonic receiver (nXn receiver array and amplification/noise removal filter), and 5 is a computer. A burst wave as shown in FIG. 2 is irradiated from the ultrasonic transmitter 2 toward the target object 1. In the area where waves exist in FIG. 2, the waveform is a sine wave (for example, 40 KHz). At this time, the sound pressure of the ultrasonic waves measured by the ultrasonic receiver 4 is passed through the A/D converter 3 to be input into the computer 5 and displayed as shown in FIG. 4. In calculator 5, this waveform is
Cut in the time direction as shown. The cutting interval can be set to every integer multiple of one cycle. At this time, by multiplying the propagation speed in the time direction (since it is the time of the round trip distance,
(When extracting distance information, multiply by 1/2) Information regarding the depth of the target object can be obtained. Next, regarding the image reconstruction method regarding the XY force directions, since the measured sound pressure can be regarded as a continuous sine wave in each cut section, a normal acoustic holography method can be used. When the ultrasonic receiver array is nXn and the number of divisions in the depth direction is m, the information obtained is nXnXm as shown in FIG.

Next, a first embodiment of claim 2 of the present invention will be described based on FIG. Reference numeral 21 represents the outline (nXnXm) of the target object obtained as described above, 22 represents a three-layer feedforward type neural network, and 23 represents the output results of the neural network.

A neural network for identifying K types of target objects is configured as follows. Since the outline 21 of the target object consists of nXnXm information, the input of the neural network 22 is also H.
Make it XHXm pieces. The number of units in the intermediate layer can be determined as appropriate. The number of units in the output layer is set to . The input data is the outline of the target object (HX n Xm
When the target object belongs to the first category, the training data is (1, 0, 0, ... O),
When it belongs to the second category, (0, 1, 0, 0, , 0
), 1. In this way, given a set of input data and corresponding output data, it is possible to construct a neural network that produces the desired output using the well-known back propagation method.

Next, a second embodiment of claim 2 of the present invention will be described. In the above, a neural network for identifying different types of target objects has been configured, but a neural network for estimating the position of a target object may be configured as follows. That is, the input, input layer, and middle layer of the neural network are the same as above, but the output layer is estimated in P stages in the X direction, Q stages in the Y direction, and R stages in the Z direction. Let the number of units in the output layer be (P+Q+R).

The learning data is configured as follows. That is, when the position of the target object belongs to the above stage (S, t, u), the Sth, (s+t)th, (s + t 10u)
) It is sufficient to set only the th unit to 1 and the rest to 0.

The learning method is the same as above.

Next, a third embodiment of claim 2 of the present invention will be described. A neural network for estimating the angle at which a target object is placed may be configured as follows.

That is, the input, input layer, and intermediate layer of the neural network are the same as above, but the output layer is assumed to be estimated in R stages in the angular direction, and the number of units in the output layer is R. The learning data is configured as follows. That is, when the position where the target object is placed belongs to the above-mentioned stage S, it is sufficient to set only the S-th unit to 1 and the rest to O, and the learning method is the same as above.

A fourth embodiment of claim 2 of the present invention will now be described with reference to FIG. 31 is the outline of the target object obtained as described above (nXnXm), 32 is a three-layer feed forward type neural network, and 33 is an input layer unit of the neural network, which represents the output result of the neural network. The number is bXbXm (where b < = n
), the number of units in the middle layer can be selected as appropriate.

The number of units in the output layer can be selected to the desired display size, for example cXcXd. The learning data is (
cX (n-b) ) X (cX (n-b) )
Configure to a size of Xd (using the precise shape of the target object). The neural network according to the embodiment of the present invention is characterized in that it learns and outputs while scanning input and output, respectively.

FIG. 9 is a block diagram for explaining another embodiment of the three-dimensional object imaging method according to the present invention. array).

Here, the coordinates of the receiver are represented by (x+y+u), and the coordinates of the target object are represented by (xl, yl, zl).

Assuming that the reflection coefficient of the target object is ξ(x' * y') and the surface equation is z'=ζ(x' + y'), the target object 12 is The wave number vector k in= (k
sinθ, 0. −k cos θ) at time t=
When irradiating from zero (θ is the irradiation angle), the sound pressure Pin(r, t) of the incident ultrasound is Pin(r, t)”θ(ωt-kin(r-r,) ex
It is given by p(jkin·(r−rO)−jωt).

Here, ω is the angular frequency, θ(x) is O when x<0,
Represents a function that becomes 1 when x≧0.

Based on the above settings, the theoretical value P(r, t) of the sound pressure measured at time t by the receiver 13 placed at position r is P(r, t
)=jexp (jkr)F (r)/ (4πr)
/ / dx'dy'exp(jV-r')・ξ
(X'ty')e(Ir'-r61÷lr'-rl)・
...(1). Here, for simplicity, the notation r = r V = (Vx, Vy, Vz) Vx = -k (x/r - sin θ) Vy = -k (y/r) Vz = -k (z/r + cos θ) rl = (xl +y', ζ(x', y'))F(r
)=IVI”/Vz. By performing mathematical transformation from this equation, the following equation can be derived.

ξ(x'+y')θ(cT+ (1+cos O)ζ(
x'ty' ))exp(-jk(1+cos O)
ζ (X'ty) = (kz)2/ πexp (-jkx
'sinθ) f fdxdyP(r, T+(r+r,
)/c)exp(jk(xx'+yy')/r)
- (2) Here, C represents the speed of sound, and T represents the time parameter. Using this equation, the shape of the target object can be calculated from the sound pressure measured by the receiver array 13.

The three-dimensional image of the target object obtained by this method (the value on the left side of the above equation) is a value A (X' +
Y' + Z''), and theoretically, it becomes.

For simplicity, the image obtained by taking the absolute value of A(x'ey' + Z') is again expressed as A(X' my
'#Z'').

Image A(x'
*y'', 2''), for example, the following method can be considered as a method for capturing an accurate object image.

First, image processing is performed on the obtained image to calculate a plurality of images (n images) consisting of local image feature amounts. (Bl
(X'+y'+Z') s 1=Q, l, 2. −・
, n).

The calculation method for image B+(X'+yZZ') is convolution processing, Bt(x'ty'+z')=ΣΣΣ h, (x' −
x, y″−y+Z’ Z) A(x+y+z
) y z . Here, hl (xlytz) can be, for example, an identity filter (in this case, the obtained image matches the input image), a smoothing filter, a Laplacian, or a filter that extracts only 2 = constant planes. It is also possible to use a filter that extracts only planes facing in a specific direction.

Next, a method for reconstructing the precise shape of the target object from the n feature images obtained in this way will be described. A multilayered neural network as shown in FIG. 10 is used to reconstruct a three-dimensional image. The number of input layers is n, the number of outputs is 1, and the number of intermediate layers is variable. The image obtained by the neural network is C(x'
(at(x'+y't
z') ;x=0. ”'y n-1) is used.

The training data is C(x' ty' tz').

Given input data and desired training data for it, a neural network of the type shown in Figure 1O can be trained by a well-known method, ie, pack propagation. Note that one network estimates only one point in the obtained image, so the neural network estimates each point by
It will be used while moving and scanning x', y', and Z'. In this way, the obtained neural network is based on the original image A (X'*y'tZ') (=B
a (X'tY'tZ')), but also learns how to accurately reconstruct the target object from the local features of the image, so even when an unknown object is given as input, , it is possible to construct a high-definition image. It goes without saying that the present invention is also applicable to the neural networks shown in FIGS. 7 and 8 described above.

FIG. 11 is a flowchart for explaining the three-dimensional object imaging method shown in FIG. Below, each step will be explained in order.

Furrow ↓: First, measure the sound pressure P (r, t).

Next, we used the audiovisual method to create a three-dimensional image A.
Find (x'my'tz' ).

Furrow master: Next, convolution calculation is performed using multiple image processing filters, and multiple feature quantities B+ (X'+Y''+z') (j"oe1+"・n
).

Morphism fPA-: Next, a neural network is applied to obtain a reconstructed image C(X'*y'tZ').

In this way, a high-definition image can be reconstructed by using local feature amounts of the target object.

Effects As is clear from the above explanation, according to the present invention, it is possible to identify an object having a three-dimensional shape, estimate the position and rotation angle, and reproduce the image. In particular, it becomes possible to recognize metals, transparent objects, etc., which were previously impossible to recognize with a TV camera. As a result, significant contributions can be expected in FA.

In addition, it is possible to capture high-definition images of three-dimensional objects that have never been studied using ultrasound, and as a result,
Significant improvements in performance can be achieved in three-dimensional object recognition using ultrasound.

[Brief explanation of drawings]

FIG. 1 is a diagram for explaining an embodiment of the ultrasonic three-dimensional object imaging method according to the present invention (Claim 1), and FIGS. 2 and 3 are diagrams for explaining the present invention in detail. , Figure 4 is
Figure 5 shows the measured ultrasound sound pressure. Figure 5 is a diagram obtained by cutting the waveform shown in Figure 4 in the time direction. Figure 6 shows an example in which the ultrasound receiver array is nXn and the number of divisions in the depth direction is FIG. 7, which is a diagram showing information obtained when m pieces of information, is a diagram showing information obtained when
FIG. 8 is a diagram for explaining the fourth embodiment of claim 2 of the present invention, and FIG. 9 is a diagram for explaining the first embodiment of the present invention.
A diagram for explaining another embodiment of the present invention, FIG.
FIG. 11, which is a diagram showing the neural network, is a flowchart for explaining the three-dimensional object imaging method in FIG. 9. 1...Target object, 2...Ultrasonic transmitter, 3...A
/D converter, 4... Ultrasonic wave receiver (receiver array and amplification/noise removal filter), 5... Computer. Figure 1 Figure 2 Figure 4

Claims (1)

[Claims] 1. Ultrasonic three-dimensional object recognition method that identifies the target object, estimates its position and rotation angle, and reproduces the image by irradiating the target object with ultrasonic waves and measuring its scattered waves. , a burst wave is irradiated onto the target object from one or more transmitters, the scattered waves are measured by multiple receivers, images are reconstructed using acoustic holography in the X and Y directions, and images are reconstructed in the Z direction. This is an ultrasonic three-dimensional object imaging method characterized by reproducing images by temporal separation. 2. In the ultrasonic three-dimensional object recognition method, which identifies the target object, estimates its position and rotation angle, and reproduces the image by irradiating the target object with ultrasonic waves and measuring the scattered waves, one or more A burst wave is irradiated from a transmitter to the target object, and the scattered waves are measured by multiple receivers. Images are reproduced using acoustic holography in the X and Y directions, and temporal separation is performed in the Z direction. The image is reconstructed by performing the following steps, and by having a neural network learn the outline of the object obtained in this way, the object is identified, its position and rotation angle are estimated, and the image is reconstructed. Sonic three-dimensional object imaging method. 3. The target object is irradiated with ultrasonic waves, the scattered waves are measured by multiple receivers in an array, and the X and Y directions are separated by acoustic holography, and the Z direction is separated in time and the target object is detected. Obtain a three-dimensional image using ultrasonic waves, then use multiple image processing filters (convolution processing) to obtain multiple local image features, and use the image features as input to obtain an accurate image of the object. An ultrasonic three-dimensional object imaging method characterized in that shape reconstruction is performed by learning a neural network.