JPH07146366A - Device for detecting traveling information on object - Google Patents

Abstract

PURPOSE:To measure the position, traveling direction, traveling speed, etc., of a traveling object at a dead angle. CONSTITUTION:The position of a traveling object 13 is found in such a way that acoustic waves are radiated to a passage 24 by driving a loudspeaker 21 with a sound source signal x(k) and acoustic waves in the passage 25 are received with a microphone 22. Then an impulse response is estimated by inputting the sound receiving output y(k) of the microphone 22 and the sound source signal x(k) to an acoustic echo canceler 23 and the acoustic wave propagating time along the shortest route from the microphone 22 of the object 13 is found. Finally, the position of the object 13 is found from the propagating time.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device which is applied to, for example, surveillance of an intruder in a room and detects movement information such as a position, a moving direction, a moving speed, a presence direction of a moving object.

[0002]

2. Description of the Related Art Conventionally, a video camera, an optical sensor, an ultrasonic sensor, etc. have been used to monitor the movement of an object and the presence or absence of an intruder in a room. These are methods of detecting movement information of an object by detecting light or ultrasonic waves reflected by the target object. For example, as shown in FIG. 4, a monitoring device 12 such as a video camera, an optical sensor, or an ultrasonic sensor is provided in one corner of the upper part of the room 11, and when an object 13 enters the field of view of the monitoring device 12, the object 13 The various movement information of the object 13 that has been detected can also be known.

[0003]

In the conventional device, the obstacle 14 exists between the monitoring device 12 and the object 13, and the blind spot of the monitoring device 12, that is, the place where the monitoring device 12 cannot directly see the object. The movement information of a certain object 13 cannot be detected at all.

[0004]

According to the present invention, impulse responses between two or more points regarding sound waves are continuously measured by the impulse response measuring means, and the movement information detecting means determines the difference in the measured impulse responses. In the, the time to propagate in the shortest path from the moving object to the impulse response measuring means, the propagation time, or the position of the moving object, the moving direction, the moving speed, the presence direction based on the propagation time difference of a plurality of impulse responses. Movement information is estimated.

That is, according to the present invention, sound wave diffraction, scattering,
By utilizing the reflection phenomenon and the variation of the impulse response between two or more points, or the equivalent thereof, the above-mentioned problems of the prior art can be overcome, and an object in a blind spot such as a video camera can be overcome. It enables detection of movement information such as movement direction, position, and speed. It should be noted that the ultrasonic sensor of the prior art utilizes the straightness of the ultrasonic wave as in the principle of the optical sensor. However, the term "sound wave" in the present invention means a sound wave having an audible frequency at which diffraction, scattering and reflection phenomena can be fully utilized. Specifically, ultrasonic waves are several tens of kHz
It means a sound wave having a frequency of z or more, and the "sound wave" of the present invention means an audible sound wave having a frequency of ten and several kHz or less.

[0006]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The principle of the present invention will be described with reference to FIG.
In FIG. 1A, parts corresponding to those in FIG. 4 are designated by the same reference numerals. In the present invention, there is provided an impulse response measuring means 20 for measuring the impulse response between two points regarding the sound wave in the space to be monitored, in this example the room 11. That is, the speaker 21 provided on one side of the room 11 is driven by the sound source signal x (k), and sound waves are emitted from the speaker 21 to the room 11.
The sound wave emitted from the room 11 is picked up by the microphone 22 provided on the same side as the speaker 21, and the output y (k) and the sound source signal x (k) are supplied to the impulse response calculation unit 23. .

The sound wave radiated from the speaker 21 also propagates to the other side of the obstacle 14 (left side in the figure) due to a diffraction effect or the like. At this time, the sound waves reaching the object 13 are
It is affected by scattering and reflection. After that, the sound wave is reflected by the wall surface or the like on the side opposite to the speaker 21 (on the left side in the drawing) with respect to the obstacle 14, propagates again to the right side of the room 11, that is, the speaker 21 and the microphone 22 side, and is received by the microphone 22. . What is important here is that the position of the object 13 has a different effect on the sound wave by the object 13. Therefore, when the sound waves returning from the left side of the figure are continuously observed, the way the sound waves return when the object 13 moves differs. By utilizing this, it is possible to detect the movement information of the object. However, in reality, it is not possible to select and observe only the sound waves returning from the left side. Therefore, in the impulse response calculation unit 23, the speaker 21
To the microphone 22 to measure the acoustic impulse response. Specific means for measuring the impulse response is, for example, Nakamizo: “Signal analysis and system identification” by Corona (19
88) and the like.

Since the impulse response represents the acoustic transfer characteristics of the entire room 11, when the object 13 moves, it is assumed that the object 13 is on the other side of the obstacle 14 with respect to the speaker 21 and the microphone 22. Also, the impulse response is expected to change. The change in the impulse response is because the sound wave is no longer affected by the object at the place where the object was, and the sound wave is affected by the object at the place where the object is moved. Occurs. That is, the change in the impulse response due to the movement of the object is transmitted from the speaker 21 to the moving object 13
And reaches the microphone 22. Therefore, the change in the impulse response due to the movement of the object 13 is
It has a large amplitude after a time delay corresponding to the time required for a sound wave to propagate through the shortest path from the speaker 21 to the moving object 13 and from the moving object 13 to the microphone 22. This delay time will be referred to as "shortest arrival time" in this specification. FIG. 1B is a schematic diagram of changes in the impulse response due to the movement of the object 13 (differences in impulse responses measured at two different times), and the earliest time with a large change in this waveform is the shortest arrival time. .

FIG. 1C shows the moving object 1 from this shortest arrival time.
A simple example of estimating the position of 3 is shown. The speaker 21 and the microphone 22 are provided at one end of the L-shaped curved passage 25, and the moving object 13 is located at the curved end of the passage 25 with respect to the speaker 21. A dashed line with an arrow coming out of the speaker 21, folded back by the moving object 13 and reaching the microphone 22 indicates a sound wave propagation path. As shown in this figure, if the moving object 13 is sufficiently larger than the distance between the speaker 21 and the microphone 22, the distance from the speaker 21 and the microphone 22 to the moving object 13 is about half of the distance obtained by multiplying the shortest arrival time by the speed of sound. It is estimated to be. In FIG. 1C, since the speaker 21 and the microphone 22 are arranged at one end of the passage 25, the position of the moving object can be specified from one shortest arrival time.

When the speaker 21, the microphone 22 and the moving object 13 are arbitrarily arranged in three dimensions, the position or direction of the object can be estimated by using a plurality of microphones. The shortest arrival time differs for each microphone. For example, a microphone closer to a moving object has a shorter shortest arrival time than a microphone far from a moving object. The position and direction of a moving object can be estimated from such a difference in the shortest arrival time by a method known in the fields of sonar and radar. Further, by detecting the position of the moving object at time intervals and measuring the change in the position, the moving speed and moving direction of the moving object can be detected.

Even if the sonar's principle of applying a sound wave to an object and estimating the position of the object from the reflected wave is used as it is in an environment with a lot of reflected sound, such as in a room, it will be reflected in many other reflected sounds in the room. Since the reflected sound from the object is buried, it is difficult to estimate the position of the object. For example, in the impulse response in FIG. 1C, the reflected sound from the moving object is buried in many reflected sounds from the passage and the wall surface. However, in the present invention, it is possible to estimate the direction or the position of the moving object by utilizing the time change of the impulse response, paying attention to the fact that the reflected sound from the wall surface does not change with time.

As described above, according to the present invention, it is possible to detect the movement information of the object at the blind spot position of the monitoring device, which is impossible by the conventional method. FIG. 2A shows an acoustic echo canceller 51 as the impulse measuring means 20.
An example using is shown. As is well known, the acoustic echo canceller 51 is a device that prevents the sound sent from the speaker 21 from being received by the microphone 22 and causing problems such as howling. The operation of the acoustic echo canceller 51 is such that the impulse response between the speaker 21 and the microphone 22 is estimated by the impulse response estimation unit 53, and the filter having the estimated impulse response and the sound source signal x (k) of the speaker 21 are convoluted. The pseudo echo y (k) is synthesized by the convolution operation in the unit 52, and this is subtracted from the output y (k) of the microphone 22 to obtain the signal (acoustic echo) output from the speaker 21 and received by the microphone 22. ) Is to be erased. The estimation of the impulse response in the acoustic echo canceller 51 usually uses an adaptive algorithm. The adaptive algorithm is an algorithm that sequentially estimates an impulse response at each sample time using the digitized sound source signal x (k) and the residual echo e, and is a learning identification method, L
The MS method and the like are known as typical examples. For details of the acoustic echo canceller 51, see Japanese Patent Application No. 4-44649.
Details on "Echo canceller" etc.

2A, the sound source signal x (k) is input and the acoustic echo canceller 51 is operated. The sound source signal x (k) is preferably a stationary signal with a wide frequency band (for example, white noise or pseudo voice), but may be a signal such as a musical sound or a voice signal. After the operation of the acoustic echo canceller 51 has started, and after the time required for the adaptive algorithm to converge (usually about several seconds), it is considered that the impulse response estimation unit 53 has successfully estimated the impulse response. Then, the estimated value of the impulse response is updated every sample time. In other words, continuous impulse response measurements (estimations) are made. Then, the estimated impulse response is transferred to the object movement information detection unit 24. Object movement information detector 2
4, the impulse response h (k) = estimated at time k =
{H (k) 0, h (k) 1, h (k) 2, ...} and impulse response h (k
-M) and the differential waveform Δh (k, m).

Δh (k, m) = h (k) −h (km) (1) Next, the envelope of Δh (k, m) or the short-time power waveform is obtained. For example, using the waveform shown in FIG. 1B as Δh (k, m)
Then, the short-time power has the waveform shown in FIG. 2B. The time t0 when the envelope or the time power waveform first exceeds the predetermined threshold value TΔ is obtained. Then, the sound source (speaker 21) is obtained by multiplying the sound velocity c by this t0.
-Moving object 13-The acoustic propagation path length of the microphone 22 is obtained, and the position of the moving object 13 as shown in FIG. 1C is detected by the above-described principle.

On the other hand, the residual echo e obtained by the acoustic echo canceller 51 is used to detect the fluctuation amount of the impulse response.
It is also possible to use (k). The impulse response between the speaker 21 and the microphone 22 is usually the object 1
It changes rapidly with the movement of 3. However, the impulse response estimated by the acoustic echo canceller 51 cannot follow abrupt changes, resulting in a delay in estimation. In other words, immediately after the actual impulse response changes from ha to hb, the impulse response estimated value of the acoustic echo canceller 51 remains at the value of ha before the change. Therefore, the pseudo echo y '(k) and the acoustic echo y (k) are convolved with the input signal x (k), respectively, and y' (k) = x (k) * ha (2) y (k) = It is expressed as x (k) * hb (3). However, * represents a convolution operation. From this, the residual echo e (k) is expressed as e (k) = y (k) -y '(k) = x (k) * (hb-ha) (4). From the equation (4), it can be seen that the residual echo e (k) is an amount that reflects the amount of change (hb-ha) in the impulse response.

FIG. 3A shows a second embodiment of the present invention 1
For each speaker 21, two microphones 22a,
22b are arranged along the passage 25, and the sound source signal x (k) of the speaker 21 and the microphones 22a, 22 are provided.
The respective outputs y1 (k) and y2 (k) of b are supplied to the acoustic echo cancellers 51a and 51b, respectively. The moving object 13 is located in a curved and extended portion with respect to the passage in which the speaker 21 and the microphones 22a and 22b are arranged. The residual echoes e1 (k) and e2 (k) from the acoustic echo cancellers 51a and 51b are detected by the object movement information detector 24.
Entered in. In this embodiment, a procedure for estimating in which direction the moving object 13 exists by using the residual echo will be shown. The residual echoes e1 (k) and e2 (k) can be expressed as a signal obtained by convolving the variation amount of the impulse response and the sound source signal x (k) as in the equation (4) as in the following equation.

E1 (k) = x (k) * Δh1 (5) e2 (k) = x (k) * Δh2 (6) where Δh1 and Δh2 are between the speaker 21 and the microphones 22a and 22b, respectively. Means the amount of change in the impulse response of. As shown in FIG. 3B, the peaks k1 and k2 of the waveforms of the changes Δh1 and Δh2 of the impulse response have large peaks at the respective microphones 22a and 2a.
2b is the shortest arrival time, and the difference corresponds to the difference in both distances from the moving object 13 to the microphones 22a and 22b. Since the residual echoes e1 (k) and e2 (k) are convolved with the sound source signal x (k), it is not possible to directly know the difference in the distance, but for example, if the sound source signal x (k) is white noise. , Can be estimated by calculating the autocorrelation function φ (i) of e1 (k) and e2 (k) as in the following equation.

Φ (i) = Σ _m e1 (m) e2 (m + i) (7) The reason why the time difference, that is, the distance difference can be estimated from φ (i) will be briefly described below. φ (i) can be transformed as follows. _{φ (-i) = Σ m e1} (m) e2 (- (i-m)) = e1 (i) * e2 (-i) = x (i) * Δh1 (i) * x (-i) * Δh2 (-I) = x (i) * x (-i) * Δh1 (i) * Δh2 (-i) = σx'2δ (i) = σx'2Δh1 (i) * Δh2 (-i) (8) where , Σx′2 is the variance of x, δ is Kronecker δ, and δ (i) = 1 (i = 0) 0 (i ≠ 0) (9). That is, since the sound source signal x (k) is white noise, its autocorrelation is 1 when i = 0 and 0 when i ≠ 0. For simplification, Δh1 (i) and Δh2 (i) are approximated as Δh1 (i) = δ (i−k1) (10) Δh2 (i) = δ (i−k2) (11) with a single reflected sound. Then, φ (i) = σx′2δ (i− (k2-k1)) (12), and when φ (i) ≠ 0, i = k2-k1 and this k2-k1 is shown in FIG. Is positive. Therefore, since e1 (k) is ahead of e2 (k),
It is estimated that the moving object 13 is closer to the microphone 22a than the microphone 22b and is on the left side in the figure.

In the method of obtaining the shortest arrival time for each microphone, the calculation of the equation (1) needs to be performed in the object movement information detecting section 24, whereas this residual echo e
Since (k) is an amount that is always calculated when impulse response estimation is performed in the acoustic echo canceller, the method that uses the residual echo significantly reduces the amount of calculation. As described above, the shortest arrival time to the moving object is obtained from the change in one impulse response between one speaker and the microphone, and the position of the moving object can be known from this shortest arrival time. From the time change, whether the moving object 13 is approaching or moving away, that is, the moving direction and the moving speed can be obtained.

On each time axis, a plurality of impulse responses between one speaker and a plurality of microphones or a plurality of impulse responses between a plurality of speakers to which the same drive signal is supplied and one microphone. A plurality of shortest arrival times can be obtained from the changes, and the position of the moving object can be more accurately obtained from the intersection of the circles with the distances from each to the moving object. , You can know the moving speed. In any of the above cases, residual echo may be used to detect a change in impulse response.

Further, when a plurality of impulse responses are measured as described above, it is possible to know the propagation time difference relating to them, that is, the direction of existence of the moving object from the equation (12). The position of the moving object can be known from the intersection of the directions, and the moving direction and the moving speed can be known from the change in the position.

[0022]

In the detection of a moving object, it is impossible for a video camera to detect the moving information of an object located out of the field of view. However, the present invention utilizes the wave nature of sound to make a speaker and a microphone. Therefore, it is possible to detect the moving information such as the position, moving direction, moving speed object, etc. of a moving object in a place where the line of sight cannot be seen directly.

Further, since the movement is detected from the change in the impulse response, there is a feature that the sensitivity to the movement of the object is high.

[Brief description of drawings]

FIG. 1A is a diagram showing an example of the relationship between an impulse response measuring means for explaining the principle of the present invention, a moving object, and an obstacle, B is a diagram showing an example of changes in the impulse response, and C is this diagram. It is a figure which shows the Example of invention.

2A is a block diagram showing an embodiment of the present invention in which an acoustic echo canceller is used as an impulse response measuring means, FIG.
FIG. 6 is a diagram showing an example of short-time power of a change amount of impulse response.

FIG. 3A is a block diagram showing an embodiment of the present invention when two impulse response measurements are performed, and FIG. 3B is a diagram showing an example of changes in the two impulse responses.

FIG. 4 is a diagram showing a state where a conventional moving object cannot be monitored due to an obstacle.

 ─────────────────────────────────────────────────── ─── Continued Front Page (72) Inventor Yoichi Haneda 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nihon Telegraph and Telephone Corporation

Claims (3)

[Claims] 1. An impulse response measuring means for continuously measuring an impulse response between two or more points regarding sound waves, and a shortest path from a moving object to the impulse response measuring means based on the difference in the measured impulse responses. And a movement information detecting means for estimating movement information such as a position, a moving direction, a moving speed, and an existing direction of the moving object based on a propagation time or a propagation time difference regarding a plurality of impulse responses. An object movement information detection device characterized by: 2. The object movement information detecting device according to claim 1, wherein the impulse response means is an acoustic echo canceller. 3. The object movement information detection device according to claim 2, wherein the residual echo of the acoustic echo canceller is used as the difference in the impulse response.