JP2011163776A - Device for monitoring sound source - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily specify a sound source with regard to a sound source monitoring device for monitoring the sound source. <P>SOLUTION: This device for monitoring the sound source is equipped with: a glass monitor which is placed before a person's eye and displays an image superimposed on a field of vision seen through; a sound sensor which is placed in a fixed position to the glass monitor and which catches a sound emitted from inside the field of vision obtained by seeing through the glass monitor and generates a sound signal; a sound source identifying part that identifies the sound source in the field of vision, based on the sound signal; and a sound source display part that displays the image showing the sound source at a position located on the glass monitor and superimposed on the sound source on the field of vision obtained by seeing through the glass monitor. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a sound source monitoring apparatus for monitoring a sound source.

  In various fields, the sound source position is correctly identified, such as where the sound is generated. For example, when an abnormal sound is heard in various facilities and devices, it is required to identify the source of the abnormal sound and prevent a failure in advance. However, it is quite difficult to identify the source of sound by listening to it with the human ear. Depending on the frequency of the sound, even if the sound is heard, the direction of the sound source is often not well understood. In the case of ultrasonic waves, the sound itself cannot be heard. In addition, for example, depending on the equipment or device, there may be a scene in which the sound is heard only at the initial stage of the operation and cannot be heard when the equipment is warmed by the operation, and it is difficult to catch the sound.

  Although it is conceivable to construct a large-scale device for identifying a sound source to accurately capture the sound source, it is required to carry it easily and easily identify the sound source.

  Here, Patent Document 1 shows a head-up display device that converts voice data collected by a sound collecting device (microphone) into text data and displays it as characters, although it is not clear specifically, It is described that the generation position or direction of the sound source is displayed.

  Patent Document 2 discloses a technique for specifying the location of a sound source by beam forming calculation from signals from a plurality of sound sensors in a leakage monitoring device that monitors leakage in a nuclear power plant.

JP-A-10-123450 Japanese Patent Laid-Open No. 07-110393

  However, in the head-up display device of Patent Document 1, since there is no relationship between the field of view and the position / direction of the sound source, for example, it is possible to know whether or not the wearer is facing the speaker (sound source). Even if it can, it is difficult to accurately capture the sound source. Patent Document 2 does not show the field of view, and it is difficult to specify a place that relies only on sound.

  An object of this invention is to provide the sound source monitoring apparatus which can specify a sound source easily in view of the said situation.

Of the sound source monitoring devices of the present invention that achieve the above object, the first sound source monitoring device is:
A glass monitor that displays an image that is placed in front of a person and that overlaps the field of view,
A sound sensor that is placed at a fixed position relative to the glass monitor and generates a sound signal by capturing sound emitted from within the field of view obtained by seeing through the glass monitor;
A sound source identification unit for identifying a sound source in the field of view based on the sound signal,
A sound source display unit that displays an image representing a sound source is provided at a position on the glass monitor that overlaps the sound source on the field of view obtained by seeing through the glass monitor.

  The first sound source monitoring device of the present invention includes a glass monitor and a sound sensor, identifies a sound source based on a sound signal obtained by the sound sensor, and displays an image representing the identified sound source on the glass monitor. It is displayed at a position that overlaps the sound source on the field of view obtained by seeing through the glass monitor, and the image representing the sound source can be seen on the view that is seen through the glass monitor, so that the sound source can be easily known. it can.

Here, in the first sound source monitoring device,
When the sound source identification unit has a plurality of sound sources, the sound source is identified based on the sound signal,
The sound source display unit displays each image representing each of the plurality of sound sources at each position overlapping with each of the plurality of sound sources on a view obtained by seeing through the glass monitor on the glass monitor. Is preferred.

  By identifying and displaying a plurality of sound sources, even when there are a plurality of sound sources in the field of view, the plurality of sound sources can be known simultaneously.

The second sound source monitoring device of the sound source monitoring device of the present invention is:
A glass monitor that is placed at the eyes of a person and displays an image;
A camera that is placed at a fixed position with respect to the glass monitor and that captures a forward field of view facing a person and generates an image signal;
A sound sensor that generates a sound signal by capturing sound emitted from within the field of view captured by a camera placed at a fixed position with respect to the glass monitor;
A sound source identification unit for identifying a sound source in the field of view based on the sound signal,
An image display unit is provided on the glass monitor for displaying an image obtained by superimposing an image representing a sound source on a position overlapping with the sound source on the image based on the image signal.

  Although the glass monitor of the second sound source monitoring device of the present invention cannot directly visually recognize the field of view, the glass monitor is provided with the above-described camera and, like the first sound source monitoring device, on the field of view captured by the camera. An image representing the sound source can be seen and the sound source can be easily known.

Here, in the second sound source monitoring device,
When the sound source identification unit has a plurality of sound sources, the sound source is identified based on the sound signal,
The sound source display unit displays, on the glass monitor, an image obtained by superimposing images representing the plurality of sound sources on the image based on the image signal at positions overlapping the sound sources on the image. It is preferable.

  Also in the second sound source monitoring device, a plurality of sound sources can be known at the same time by identifying a plurality of sound sources and displaying an image in which each image representing each sound source is superimposed on an image based on the image signal.

In both the first sound source monitoring device and the second sound source monitoring device,
The sound sensor comprises a plurality of sound sensors placed at positions fixed to the glass monitor and separated from each other,
It is preferable that the sound source identification unit identifies a sound source by beam forming calculation based on a plurality of sound signals obtained by a plurality of sound sensors.

  When the beam forming calculation is employed, a plurality of sound sensors are required, but the sound source can be measured in a state where these sound sensors are fixed to the glass monitor. In addition, some sound sensors are extremely small, and there are few space restrictions.

  However, the sound source is identified by mechanically scanning a sound sensor with strong directivity or arranging sound sensors with strong directivity in each direction in the field of view without adopting beamforming calculation. May be.

  According to the sound source monitoring device of the present invention, it is possible to visually recognize a sound source in a direction facing a person by wearing a glass monitor, and the sound source can be easily identified.

1 is an external perspective view of a sound source monitoring device as one embodiment of the present invention. It is a figure which shows the state which mounted | wore with the sound source monitoring apparatus shown in FIG. It is a block diagram which shows the signal processing part of the sound source monitoring apparatus shown in FIG. 1, FIG. It is a principle explanatory view of beam forming. It is a principle explanatory view of beam forming. It is a figure which shows an example of the mark showing the visual field seen through the glass monitor shown in FIG. 1, FIG. 2, and the sound source position superimposed on the visual field. It is a figure which shows the mounting state of the sound source monitoring apparatus of 2nd Embodiment of this invention. It is the schematic diagram which showed the whole picture of the sound source monitoring apparatus of 2nd Embodiment shown in FIG.

  Embodiments of the present invention will be described below.

  FIG. 1 is an external perspective view of a sound source monitoring device as an embodiment of the present invention, and FIG. 2 is a diagram showing a state in which the sound source monitoring device shown in FIG. 1 is mounted.

  A sound source monitoring device 10 shown in FIG. 1 is used with a mounting portion 11 mounted on the head as in the case of headphones and a glass monitor 12 placed on the right eye tip. The glass monitor 12 is fixed to the mounting portion 11. However, it is desirable to provide a fine-tuning mechanism so that the glass monitor 12 is correctly positioned at the tip of the right eye with the mounting portion 11 mounted on the head. The glass monitor 12 has a structure capable of displaying an image and seeing the visual field through the glass monitor. A small sound sensor 13 is provided at each of the four corners of the glass monitor 12, and a sound emitted from the direction in which the wearer faces, that is, from within the field of view of the wearer is picked up.

  Furthermore, the sound source monitoring device 10 includes an operation unit 14, which displays a frequency selection dial 15 for selecting a sound frequency for sound source identification and a sound source to be displayed on the glass monitor 12. A density adjustment dial 16 for adjusting the density of the image is provided.

  FIG. 3 is a block diagram showing a signal processing unit of the sound source monitoring apparatus shown in FIGS. The signal processing unit 20 is an example of a sound source identification unit and a sound source display unit according to the present invention.

  The sound signals obtained by each of the four sound sensors 13 are input to the A / D converter 21 and converted into digital sound signals. The digital sound signal is input to the filter 22. The filter 22 is a band-pass filter that extracts a signal component in a frequency band suitable for the center frequency, with the frequency corresponding to the frequency selected by the frequency selection dial 15 as the center frequency. With this filter 22, it is possible to search for a sound source emitting sound in a frequency band desired by the wearer.

  The signal component extracted by the filter 22 (here, the extracted signal component is also referred to as a sound signal) is input to the beam forming unit 23. In this beam forming unit 23, beam forming calculation is performed, and the sound source position is identified. The beam forming calculation will be described later.

  The sound source position information obtained by the beam forming unit 23 is input to the image generation unit 24. The image generation unit 24 generates an image representing a sound source to be displayed on the glass monitor 12 and displays the image on the glass monitor 12. In generating the image of the sound source, the image generation unit 24 generates an image having a density corresponding to information from the density adjustment dial. Here, the image generation unit 24 may generate an image representing the sound pressure distribution over the entire surface of the glass monitor 12 in shades or colors, or only the point where the sound is specifically concentrated is spot-like. May be displayed. Further, when displaying in a spot manner, when there are a plurality of points where sounds are concentrated, each of the plurality of points may be displayed in a spot manner.

  4 and 5 are diagrams for explaining the principle of beam forming. Here, for explanation of the principle, it is assumed that the five sound sensors 13 are arranged in a straight line to form one sensor array 25. However, the same principle holds if there are two or more sound sensors.

  As shown in FIG. 4, when the sound source 26 is present in front of the sensor array 25, the sound signals obtained by receiving the sounds by the sound sensors 13 are added as they are by the adder 27, and are shown in FIG. 4. As described above, a sound signal (A) that strongly reacts to the sound source 26 in the front is obtained.

  On the other hand, when the sound source 26 exists at an oblique position as shown in FIG. 5, the sound signals obtained by the sound sensors 13 are added as they are as shown in FIG. 4, and the sound signal (B) shown in FIG. As a result, the signal is attenuated. In order to obtain a sound signal that reacts strongly to the sound source 26 at an oblique position, a delay device 28 that compensates for the delay of the sound reaching each sound sensor 13 is placed and received by each sound sensor 13. The sound signals are delayed by each delay unit 28 and added together. For example, in the case of the delay device 28A shown at the top of FIG. 5, the delay time in each delay device 28 is the same as the time required for the sound to travel by the distance of (d1-d2). The same applies to the other delay devices 28. When the sound signals obtained by the respective sound sensors 13 are delayed and added to each other in this way, sound signals that react strongly to the sound source 26 at an oblique position as shown in FIG. See A)). When the delay patterns by the plurality of delay devices 28 are variously adjusted, sound signals that are strongly responsive to sound sources at various positions can be obtained, and the direction in which the sound sources exist can be known. Further, the sound pressure can be obtained from the level of the sound signal at that time. Further, by adjusting the delay pattern in various ways, even when there are a plurality of sound sources, it is possible to know the direction in which each of the plurality of sound sources exists. Here, an example in which a plurality of sound sensors 13 are arranged in a straight line has been described. However, if the sound sensors 13 are arranged on a two-dimensionally expanding surface as shown in FIGS. Thus, the two-dimensional direction of the sound source position can be known.

  FIG. 6 is a diagram illustrating an example of an image (mark) representing a field of view seen through the glass monitor illustrated in FIGS. 1 and 2 and a sound source position superimposed on the field of view.

  In the example shown here, the automobile can be seen through the glass monitor 12, and two marks 32 representing the sound source are displayed on the glass monitor 12 so as to overlap the two sound source portions of the automobile.

  According to the above embodiment, as shown in FIG. 6, the image representing the sound source is displayed at a position overlapping the sound source in the field of view being observed through the glass monitor 12, rather than simply listening with the ear. The sound source position can be known with high accuracy.

  FIG. 7 is a diagram illustrating a mounting state of the sound source monitoring device according to the second embodiment of the present invention. FIG. 7 is a diagram corresponding to FIG. 2 in the first embodiment described above.

  A sound source monitoring device 50 shown in FIG. 7 includes a mounting unit 51 and a glass monitor 52. The glass monitor 52 is placed at the tip of the right eye, but the field of view cannot be seen through the glass monitor 52, and the right eye can only see the image displayed on the glass monitor 52.

  Here, a video camera 57 and a plurality (four in the example shown) of sound sensors 53 are shown around the video camera 57. The video camera 57 generates an image signal by capturing a forward view of the wearer facing his face. The plurality of sound sensors 53 capture sound generated from within the field of view captured by the video camera 57 and generate sound signals. Note that the sound source monitoring device 50 shown in FIG. 7 is not provided with an operation unit as provided in the sound source monitoring device 10 shown in FIGS.

  FIG. 8 is a schematic diagram showing the entire sound source monitoring device 50 of the second embodiment shown in FIG. Here, the glass monitor, the video camera, and the sound sensor are schematically shown separately.

  The image signal representing the field of view captured by the video camera 57 and the sound signal obtained by each sound sensor 53 are input to the input device 58, converted into a digital signal, and input to a personal computer (hereinafter abbreviated as "PC") 59. Is done. In the PC 59, the same calculation as that of the filter 22, the beam forming unit 23, and the image generating unit 24 shown in FIG. However, here, the frequency component of the band set by the keyboard operation of the PC 59 or the like is extracted from the sound signal input from the input device 58, and represents the sound source position obtained by the beam forming calculation when generating the image. An image signal in which the mark is superimposed on the field of view captured by the video camera 57 is generated. The shading of the mark representing the sound source position is set by operating the keyboard of the PC 59 or the like. The generated image signal is sent to the glass monitor 52, and an image based on the image signal, that is, an image in which a mark representing the sound source position is superimposed on the field of view captured by the video camera 57 is displayed on the glass monitor 52. Displayed and visually recognized by the wearer's right eye. When there are a plurality of sound sources in the field of view, an image in which a plurality of marks are superimposed on an image based on the image signal is displayed.

  In the sound source monitoring device 50 according to the second embodiment, the image captured by the video camera 57 and information representing the identified sound source position are stored in the PC 59 and used for later analysis. That is, the sound source monitoring device 50 according to the second embodiment can recognize the sound source position on the spot using the glass monitor 53. Even after the monitoring is completed, for example, the generation of sound and the operation timing of the device Match / mismatch and various other detailed analysis.

  Here, the sound source is identified by beam forming calculation using a plurality of sound sensors. However, in the present invention, it is not always necessary to employ beam forming calculation. For example, a microphone with a strong directivity and a scanning mechanism that sequentially directs the microphone in each direction in the field of view may be provided, and the sound source may be located in the direction in which the microphone captures the sound strongly. A plurality of highly directional microphones whose orientations are adjusted so as to capture the emitted sound may be arranged side by side.

  In each of the above embodiments, the sound source is visually observed. In addition, the sound source monitoring device is added with a headphone function so that the sound captured by the sound sensor can be heard with the ear. Good.

  In each of the embodiments described above, the case where two marks representing a sound source are displayed so as to overlap the two sound source portions of the automobile has been described. However, as many marks representing the sound sources are displayed as the number of sound sources. In addition, for example, only a mark representing a sound source in which the maximum sound pressure is detected among a plurality of sound sources may be displayed. However, by displaying a plurality of marks representing a plurality of sound sources, the plurality of sound sources can be notified simultaneously.

DESCRIPTION OF SYMBOLS 10,50 Sound source monitoring apparatus 11,51 Installation part 12,52 Glass monitor 13,53 Sound sensor 14 Operation part 15 Frequency selection dial 16 Contrast adjustment dial 21 A / D converter 22 Filter 23 Beam forming part 24 Image generation part 25 Sensor Array 26 Sound source 27 Adder 28 Delay device 32 Mark 57 Video camera 58 Input device 59 PC

Claims (5)

A glass monitor that displays an image that is placed in front of a person and that overlaps the field of view,
A sound sensor that generates a sound signal by capturing a sound emitted from within a field of view obtained by seeing through the glass monitor placed at a fixed position with respect to the glass monitor;
A sound source identification unit for identifying a sound source in the field of view based on the sound signal;
A sound source monitoring apparatus comprising: a sound source display unit that displays an image representing a sound source at a position on the glass monitor that overlaps a sound source in a field of view obtained by seeing through the glass monitor. The sound source identification unit identifies the plurality of sound sources based on the sound signal when there are a plurality of sound sources;
The sound source display unit displays each image representing each of the plurality of sound sources at each position overlapping with each of the plurality of sound sources on a view obtained by seeing through the glass monitor on the glass monitor. The sound source monitoring device according to claim 1. A glass monitor that is placed at the eyes of a person and displays an image;
A camera that is placed at a fixed position with respect to the glass monitor and that captures a forward field of view facing a person and generates an image signal;
A sound sensor that generates a sound signal by capturing sound emitted from within the field of view captured by the camera placed at a fixed position with respect to the glass monitor;
A sound source identification unit for identifying a sound source in the field of view based on the sound signal;
A sound source monitoring apparatus comprising: an image display unit configured to display an image on which an image representing a sound source is superimposed on an image based on the image signal on a position overlapping with the sound source on the image on the glass monitor. The sound source identification unit identifies the plurality of sound sources based on the sound signal when there are a plurality of sound sources;
The sound source display unit displays, on the glass monitor, an image in which images representing the plurality of sound sources are superimposed on the images based on the image signals at positions overlapping the sound sources on the image. The sound source monitoring device according to claim 3, wherein the sound source monitoring device is a device. The sound sensor comprises a plurality of sound sensors placed at positions fixed to the glass monitor and separated from each other,
5. The sound source monitoring according to claim 1, wherein the sound source identification unit identifies a sound source by a beamforming operation based on a plurality of sound signals obtained by the plurality of sound sensors. apparatus.

Images