JP2015152316A - Sound visualizing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound visualizing apparatus that enables sound source information to be shared even by a large number of persons and moreover is adaptable to a situation of simultaneous presence of multiple sound sources.SOLUTION: A sound visualizing apparatus is equipped with a microphone array 20 comprising a plurality of microphones 20a that receives sounds emitted from sound sources in a search area 100, an arithmetic device 40 having a sound source searching unit for searching the search area 100 for sound sources on the basis of sound signals picked up by the microphones 20a and an image generating unit that generates sound source signals representing a sound source image representing the sound source positions found by the sound source searching unit, and a projector 50 to which the sound source signals generated by the arithmetic device 40 are inputted to project the sound source images based on the sound source signals onto the search area 100.

Description

  The present invention relates to a sound source visualization apparatus that visualizes a sound source.

  Conventionally, a device for visualizing a sound source is known.

  For example, Patent Documents 1 to 3 disclose an apparatus that displays a mark representing a sound source position superimposed on an image obtained by photographing a search area with a camera or the like.

  Patent Documents 4 to 5 disclose devices that indicate a sound source position by irradiating a laser beam toward a sound source in an exploration area.

JP 2004-77277 A JP 2009-257944 A JP 2011-35472 A JP2013-250111A JP 2013-250112 A

  In the case of the device disclosed in the above-mentioned Patent Documents 1 to 3 and displaying a mark representing the sound source position superimposed on the photographed image, both the display on the display screen and the actual sound source search area are observed. Both of them need to be associated with each other in the head, which is not intuitive and may cause a mistake in the position of the sound source. Also, only the observer located at a position relative to the display screen can confirm the sound source position, and it is difficult to share the sound source position information with collaborators slightly away from each other.

  In addition, in the case of the apparatus that irradiates laser light toward the sound source in the exploration area disclosed in the above-mentioned Patent Documents 4 to 5, the sound source can be directly pointed to understand intuitively, and the exploration area The sound source position information can be shared with the collaborators who are in a position where it can be visually recognized. However, this laser beam pointing device has a problem that it is difficult to point simultaneously when there are a plurality of sound sources in the exploration area. Further, if the sound source probe and the laser irradiation unit are at different positions, parallax occurs, and there is a possibility that the search position cannot be accurately indicated by the laser beam. In Patent Document 5, parallax is eliminated by mechanically changing the positions of the sound source search probe and the laser irradiation unit. However, with such mechanical movement, the movement itself is difficult, and it takes time to point to the sound source, and there is a problem that the real-time property is lacking when the sound source position moves in time.

  In view of the circumstances described above, an object of the present invention is to provide a sound source visualization apparatus that can share sound source position information even with a large number of people and can handle a plurality of sound sources simultaneously.

The sound source visualization device of the present invention that achieves the above object is as follows.
A probe having a sound sensor for receiving sound emitted from a sound source in the exploration area;
A sound source search unit that searches a sound source position in the search area based on a sound signal received by a sound sensor, and an image that generates a sound source image signal that represents a sound source image that represents the sound source position searched by the sound source search unit An arithmetic unit having a generation unit;
And a projector for projecting a sound source image based on the sound source image signal generated by the arithmetic unit to the search area.

  Since the sound source visualization device of the present invention projects a sound source image on a search area using a projector such as a liquid crystal projector, the plurality of sound sources are projected even if a plurality of sound sources exist simultaneously in the search area. The sound source position information can be shared by many people. Further, even when the sound source moves in time, the projected sound source position can be moved with real-time characteristics.

  Here, in the sound source visualization device according to the present invention, the sound source visualization device includes a camera that captures an exploration region, and the arithmetic device is projected by a projector based on a captured image obtained by capturing with the camera. It is preferable to include an image adjustment unit that adjusts the sound source image signal generated by the arithmetic device so that the sound source position on the sound source image matches the sound source position obtained by the search by the arithmetic device.

  In the case of the sound source visualization device of the present invention, the sound source image projected by the projector is captured by the camera, so that the sound source position searched by the sound source search unit and the sound source position indicated by the projected sound source image are between Even if there is a parallax between the probe and the projector by knowing the positional deviation, it is possible to correct the parallax and correctly indicate the sound source position.

  According to the present invention described above, a sound source visualization apparatus is provided that can cope with a case where a plurality of sound sources exist simultaneously and that can share sound source position information among a large number of people.

It is a schematic diagram of the sound source visualization apparatus of the present invention. It is a flowchart which shows the calculation content in the arithmetic unit which comprises the sound source visualization apparatus shown in FIG. It is a schematic diagram of the sound source image projected on the search area.

  Embodiments of the present invention will be described below.

  FIG. 1 is a schematic diagram of a sound source visualization apparatus according to the present invention.

  The sound source visualization device 10 includes a microphone array 20 having a plurality of microphones 20 a, and an image capturing camera 30 is disposed at the center of the microphone array 20. The microphone 20a, the microphone array 20, and the camera 30 correspond to examples of a sound sensor, a probe, and a camera, respectively, according to the present invention.

  In general, a plurality of methods such as a beam forming method (BF method), a sound intensity (SI method), and an envelope intensity method (EI method) are known as sound source search methods. When the SI method and the EI method are adopted, the direction of arrival of sound is directly searched. In the case of the BF method, since the intensity distribution of sound pressure is known, the direction of the maximum position of the sound pressure is obtained as a sound source. All of these sound source search methods are well-known techniques, and the present invention does not ask the sound source search method itself, and therefore a detailed description of the sound source search method is omitted here. The embodiment described here will be described with the BF method in mind.

  The camera 30 is arranged at the center of the microphone array 20 in the same direction as the microphone 20a. When the camera 30 cannot be placed at the center of the microphone array 20, the camera 30 is positioned close to the center of the microphone array 20 so that the parallax between the center of the microphone array 20 and the camera 30 is within the allowable error range. Placed in.

  In addition, the sound source visualization device 10 includes an arithmetic device 40 and a projector 50.

  In the arithmetic unit 40, the sound source position in the search area 100 to which the microphone 20 is directed is searched based on the sound signal received by the microphone 20a, and the sound source image signal representing the sound source image representing the sound source position is obtained. Generated. The generated sound source image signal is input to the projector 50 after the position adjustment of the sound source image is performed as described later. The projector 50 is arranged at a position having parallax with respect to the microphone array 20 and the camera 30.

  The projector 50 displays a sound source image based on the input sound source image signal on a liquid crystal screen in the projector 50, and the sound source image on the liquid crystal screen is projected toward the search area 100. Therefore, the camera 30 can obtain a captured image of the exploration region 100 in a state where the projected sound source image is superimposed.

  A captured image obtained by the camera 30 is input to the arithmetic unit 40. In this arithmetic unit 40, the sound source image signal representing the generated sound source image is searched based on the sound signal obtained by the microphone array 20 on the sound source image projected by the projector 50 based on the photographed image. Adjust so that it matches the sound source position.

  The adjusted sound source image signal is input to the projector 50, and the projector 50 projects a sound source image based on the adjusted sound source image signal toward the search region 100. This adjustment corrects the parallax even though there is parallax between the microphone array 20 and the camera 30 and the projector 50, and indicates the correct sound source position.

  The arithmetic device 40 corresponds to an example of an arithmetic device according to the present invention, and the projector 50 corresponds to an example of a projector according to the present invention.

  FIG. 2 is a flowchart showing the calculation contents in the calculation device 40 constituting the sound source visualization device 10 shown in FIG.

  FIG. 3 is a schematic diagram of a sound source image projected on the search area.

  As shown in FIG. 2, the arithmetic device 40 first performs sound source search based on the sound signal obtained by the microphone array 20 (step S01). Here, the sound source position obtained by the sound source search is referred to as a sound source position (1). A plurality of sound source positions obtained by this sound source search may exist in the search region 100.

  Next, a sound source image representing the sound source position (1) obtained by this exploration is created and sent to the projector 50, and the sound source image is projected toward the exploration region 100 by the projector 50, and the sound source image is projected. The area 100 is photographed by the camera 30 (step S02).

  FIG. 3A is a diagram in which a sound source image representing the sound source position (1) is superimposed on an accurate position on the search region 100. FIG.

  That is, here, the sound source exists in the central portion of the large mountain shape of the two large and small mountain shapes.

  FIG. 3B shows a photographed image obtained by photographing the search area 100 on which the sound source image is projected by the projector 50 with the camera 30. On the captured image, a position slightly deviated from the correct sound source position is indicated as the sound source position.

  Then, next, a sound source position is identified based on this captured image (step S03). Here, the identified sound source position is referred to as a sound source position (2). As described above, here, the sound source position (2) is slightly deviated from the correct sound source position.

  Next, the difference between the position of the sound source position (1) obtained by the search based on the sound signal and the sound source position (2) obtained from the photographed image is calculated, and the sound source position (2) is compared with the sound source position (1). ) Is calculated in which direction and how much (step S04).

  Then, it is determined whether or not the difference between the calculated image position (2) and the image position (1) is within a predetermined allowable error (step S05). If the allowable error is exceeded, the sound source image is shifted based on the difference so that the difference decreases (step S06). A new sound source position (2) is identified based on the photographed image obtained by photographing the sound source image after the shift (step S03), a difference calculation is performed (step S04), and the difference is within an allowable error. It is determined whether or not (step S05). When the difference is within the allowable error (step S05), the process returns to step S01. Thereby, it is possible to cope with the case where the sound source position moves in time.

  FIG. 3C is a diagram showing a sound source image that is projected onto the exploration area 100 and whose position is adjusted within an allowable error.

  In the case of this embodiment, a more accurate sound source position is indicated by shifting the sound source image in this way.

  Although the case where there is one sound source in the exploration area 100 is shown here, projection and position adjustment are performed in the same manner when there are a plurality of sound sources.

DESCRIPTION OF SYMBOLS 10 Sound source visualization apparatus 20 Microphone array 20a Microphone 30 Camera 40 Calculation apparatus 50 Projector 100 Search area

Claims (2)

A probe having a sound sensor for receiving sound emitted from a sound source in the exploration area;
A sound source search unit that searches for a sound source position in the search region based on a sound signal received by the sound sensor, and a sound source image signal that represents a sound source image that represents the sound source position searched by the sound source search unit. An arithmetic unit having an image generation unit to generate;
A sound source visualization apparatus, comprising: a projector that inputs a sound source image signal generated by the arithmetic unit and projects a sound source image based on the sound source image signal onto the search area. The sound source visualization device includes a camera that images the exploration area,
Based on a photographed image obtained by photographing with the camera, the arithmetic device is arranged so that a sound source position on a sound source image projected by the projector matches a sound source position obtained by exploration with the arithmetic device. The sound source visualization apparatus according to claim 1, further comprising an image adjustment unit that adjusts a sound source image signal generated by the arithmetic device.

Images