JP2008244865A - Sound field switching system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system capable of reproducing a sound field without a sense of incompatibility in the sense of direction and being rich in presence, while matching sound fields constituted of actually collected sounds between a video image and a real space. <P>SOLUTION: A sound field switching system comprises: a plurality of sound collecting means 3 disposed at positions separated from each other while fixing the positions for collecting sounds in a real space wherein the system is disposed; a motion picture shooting means 2 moving within an area defined by positions, in a plane view, at which the plurality of sound collecting means 3 are disposed; a multi-channel acoustic signal storage means 4 for storing collected acoustic signals as a channel acoustic signal for each sound collecting means 3; a shooting position signal storage means 5; a video signal storage means 6 for storing video signals of the motion picture shot; a plurality of sound playback means 9 for playing back the stored channel acoustic signals for each corresponding channel; a video image playback means 10 for playing back the stored video signals; and a sound field switching means 8 for switching channels of acoustic signals inputted to the sound playback means 9 in accordance with a shooting position of the shot video image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a system for switching a sound field of recorded sound in accordance with a photographed video.

Conventionally, when a subject to be moved moves, the camera for video recording is designed for multi-channel recording so that the sound image can be changed and played back along with the recorded video at the same time. A device equipped with a plurality of directional microphones has been proposed.
For example, Patent Document 1 includes four microphones that are directed to the front right, front left, rear right, and rear left at the top of the video camera, and one microphone for narration, for a total of five channels. A technique for recording is disclosed.
In Patent Document 2, many microphones for collecting sounds from four or more directions are arranged in a video camera, and four channels are generated by using a voice synthesis technique. There is disclosed a technique of converting into a signal and synthesizing and outputting it as an audio signal of three or more channels at the time of reproduction.
Furthermore, Patent Document 3 discloses a technique in which a plurality of microphones necessary for collecting 3-1 type 4-channel sound are concentrated in one place and mounted on a camera. This document further describes a technique in which a plurality of microphones arranged in a concentrated manner are arranged apart from the camera body and the microphone group is rotated in conjunction with the rotation operation of the camera.
Japanese Patent Laid-Open No. 2005-341073 JP-A-2005-223706 Japanese Utility Model Publication No. 6-7368

However, as disclosed in each patent document, when a microphone is attached to a camera, even if a directional microphone is used, the microphones are concentrated at substantially the same position. The sound field to be played was short and the presence was insufficient. In addition, since the microphone moves with the camera, the sound collection is restricted by the photographer's range of action, and there is a problem including noise accompanying the movement of the camera and the photographer.
In addition, as mentioned in Patent Document 3, in the technique of rotating a microphone concentrated in one place at a position separated from the camera in conjunction with the rotation of the camera, the distance between the microphones is short and concentrated in one place. In addition, since the camera is fixed in position and rotated just like the microphone, a sound field with a sufficient presence cannot be reproduced. There is also a problem of noise when rotating the microphone.

Thus, a system that can reproduce a realistic sound field according to an image when a shooting site changes due to movement of a camera or the like has been desired.
In addition, a technology has been developed in which a series of positions where a sound image should be localized in a virtual space such as a game is sequentially set by a movement trajectory input means to determine a movement trajectory and a moving speed of the sound image, thereby reproducing sound. . However, this synthesizes a sound image in a virtual space, and does not match a sound field collected in real space with a moving image.

  The present invention has been made to solve the above-mentioned problems, and the sound field actually collected in the real space is matched with the image, and the sound field is reproduced with a sense of direction without any sense of incongruity. It aims at providing the system which can do.

  In order to achieve such an object, the sound field switching system according to the present invention includes a plurality of acoustic sound collecting means arranged to fix the positions at positions spaced apart from each other, and to collect the sound of the arranged real space. Moving image capturing means that moves at least within an area defined by the arrangement position of the plurality of sound pickup means in plan view, and the collected sound signals as channel sound signals for each sound pickup means A multi-channel sound signal storage means for storing; a video signal storage means for storing a video signal of a captured video; a plurality of sound playback means for reproducing the stored channel sound signal for each corresponding channel; Video reproduction means for reproducing the recorded video signal, and sound field switching means for switching the channel of the audio signal input to each sound reproduction means in accordance with the shooting position of the shot video. And wherein the Rukoto.

According to the present invention, the acoustic signals of each real space picked up by the plurality of sound collecting means that pick up the sound of the arranged real space are arranged at positions spaced apart from each other, Each is stored as one channel acoustic signal. On the other hand, the moving image shooting means takes a picture of a shooting target while moving within an area defined by the arrangement position of the plurality of sound pickup means in plan view, and the shot video signals are also stored. Note that the moving range of the moving image photographing means moves at least within the area, but may further move beyond the area.
Then, when the sound signal and the video signal are output from the storage means and reproduced by the reproduction means, the channel of the sound signal in the sound reproduction means is switched corresponding to the photographing position of the photographed video. Therefore, the multi-channel sound that is stored once by the storage means and then played back through appropriate editing as necessary matches the video, and the actual sound collected actually has a sense of presence. Can be reproduced.

  In the present invention, when the sound field switching unit is controlled to be switched by the shooting position signal detected by the shooting position detection unit provided in the moving image shooting unit and stored in the storage unit, The shooting position detection unit provided in the shooting unit automatically detects the shooting location and shooting direction corresponding to the video, and the detection signal is temporarily stored in the storage unit. Since the stored shooting position signal is used when switching the acoustic signal channel corresponding to the shooting position, the sound field can be automatically switched in accordance with the image.

  Further, in the present invention, instead of moving the moving image shooting unit, the video signal shot by the plurality of moving image shooting units arranged in the area is switched to correspond to the shooting position of the shot video. Thus, when switching the channel of the sound signal input to each sound reproducing means, it is not necessary to move the photographing means such as a camera greatly, and also when switching the sound signal channel corresponding to the photographing position, It is only necessary to perform switching corresponding to the photographing means.

  ADVANTAGE OF THE INVENTION According to this invention, the sound field which consists of the sound actually picked up by the image | video and the real space corresponds, and can provide the system which can reproduce the sound field overflowing in a sense of direction without a sense of incongruity. .

Hereinafter, preferred embodiments of a sound field switching system according to the present invention will be described in detail with reference to the drawings. In the description, the same reference numerals are used for the same elements, and redundant descriptions are omitted.
FIG. 1 is a conceptual diagram showing a sound field switching system 1 according to an embodiment of the present invention. In this example, microphones a to d, which are sound pressure type microphones as the sound pickup means 3, are arranged at fixed positions at four positions separated from each other. The sound signal picked up by the microphone a is stored in the multi-channel sound signal storage means 4 as the channel c1. Similarly, acoustic signals collected by the microphone b, the microphone c, and the microphone d are stored as a channel c2, a channel c3, and a channel c4, respectively. The multi-channel acoustic signal storage means 4 may be a computer memory or various recording media.
Further, the number of microphones to be arranged and the arrangement positions are not limited to four.

  On the other hand, the camera as the moving image capturing means 2 captures an object to be photographed while moving the microphones a to d mainly in an area defined by arcs or straight lines. The captured video is stored in the video storage means 6. The video storage means 6 may be a memory or a recording medium built in the camera, a computer memory for storing together with an acoustic signal, or other storage means.

The camera as the moving image photographing means 2 is separately provided with a photographing position detecting means. The shooting position is detected in order to match the image and the sound field by switching the sound field composed of the sound collected by each microphone according to the detected position signal. Therefore, it is preferable that the imaging position signal here includes a camera direction signal as well as a position signal as a place. However, when the sound field is simply switched, only one of the signals may be used. As these detection means, for example, a GPS device or a gyro can be employed.
The detected shooting position signal may be stored in the shooting position signal storage means 5 separately from the video, but if it is stored together with the signal for each frame of the video, the subsequent processing can be performed. Efficient. In this case, the video signal storage means 6 also serves as the shooting position signal storage means 5.

The sound signals of the channels c1 to c4 output from the multi-channel sound signal storage means 4 are configured to be input to the sound image / image editing means 7. The sound processing means in the sound image / image editing means 7 is, for example, as shown in FIG. 2, and a frequency adjustment / level adjustment means, a power amplification means, etc., following the sound field switching means 8 essential in the present invention. Used.
Further, the sound image / image editing means 7 is connected to each speaker through the transmission lines L1 to 4 so that the sound signals of the respective channels from the sound image / image editing means 7 are input to the speakers a to d as the sound reproducing means 9. a to d. Here, it goes without saying that the transmission line L may be wired or wireless as in the case of signal transmission in other places.

The sound field switching means 8 is for switching the channels c1 to c4 and contacting the transmission lines L1 to L4 depending on the shooting position of the camera. For example, when the camera is at the position A as shown in FIG. 1, the channel c1 of the microphone a is connected to the transmission path L1 of the speaker a, and similarly, the channel c2 of the microphone b is connected to L2 of the speaker b. Channel c3 of c is connected to L3 of speaker c, and channel c4 of microphone d is connected to L4 of speaker d.
In this example, the channels c1 to c4 are not directly connected to the transmission path for inputting the speakers a to d, but are output from the sound field switching means 8 as shown in FIG. The signal is sent from the transmission lines L1 to L4 to the speakers a to d through the frequency / level adjusting means and the power amplifying means.

From this state, for example, if the camera has moved to position B, the sound field switching means 8 performs a switching operation according to the shooting position signal during editing so that the channel c1 of the microphone a corresponds to L2 of the speaker b. Switch. Similarly, the channel c2 of the microphone b is switched to L4 of the speaker d, the channel c3 of the microphone c is switched to L1 of the speaker a, and the channel c4 of the microphone d is switched to L3 of the speaker c.
Here, the case where the camera has moved from position A to position B shown in FIG. 1 has been described. However, if the camera moves to another position or returns to the original position, the channel of the acoustic signal is correspondingly changed. Needless to say, it is switched by the sound field switching means 8.
As the sound field switching means 8 for performing such switching, in the matrix constituted by the channels c1 to c4 and the transmission lines L1 to L4 (more precisely, the contact points corresponding to those transmission paths in the editing means 7), the photographing position A matrix switch that turns on a matrix intersection determined for each signal can be used. However, the present invention is not limited to this. For example, a logic circuit that switches according to the photographing position signal may be configured by a semiconductor switch.

Further, in this example, the case where the channels c1 to c4 of the microphones a to d and the transmission lines L1 to L4 of the speakers a to d are the same number and correspond one-to-one has been described. Absent.
For example, when the number of transmission lines L is four, the number of microphones and channels c is set to be larger than that, and the sound field switching unit 8 switches the output in accordance with the shooting position signal, and then the four transmission lines. An acoustic signal may be synthesized so as to correspond to L.

  As shown in FIG. 2, the sound signal after being switched by the sound field switching means 8 is adjusted by the editing means so that the sound can be easily heard by the frequency adjusting / level adjusting means, and further output by the power amplifying means. The signal is amplified. This editing means is not particularly limited to such processing, and processing performed in normal acoustic organization may be performed as necessary. Further, in the present embodiment, since the channel is switched by the sound field switching means 8, in order to avoid a sudden and sudden change in the sound field at the time of switching, these frequency adjustment / level adjustment means are used to change the sound field. You may make adjustments that make it sound gradual. For the same reason, the sound of the channel before switching input to one transmission line L is gradually reduced using, for example, a signal arithmetic circuit, delay circuit, phase shift circuit, etc. The acoustic signal may be gradually increased.

  Furthermore, in this example, after the channel is switched by the sound field switching means 8, signal processing is performed as necessary, and then output to the transmission lines L1 to L4. However, the multi-channel acoustic signal storage means 4 After performing necessary processing on the sound signal for each channel c output from, the channel may be switched by the sound field switching means 8 and output directly to the transmission lines L1 to L4.

  On the other hand, the video signal stored in the video storage unit 6 is edited by the sound image / image editing unit 7 as necessary, and then reproduced by the video reproduction unit 10 in synchronization with the sound. Here, the video reproduction means 10 may be composed of a video projection device and a screen, or may be a display or the like. In addition, although the description has been given here as a single screen, the same processing can be performed even with a plurality of screens, and the presence can be increased.

Next, the operation of the system of this embodiment will be described with reference to FIG.
In the area where microphones a to d are arranged at four locations in advance, the photographer moves while photographing with the camera. At that time, for example, when the camera moves from FIG. A to FIG. B, the shooting position signal detected by the shooting position detection means provided in the camera is stored together with the video signal. The acoustic signals picked up by the microphones a to d during this period are stored as channels c1 to c4.
At the time of editing, as described above, when the photographing position signal is switched from the signal at the position A to the signal at the position B, the sound field switching means 8 performs the switching operation, and the channel c1 of the microphone a is transmitted corresponding to the speaker a. Switching from the path L1 to correspond to L2 of the L2 speaker b. Similarly, the channel c2 of the microphone b is changed from L2 corresponding to the speaker b to L4 of the speaker d, the channel c3 of the microphone c is changed from L3 corresponding to the speaker c to L1 of the speaker a, and the channel c4 of the microphone d is changed to the speaker d. The corresponding L4 is switched to correspond to L3 of the speaker c.

By such switching, the sound field is as follows for the viewer who is viewing the video of the video playback means 10 (turning to the right in the figure) positioned at the center of the speakers a to d. Change.
When the image taken by the camera at the position A is displayed before switching, the sound picked up by the microphone a can be heard from the left front speaker a. Similarly, sound from the microphone b is heard from the right front speaker b, sound from the microphone c is heard from the left rear speaker c, and sound from the microphone d is heard from the right rear speaker d.
When the image taken by the camera at position B is displayed after switching, the sound picked up by the microphone a can be heard from the speaker b on the right front. Similarly, sound from the microphone b is heard from the right rear speaker d, sound from the microphone c is heard from the left front speaker a, and sound from the microphone d is heard from the left rear speaker c.
That is, as the image is rotated 90 degrees to the left, the sound field is also rotated correspondingly. Not only when the photographing means moves from position A to position B, but also when it moves to another position or returns to the original position, the sound reproduced from each speaker according to that position, that is, according to the image. It is natural that is switched.
In addition, here, two positions when the camera has moved from position A to position B have been described. However, it goes without saying that such movement of the position gradually includes a movement in the middle. Nor.

  In the present embodiment, it is possible to automatically follow an image of a camera that moves freely, and to change the sound field by sound reproduced from a plurality of positions. The reproduced sound is fixedly arranged at sound collecting points separated from each other independently of the movement of the camera, and the sound in the real space is picked up at that point, and the open feeling with little noise is obtained. It is said that there is a sound.

  As an application example of the present embodiment, the present invention can also be applied to a case where microphones are respectively installed at four corners of the vehicle and the inside and outside of the vehicle are photographed while moving the camera inside the vehicle. In this case, the real video and the real sound field inside and outside the vehicle can be reproduced as full of realism.

Next, referring to FIG. 3, an embodiment will be described in which video signals captured by a plurality of moving image capturing means 2 are switched instead of moving the moving image capturing means 2.
In this example, a plurality of cameras A to D having different shooting ranges are arranged in an area where microphones a to d are arranged. The video image | photographed with each camera is memorize | stored in the video memory | storage means 6 with the signal which specifies a camera camera. When editing the image, the video of each shooting camera is switched and edited, and a signal for specifying the camera number is input as a switching signal from the camera number command means 20 to the sound field switching means 8, The channel c is switched.
In some cases, the camera may be switched during shooting, and only a single video signal may be stored in the video signal storage unit 6 together with the shooting camera number.
In addition, since description of the other location in this embodiment is the same as that of embodiment of FIG. 1, it abbreviate | omits.
In this embodiment, the movement range of the camera may be narrow.

Next, with reference to FIG. 4, an embodiment in which the sound field switching command is manually performed will be described. In the example of FIG. 1, a signal for switching the sound field is automatically obtained along with the movement of the camera, and the command to the sound field switching means 8 is also automatically performed. On the other hand, in the case of the embodiment of FIG. 4, a person who has a memory in the captured image inputs a switching signal to the sound field switching means 8 along with the change of the image at the time of editing. The signal channel can be switched. As a further modification of this example, it is also possible for the viewer to issue a command to the sound field switching means 8 while watching the screen. In these cases, the switching signal can be output by, for example, pseudo-designating the position of the camera in an input device that pseudo-displays the arrangement of the microphones a to d. . Further, it is not necessary to provide the camera with means for detecting the shooting position.
In addition, since description of the other location in this embodiment is the same as that of embodiment of FIG. 1, it abbreviate | omits.

  In this embodiment, part of the processing is performed manually, but it is not necessary for the moving image photographing means 2 to include a photographing position detecting means, and no means for processing the photographing position signal is necessary. The entire system can be simplified.

It is a conceptual diagram of the system of embodiment of this invention. It is explanatory drawing of the sound image and image editing means of embodiment of this invention. It is a conceptual diagram of other embodiment of this invention. It is a conceptual diagram of further another embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Sound field switching system, 2 ... Moving image photographing means, 3 ... Sound pickup means, 4 ... Multi-channel sound signal storage means, 5 ... Shooting position signal storage means, 6 ... Video signal storage means, 7 ... Sound image / image editing Means 8 sound field switching means 9 sound reproduction means 10 video reproduction means

Claims (3)

A plurality of sound collecting means arranged to fix the positions at positions separated from each other and pick up sound of the arranged real space;
Moving image capturing means that moves within an area defined by an arrangement position in plan view of the plurality of sound pickup means;
Multi-channel acoustic signal storage means for storing the collected acoustic signals as channel acoustic signals for each acoustic sound collection means;
Video signal storage means for storing a video signal of a captured video;
A plurality of sound reproduction means for reproducing the stored channel sound signal for each corresponding channel;
Video playback means for playing back the stored video signal;
A sound field switching system, comprising: a sound field switching means for switching the channel of the sound signal input to each of the sound reproduction means corresponding to the shooting position of the shot video.   2. The sound field according to claim 1, wherein the sound field switching unit is switch-controlled by a shooting position signal detected by a shooting position detection unit provided in the moving image shooting unit and stored in a storage unit. Switching system. Instead of moving the moving image shooting means, each video signal shot by a plurality of moving image shooting means arranged in the region is switched, so that the respective shooting positions of the shot videos are The sound field switching system according to claim 1, wherein the channel of the sound signal input to the sound reproducing means is switched.

Images