JPH06303474A - Camera device - Google Patents

Abstract

PURPOSE:To provide a camera device which can regenerate a more real AV sound field, can more accurately relate the information on the positions, the sizes, etc., of the video sound sources with the video information, and then can easily edit these information. CONSTITUTION:The microphones 11, 12 and 13 are placed independently with each other to each of N pieces of pickup desired sound sources among plural sound sources. Then the position information MP1, MP2 and MPM received from the position detectors 21, 22 and 23 placed near the microphones 11-13 respectively are multiplexed on a video signal V1 by a position/video information multiplexing device 52 of a camera device 50 as the coordinates P1, P2 and P3 defined on an image photographed by a camera 51. Thus a multiplex signal IM is transmitted. As a result, a more real AV sound field can be regenerated and the information on the positions, the sizes, etc., of the video sound sources can be easily and accurately related with the video information.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera device suitable for use as a sound collecting / recording device for reproducing stereoscopic video / stereo sound.

[0002]

2. Description of the Related Art Conventionally, a sound having a spatial spread,
How faithfully reproduces (plays) sounds such as voice as an actual sound image
Various recording methods and reproducing methods have been proposed and tried. In particular, in a stereoscopic television, there is a demand for more faithful reproduction of sound as well as stereoscopic reproduction of the image. For example, the “intense stereo system”, which is a general sound collecting and reproducing system, collects sound by using a large number of microphones as sound collecting means, and level differences among a plurality of defined channels, for example, N channels. , Information such as time difference is added and recorded separately.
At the time of reproduction, a virtual sound image is generated by N speakers arranged at a predetermined position. In addition, a binaural sound collection reproduction system and a one-point sound collection reproduction system are also known, but these also generate a virtual sound image.

[0003]

By the way, since the conventional sound collecting and reproducing systems represented by the above-mentioned examples all reproduce the virtual sound image, the original purpose of still generating the real sound image is achieved. Can not do it. As a result, there is still a problem caused by the virtual sound image, for example, the reproduction position of the sound is limited, the image of the sound is not formed when the position is displaced, and there is a problem that the virtual sound image has no sense of reality. In addition, the listening point for effectively reproducing the sound image is limited to a limited area.

On the other hand, in the conventional sound collecting and reproducing method of a source accompanied by a general image, in order to match the position of the sound source on the screen with the position of the reproduced sound image as much as possible, the center of the screen,
Speakers are arranged as real sound images on the backsides of the left and right three places (or five places by further dividing the inside), and sound reproduction is performed on the premise of this positional relationship.
However, if there are many sound sources, it is necessary to assign one speaker to each sound source, and if the screen is large or a small number of speakers are used, the sound source image on the screen and the position It was impossible to control the size of the sound source and the size of the sound source.

Even if a large number of speakers can be arranged, it is not easy to control the position and size of the sound source (Japanese Patent Laid-Open No. 2-224495). This is because the information recorded on the recording medium is not the information about the position and size of the sound source on the image but the control signal to the speakers (variable mixers) arranged in a matrix. This is because the information was not compatible with the speaker. On the other hand, a method has been proposed in which information about the position and size of a sound source on a video is simultaneously multiplexed or independently recorded or transmitted with a video signal (on a recording medium). Since it is separate from the camera device, it is not only inconvenient to handle, but it is also inconvenient and inaccurate to associate information about the position and size of the sound source on the image with the image signal.

The present invention has been made in view of the above-mentioned circumstances, and more realistic AV sound field can be reproduced, and information on the position and size of a sound source on the image can be easily and accurately imaged. Can be associated with information, and
An object is to provide a camera device that can be easily edited.

[0007]

In order to solve the above-mentioned problems, the camera device of the present invention is provided with an image pickup means for photographing a sound source together with the background, and the sound source position information on the photographed video together with the video signal. Output. Preferably, the camera device includes a multiplexing means for multiplexing the sound source position information with the video signal. Further, preferably, the camera device outputs the sound source position information independently of the video signal. Further preferably, the camera device is arranged in the vicinity of the sound collecting means for collecting the sound of the sound source and outputting it as an audio signal,
A position detecting means for detecting the position of the sound source or the sound collecting means and outputting it as position information, and multiplexing the sound source position information by the image pickup means, the audio signal by the sound collecting means and the position information by the position detecting means, Or output independently.

[0008]

In the present invention, the image pickup means for photographing the sound source is provided together with the background, and the sound source position information on the photographed image is output together with the image signal. Therefore, since the sound source position information on the video is simultaneously transmitted with the relative position coordinates on the video together with the absolute coordinates of the camera device, a more realistic AV reproduced sound field can be reproduced. Further, in the above camera device, if the sound collection (sound source) position information is simultaneously sent out together with the sound signal and the video signal, by using the real sound image at the time of reproduction,
A more realistic AV (sound-video) reproduction sound field can be reproduced. Further, in the above camera device, various sound source position information is sent simultaneously by the camera device independently of the video signal and the audio information, so that only the audio information and the video information are used alone, or only the sound source position information is used. Can be changed very easily, which makes it easy to edit. In the above camera device, by integrating the camera device for recording the image and the microphone for collecting the sound, it becomes easy to associate the position of the sound source on the image, and the information about the size thereof with the image information, It is accurate and easy to handle.

[0009]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows a block diagram of a sound collecting device of a sound collecting system (system or device) to which a camera device as a first embodiment of the present invention is applied. The sound collecting device shown in FIG. 1 will be described in connection with a specific example, for example, when performing live sound collection of a performance at a concert hall. Generally speaking, sound processing is
For example, the position and size of the sound source to be collected and reproduced, such as the performance sound of a concert, the engine sound in a car stadium,
Sound collection microphone Position, sensitivity characteristics, sound collection conditions such as directivity, editing method and device conditions, playback method, performance of device used for sound processing, etc. From the viewpoint of whether it is possible to generate a sound image, the conditions in each processing process will be determined again. This sound pickup device picks up sounds from the plurality of sound sources so that an actual sound image can be played in consideration of recording and playing.

In FIG. 1, in addition to a plurality of musical instruments such as a piano and a violin as sound sources to be picked up at the concert hall, there are sounds from the audience at the concert hall. The sound collecting device shown in FIG. 1 has a plurality of microphones, and in this example, a first microphone 11 that detects sound from a piano, a second microphone 12 that detects sound from a violin, and a venue. It has an M-th microphone 13, an M + 1-th microphone 14, and an N-th microphone 15 for detecting the sound from.

Further, the first microphone 11 has a first position detecting device 21 for providing information indicating its position, and the second microphone 12 has a second position for providing information indicating its position. The detection device 22 and then the Mth microphone 13 have a third position detection device 23 which provides information indicating its position. On the other hand, the M +
No position detecting device is provided in the first microphone 14 and the Nth microphone 15. Further, the sound collecting device has a recording / reproducing device 40.

The first microphone 11 is arranged, for example, in the vicinity of a piano, and mainly detects a sound having a piano as a sound source and outputs it as a sound signal S ₁ to the recording / reproducing device 40. Similarly, the second microphone 12 is arranged, for example, in the vicinity of the violin, detects the sound mainly having the violin as a sound source, and outputs it as the sound signal S ₂ to the recording / reproducing device 40, and the M-th microphone. Microphone 1
3 detects the sound of the concert hall and outputs it as a sound signal S _M to the recording / reproducing device 40. On the other hand, the (M + 1) th microphone 14 and the Nth microphone 15
Detects other sounds and outputs them to the recording / reproducing device 40.

The position information supplied from the first position detecting device 21 to the Nth position detecting device 23 is the predetermined reference position P (X ₀ , Y ₀ , Z ₀ ) of the concert hall 3,
For example, using the position of the conductor as the origin of the coordinate position, the relative or absolute position P (x _i , y _i ,
z _i ). Further, the position information is not limited to the rectangular coordinates, and for example, the polar coordinate system P (r _i , φ _i , θ _i )
But it's okay.

Musical instruments such as a piano and a violin in a concert hall are usually arranged at fixed positions in the concert hall. Therefore, in this example, the first position detecting device 21 and the second position detecting device 22
The M-th position detecting device 23 outputs the position information detection signals MP ₁ , MP ₂ and MP _N for the fixed piano, violin, and the entire venue respectively to the position information / video information multiplexing device 52 in the camera device 50. Supply to.

The camera device 50 comprises a camera 51, a position information / video information multiplexing device 52, and a position detection device 53 for providing position information of the camera 51. The camera 51
This is a stereoscopic camera for photographing the entire concert hall, and its video signal V1 is output to the position information / video information multiplexing device 30. The position detecting device 53 is arranged near the camera 51. For position detection (calculation),
For example, a method using a gyroscope (x, y, z: multiplexed signal is also possible) output calibrated at a reference position,
There are methods that use satellite radio waves such as satellites (GPS), and methods that measure the distance (time) from a plurality of ultrasonic sound sources. Next, the monitor 60 displays position information P ₁ (x, y, z), P ₂ (x, y, z), P of the image on the screen.
Each of ₃ (x, y, z) is output to the position information / video information multiplexing device 52.

The position information / video information multiplexing device 52 includes position information MP ₁ , MP ₂ , MP _M and position information P ₁ , P ₂ ,
P ₃ is multiplexed with video signal V 1 (multiplexing)
Then, it is output to the recording / reproducing device 40 as a multiplexer signal IM.

As described above, in the case of picking up a video and an audio signal at the same time, microphones are installed independently for each (N) sound sources to be picked up, and positional information is required among the microphones ( The position information of the M microphones is multiplexed with the video signal V1 and output as the coordinates on the video captured by the camera 51.

Further, the position information / video information multiplexing device 52
As a method of multiplexing position information and a video signal in
For example, a frequency multiplexing method suitable for analog signal processing or a time multiplexing method suitable for digital signal processing is performed.

The frequency multiplexing method will be described with reference to FIG. A frequency band is used for multiplexing the video signal V1, and a frequency band outside this frequency band is used for multiplexing the position information MP ₁ , MP _{2, and the} like.

Next, the time multiplexing process will be described with reference to FIG. The video information V and its position information MP are time-sequentially divided and multiplexed. As the multiplexing in the position information / video information multiplexing device 52, time multiplexing for performing digital processing is easy, but frequency multiplexing is slightly difficult to perform individually in consideration of price factors. Therefore, when frequency multiplexing is performed, the position information and the video signal may be once input to the recording / reproducing device 40, and frequency multiplexing may be performed in the recording / reproducing device 40. Also, the recording / reproducing apparatus 40 may perform time multiplexing processing.

In any case, from the recording / reproducing apparatus 40, the acoustic signals OS ₁ , OS ₂ , ...
..., OS _M is output, and the multiplexer signal OM in which the video signal and the position signal are multiplexed is the position information /
It is output to the video output separation device 70.

The recording on the N audio channels in the recording / reproducing apparatus 40 is recorded for each independent audio channel on various recording media such as recording on a magnetic tape, recording on an optical disk, recording on a semiconductor memory. It

The storage signal for each audio channel stored in the storage medium by the recording / reproducing device 40 is edited as needed, and at the time of reproduction, it is output as an audio signal for each channel and a position signal corresponding thereto. In the editing process, since the sound signal and the position signal are multiplexed and stored for each audio channel, the audio editing work with spatial signal processing can be performed. Further, the sound information including the position information stored in the sound collecting device can be directly reproduced without performing the editing work. Alternatively, in the recording / reproducing device 40, a signal in which sound information and position information are multiplexed and a signal in which they are not multiplexed can be directly output and directly reproduced for each audio channel without being stored in a storage medium. The reproduction process will be described later.

Further, the position information / video output separation device 70
Represents the position information OMP ₁ , O from the multiplexer signal OM.
MP ₂ , ..., OMP _M are separated and output, and the video signal OV ₁ is separated and output.

In this embodiment, when the sound source is at a fixed position, position information indicating the positional relationship is set in advance, and if the position information is used, the position detecting device may be omitted. Good.

Next, the reproducing system of the audio signal reproduced from the recording / reproducing device 40 and the position signal corresponding thereto will be described. FIG. 4 shows an array of speakers of the speaker system 140. 4 (A) shows a front view of the speaker system 140, and FIG. 4 (B) shows a sectional view thereof. In this example, M
The individual speakers SP are arranged in a concave shape.

This speaker array itself has been proposed by Olson (for example, Olson, Nishimaki, "Acoustic Engineering", 1959). In this speaker system, if a large number of speakers SP are arranged in an array and the delay amount of an applied signal is adjusted to form a focal point F of a sound wave at one point in space, the sound pressure is extremely high at one point near the listener. Based on the fact that a point is synthesized and a situation occurs where an actual sound source exists at this point, and a sound image is perceived at that position in terms of auditory psychology (for example, Komiyama et al.
"Development of sound image perspective control technology for 3D television", N
HK Giken R & D, February 1991, No. 2, page 1
0-14). Generally speaking, M speakers SP are arranged as shown in FIG.

FIG. 5 shows a perspective view of a device 100B for reproducing the first video and sound. In FIG. 5, the TV monitor device 170 has a TV monitor display unit 17. The TV monitor display unit 171 displays the first image pickup device 61 or the second image pickup device 61.
The image captured by the image pickup device 62 is displayed. A sound source displayed on the TV monitor display unit 171 such as the TV monitor display unit 171 of the piano 1.
The position at is specified by the relationship between the position of the first imaging device 61 and the position of the piano. The directivity of the beam 191 is controlled by the speaker 181 to 183 based on the sound source position information, and the direction of the corresponding sound is controlled so as to direct the sound source as a reproduced image on the display surface of the TV monitor display unit 171. As a result, the actual sound image position 17 is set at the position where the beam 191 at the position corresponding to the sound source position on the surface of the TV monitor display unit 171 is directed.
2 is defined, and the viewer feels as if the sound is emitted from the image position of the sound source displayed on the TV monitor display unit 171. That is, a real sound image is formed on the TV monitor display unit 171.

FIG. 6A is a perspective view of a device 100C for reproducing the second image and sound, and FIG. 6B is a transparent electrode plate 20.
0 shows a plan view of 0. The TV monitor device 170 shown in FIG. 6A is similar to the TV monitor device 170 shown in FIG. 7. The transparent electrode plate 200 is provided on the front surface of the TV monitor display unit 171.
Is provided. This transparent electrode plate 200 is a substantially transparent sheet member vibrated by an electric voice signal, as disclosed in the utility model registration application “speaker device” already filed by the applicant on December 17, 1983. Are formed, and sound is emitted from a position in front of the front surface. The transparent electrode plate 200 is divided into a plurality of sections 201 to 209,
Certain of these sections are selectively driven. Selection of the sections 201 to 209 is performed in the same manner as described with reference to FIG. 5, corresponding to the sound source displayed on the TV monitor display unit 171. As a result, the sound is radiated from the portion of the TV monitor display unit 171 on which the sound source is displayed, and the actual sound image is present at the corresponding position on the display screen of the TV monitor display unit 171.

FIG. 7A is a perspective view of a device 100D for reproducing the third image and sound, and FIG. 7B is a sectional view thereof. The sound and video reproducing device 100D uses a color liquid crystal display 210 as a display device. Each pixel 221 to 224 of the color liquid crystal display 210 is composed of R, G and B,
Sound output holes 211, 212, and 213 are formed around these pixels. Small speaker 2 from the sound output hole 213
The sound output from 31 is output. As described with reference to FIG. 6, the selection of the small speaker 231 is performed by the color liquid crystal display 2
It is performed corresponding to the image position of the sound source displayed on the screen 10.
Also in this embodiment, the corresponding sound is output from the sound source displayed on the screen of the color liquid crystal display 210, and the real sound image is present there.

FIG. 8 shows a sectional view of an apparatus 100E for reproducing a fourth image and sound. This sound and video reproduction device 1
00E is a projection display type display device, and an image is projected from the projector 250 onto the screen 260. A large number of sound output holes 261 to 269 are perforated in the screen 260, and the speakers 271 to 27 are provided behind the holes.
79 are provided. The speaker corresponding to the sound source displayed on the screen 260 is driven, and the screen 260
The corresponding sound is output from the image position displayed in.

FIG. 9 shows a sectional view of an apparatus 100F for reproducing a fifth image and sound. In this embodiment as well, similar to the display device shown in FIG.
1 is a display device of a projection display system for projecting an image on
Vibrating members 321-arranged on the back surface of the screen 311
The vibrating plate 313 is vibrated by 326, and the screen 3
The position corresponding to the sound source displayed on 11 vibrates, and sound is output from there. Therefore, also in this example, the actual sound is emitted from the sound source displayed on the screen 311.

Next, a second embodiment of the present invention will be described with reference to FIG.
Will be described with reference to. The same parts as those in the first embodiment described above are designated by the same reference numerals and the description thereof will be omitted. In FIG. 10, the camera device 80 of the second embodiment has a camera 81, but without using a multiplexer.
The position information MP ₁ , MP ₂ , ..., MP _{M of the} microphones and the video signal V 1 are independently output to the recording / reproducing device 40 without being multiplexed. As for the reproduction method of the audio signal and the corresponding position signal, the speaker system described above is used as in the first embodiment.

Next, FIG. 11 shows the third embodiment of the present invention.
Will be described with reference to. The same parts as those in the first embodiment described above are designated by the same reference numerals and the description thereof will be omitted. In the sound collecting device shown in FIG. 11, the microphones 11 arranged independently for each (N) sound source to be collected,
12 and 13 have variable directivity. Although the camera device 85 of the third embodiment has the camera 81,
The position information MP ₁ , MP ₂ , ..., MP _{M of} the microphones and the video signal V 1 are independently output to the recording / reproducing device 40 without using a multiplexing device. As described above, in the third embodiment, the position information (directivity characteristic) of the (M) microphones that require position information among the microphones and the video signal V1 captured by the camera 86 are displayed on the video from the monitor 60. The sound source position information VP is output to the recording / reproducing device 40. As for the reproduction method of the audio signal and the corresponding position signal, the speaker system described above is used as in the first embodiment. As for the reproduction method of the audio signal and the corresponding position signal, the speaker system described above is used as in the first embodiment.

Next, a fourth embodiment of the present invention will be described with reference to FIG.
Will be described with reference to. The same parts as those of the first to third embodiments described above are designated by the same reference numerals and the description thereof will be omitted. In the sound collecting device in FIG. 12, the microphone devices MD1 and MD2 are independently arranged for each (N) sound sources desired to be collected. The microphone device MD
1 and MD2 are microphones 11 and 12 having variable directivity, and position detection devices 21 and 22 that provide information indicating the direction of a sound source in correspondence with the microphones.
Further, a multiplexer M for multiplexing the sound signal S _{1 of the} macrophone 11 and the position information MP ₁ of the position detecting device 21.
X1, the sound signal S _{2 of the} macrophone 12 and the position detection device 2
Multiplexer MX for multiplexing the _second position information MP ₂
2 and are built in.

Further, the microphone devices MD1 and MD2 are built in the camera device 90 in the fourth embodiment, and the microphones 11 and 12 are the cameras 9 respectively.
It is directed to the direction of the sound source with 1 as the center. Therefore, the position detection devices 21 and 22 output information (angle) indicating the directions of the sound sources to which the respective microphones 11 and 12 are directed. The camera device 90 further has a built-in multiplexer 92, and the multiplexer MX of the microphone devices MD1 and MD2 described above.
1 and MX2 output multiplexed signals OM ₁ and OM ₂ , respectively,
The video signal V1 captured by the camera 91 is multiplexed and output to the recording / reproducing device 40 as a multiplexed signal OMM. Therefore, the multiplexed signal OMM is multiplexed with information including video, audio, position, direction and size.

As described above, the camera device 9 of the fourth embodiment
At 0, independently for each sound source direction you want to pick up,
A variable directivity microphone is provided, and the output signal of the microphone and the position information (direction of the sound source) of the microphone are simultaneously multiplexed and output together with the video signal V1. As for the reproduction method of the audio signal and the corresponding position signal, the speaker system described above is used as in the first embodiment. As for the reproduction method of the audio signal and the corresponding position signal, the speaker system described above is used as in the first embodiment.

The embodiment of the present invention is not limited to the above-described embodiment, but various modifications can be made. Further, in implementing the present invention, the above-described embodiments can be appropriately combined.

[0039]

As described above, according to the camera device of the present invention, the following effects can be obtained. Since the sound pickup (sound source) position information is sent at the same time as the sound signal and the video signal, a more realistic AV (sound-video) reproduced sound field can be reproduced by using an actual sound image during reproduction. Since the sound source position information on the video is transmitted simultaneously with the relative position coordinates on the video together with the absolute coordinates of the camera device, a more realistic AV reproduced sound field can be reproduced. Since the camera device that records video and the microphone that collects audio are integrated, it becomes easy and accurate to associate information about the sound source position on the video, its size, etc. with the video information, and it is easy to handle. You can easily. Various sound source position information is video signal and audio information,
Since they are independently and simultaneously transmitted by the camera device, it is possible to use only the audio information and the video information independently, and it is extremely easy to change only the sound source position information, which is easy to edit.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a sound collecting device to which a camera device according to a first embodiment of the present invention is applied.

FIG. 2 is a diagram illustrating frequency multiplexing processing of position information or sound information in the sound collecting device shown in FIG.

3 is a diagram for explaining a time division multiplexing process of position information and sound information in the sound pickup device shown in FIG.

FIG. 4 is a diagram showing an arrangement of a plurality of speakers, (A)
Is a front view and (B) is a sectional view thereof.

FIG. 5 is a perspective view of a first embodiment of a reproduction system image and sound reproduction apparatus of the present invention.

FIG. 6 is a perspective view of a second embodiment of a reproduction system image and sound reproduction apparatus of the present invention, (A) is an overall perspective view,
(B) is a front view of the transparent electrode plate.

FIG. 7 is a perspective view of a third embodiment of the image and sound reproducing apparatus of the reproducing system of the present invention, (A) is a partially enlarged view,
(B) is a sectional view thereof.

FIG. 8 is a cross-sectional view of a fourth embodiment of the video and sound reproducing apparatus of the reproducing system of the present invention.

FIG. 9 is a sectional view of a fifth embodiment of the image and sound reproducing apparatus of the reproducing system of the present invention.

FIG. 10 is a block diagram showing a configuration of a sound collecting device to which the camera device of the second embodiment of the present invention is applied.

FIG. 11 is a block diagram showing a configuration of a sound collecting device to which the camera device of the third embodiment of the present invention is applied.

FIG. 12 is a block diagram showing a configuration of a sound collecting device to which a camera device of a fourth embodiment of the present invention is applied.

[Explanation of symbols]

11, 12, 13, 14, 15 Microphones 21, 22, 23, 53, Position detecting devices 31, 32, 33, 92, MX1, MX2 Multiplexers 40 Recording / reproducing devices 50, 80, 85, 90 Camera devices 51, 81, 86,91 camera 52 position information / video information multiplexing device 60 monitor 70 position information video output separation device

Claims (4)

[Claims] 1. A camera device comprising: an image pickup means for photographing a sound source together with a background, and outputting sound source position information on the photographed video together with a video signal. 2. A multiplexing means for multiplexing the sound source position information with the video signal.
The described camera device. 3. The camera device according to claim 1, wherein the sound source position information is output independently of the video signal. 4. A sound collecting means for collecting the sound of the sound source and outputting it as a sound signal; and a sound collecting means arranged in the vicinity of the sound collecting means.
A position detecting means for detecting the position of the sound source or the sound collecting means and outputting it as position information, and multiplexing the sound source position information by the imaging means, the audio signal by the sound collecting means and the position information by the position detecting means. 2. The camera device according to claim 1, wherein the camera device outputs the data separately or independently.