JP7070910B2 - Video conference system - Google Patents

Description

The present invention relates to a video conference system, particularly a video conference system with a high sense of presence.

Conventionally, for example, Patent Document 1 is known as a document for improving the sense of presence in a conference system. The telephone conference system disclosed in Patent Document 1 is a telephone conference system that conducts a conference by remote communication between a plurality of bases, and is a means for communicating with each other using stereo headphones or a stereo earphone and a microphone. It is characterized in that each of the conference participants is provided with a rendering processing means for arbitrarily setting the sound image position of the speaker. That is, the position of the speaker is automatically determined in advance, or the setter arbitrarily determines the head-related transfer function, which is close to the direction of the speaker and is measured in advance, and is convoluted into the microphone signal of the speaker. By doing so, the voice can be heard from the direction of the speaker. Each conference participant wears a headset that integrates a microphone and headphones.

Here, a conventional example of a sound image localization method in a conference system will be described. For example, as sound collection in a conference system, one omnidirectional microphone placed on a conference table or a single / omnidirectional microphone can be used to simultaneously or individually listen to sound in all directions. There is a way to pick up the sound. A signal that compresses (encodes) the collected sound, transmits it to a remote conference room through a network such as ISDN, an in-house LAN line, or the Internet, restores (decodes) the collected sound, and returns it to audio data. Is amplified by an amplifier (amplifier) as needed, and the sound is loudened from the speaker placed on the table in the other party's conference room, the wall of the conference room, or the ceiling.

In the sound image localization method in other conference systems, as a method of reproducing one sound field in another space, a method of matching acoustic physical quantities between the sound receiving region and the reproduction space in the original space (Kirchhof-Helmholtz integral theorem). Method based on) is used. This method is an example using the boundary sound field control method. It is characterized by obtaining a speaker signal that reproduces an acoustic physical signal on the boundary of a sound receiving region in a reproduction space. Basically, if a large number of microphones are placed at the boundary of the sound receiving area in the original space to measure the sound pressure and the sound pressure gradient, and this is played back by the speaker placed at the boundary of the playback space, the sound field is directly generated in the playback space. Can be reproduced.

Further, as an example of the immersive reproduction method, there is also a method called WFS (Wave Field Synthesis: wave field synthesis method). WFS is a 3D sound system that calculates / renders in real time according to how the sound is transmitted, the delay between the right and left ears that receive the sound, the space size of the sound field, etc., and outputs it to the speaker array system. To say. Since there is no sweet spot in the WFS method, it is said that a realistic surround effect can be enjoyed without being restricted by the viewing place, and a sound field space similar to the reality can be generated as much as possible.

Japanese Unexamined Patent Publication No. 2006-279492

Here, in order to enhance the presence of the conference in a system such as a video conference system in which the voice is inseparably related to the speaker, it is necessary for the listening side to easily identify the speaker. be. That is, for example, in order to improve the presence of a conference in a video conference system with a remote location, sound image localization for matching the direction of the voice of the speaker (speaker) with the direction of the listener's vision. A loudspeaker system is required.

In this regard, in the telephone conference system disclosed in Patent Document 1, a head-related transfer function in the direction of the speaker is prepared in advance, but since the position of the sound image does not change even if the speaker moves the position, the speech is made. The visual position of the person and the position of the sound image may differ, which may cause a sense of discomfort. Moreover, since the headset must be used, there is a problem that the user may feel annoyed.

Further, in the above-mentioned method using a single microphone, since the loudened voice does not include information on the direction of the voice, the voice can be heard from the direction of the speaker. You can roughly tell who is speaking by watching the video on the TV, but it feels strange because the direction of the speaker and the direction of hearing the sound do not match.

On the other hand, in order to realize the method using the above boundary sound field control method, it is necessary to arrange a large number of monopole microphones and dipole microphones in the original sound field, and a large number of speakers in the reproduced sound field, for example, in the horizontal direction. In order to reproduce the sound field up to a frequency of 8 kHz, it is necessary to arrange microphones and speakers every about 2 cm. Therefore, there is a problem that the cost becomes high.

Further, since the WFS system uses a large number of microphones, speakers, DSPs (Digital Signal Processors) and the like, there is a problem that the cost is high and the system becomes large-scale and complicated.

The present invention has been made to solve the above problems, and an object of the present invention is to realize a video conference system having a high sense of presence with a simple configuration and at low cost.

In order to achieve the above object, the video conferencing system of the first aspect generates an ambisonic signal based on a plurality of microphones that pick up the sound of the speaker and a voice signal picked up by the plurality of microphones. An image capture of an image including a generation unit, a reproduction unit that reproduces an audio signal from the ambisonic signal received from the generation unit, a plurality of speakers that enhance the audio signal received from the reproduction unit, and a speaker. Participants who participate in the conference while viewing the video of the display unit and the sound from the plurality of speakers, including the image pickup unit that outputs a signal and the display unit that displays the image pickup signal received from the image pickup unit. The direction of the perceived sound from the plurality of speakers coincides with the direction of the speaker displayed in the video.

In the video conferencing system of the first aspect, the ambisonics signal is generated by the generation unit based on the audio signals picked up by the plurality of microphones, and the audio signal is reproduced from the ambisonics signal received from the generation unit by the reproduction unit. The audio signal received from the reproduction unit is amplified by a plurality of speakers, the image pickup signal obtained by capturing the image including the speaker is output by the image pickup unit, and the image pickup signal received from the image pickup unit is displayed on the display unit. The direction of the sound from the plurality of speakers perceived by the participants participating in the conference while watching the image of the display unit and the sound from the plurality of speakers matches the direction of the speaker displayed in the image. This makes it possible to realize a video conference system with a high sense of presence with a simple configuration and at low cost.

The video conference system of the second aspect is the one in which the plurality of microphones and the image pickup unit are arranged adjacent to each other in the video conference system of the first aspect.

In the video conference system of the second aspect, a plurality of microphones and an image pickup unit are arranged adjacent to each other. This makes it easier to associate the audio signal using the Ambisonics method with the video signal.

The video conference system of the third aspect is the video conference system of the first aspect or the second aspect, in which the plurality of speakers are arranged so as to surround at least a part of the periphery of the plurality of speakers. Is.

In the video conference system of the third aspect, the plurality of speakers are arranged so as to surround at least a part of the periphery of the plurality of speakers. This makes it easier to localize the sound image by the Ambisonics method.

The video conference system of the fourth aspect is such that in the video conference system of the third aspect, the plurality of speakers are not arranged at a position where it is predicted that none of the voices of the plurality of speakers will arrive. be.

In the video conference system of the fourth aspect, the speaker is not arranged at a position where it is predicted that none of the voices of the plurality of speakers will arrive. This makes it possible to reduce the number of speakers.

The video conferencing system according to a fifth aspect is the video conferencing system according to any one of the first to fourth aspects, wherein the generator is a transmitter amplifier circuit that amplifies the voice signal from the plurality of microphones. The reproduction unit includes an analog-to-digital conversion circuit that converts the output of the transmission-side amplifier circuit, which is an analog signal, into a digital signal, and a coding circuit that encodes the output of the analog-to-digital conversion circuit into the ambisonics signal. A decoding circuit that decodes the ambisonics signal, a digital-to-analog conversion circuit that converts the output of the decoding circuit that is a digital signal into an analog signal, and an amplifier that amplifies the output of the digital-to-analog conversion circuit and sends it to the plurality of speakers. It is provided with a receiving side amplifier circuit.

In the video conferencing system of the fifth aspect, the generator has an amplifier circuit on the transmitting side that amplifies audio signals from a plurality of microphones, and an analog-to-digital conversion circuit that converts the output of the amplifier circuit on the transmitting side, which is an analog signal, into a digital signal. It also has a coding circuit that encodes the output of the analog-to-digital conversion circuit into an ambisonic signal, and the playback unit converts the output of the decoding circuit that decodes the ambisonic signal and the output of the decoding circuit that is a digital signal into an analog signal. It is equipped with a digital-to-analog conversion circuit and a receiving side amplifier circuit that amplifies the output of the digital-to-analog conversion circuit and sends it to a plurality of speakers. This makes it possible to generate and reproduce an Ambisonics signal suitable for the environment of the conference room.

In the video conference system of the fifth aspect, the video conference system of the sixth aspect connects the generation unit and the reproduction unit, and the image pickup unit and the display unit, respectively, by a transmission line. Further comprising a network, the coding circuit comprises a transmitting circuit that outputs the ambisonics signal to the transmission line, and the decoding circuit comprises a receiving circuit that receives the ambisonics signal from the transmission line.

The video conference system of the sixth aspect further includes a network in which each of the generation unit and the reproduction unit and the image pickup unit and the display unit are connected by a transmission line, and the coding circuit transmits an ambisonics signal. A transmission circuit that outputs to a line is provided, and a decoding circuit includes a reception circuit that receives an ambisonics signal from the transmission line. This makes it possible to construct a video conference system between remote locations.

The video conference system of the seventh aspect is the video conference system of the fifth aspect or the sixth aspect, in which the decoding circuit decodes into the ambisonics signal according to the arrangement of the plurality of speakers. be.

In the video conference system of the seventh aspect, the decoding circuit decodes into an ambisonics signal according to the arrangement of a plurality of speakers. This makes it possible to realize the Ambisonics method using the existing speaker system in the conference room.

The video conference system of the eighth aspect is the video conference system of any one of the first aspect and the third to seventh aspects, wherein the image pickup unit is each of the plurality of participants located in a ring shape. And the display unit, the plurality of microphones are arranged in the center of the plurality of speakers located in an annular shape, and the generation unit positions the plurality of microphones at the positions of the imaging unit. After the coordinate conversion, the ambisonics signal is generated.

In the video conference system of the eighth aspect, the image pickup unit is arranged between each of the plurality of participants located in the ring and the display unit, and the plurality of microphones are placed in the center of the plurality of speakers located in the ring. Arranged, the generator generates an ambisonics signal after coordinate-converting the positions of the plurality of microphones to the positions of the image pickup unit. This makes it possible to collect sound by the Ambisonics method in line with the image while making the sound collection condition by the microphone better.

According to the present disclosure, a video conference system with a high sense of presence can be realized with a simple configuration and at low cost.

It is a block diagram which shows an example of the structure of the video conference system which concerns on 1st Embodiment. It is a block diagram which shows an example of the structure of the audio transmission unit, the audio reception unit, the video transmission unit, and the video reception unit of the video conference system according to the first embodiment. (A) and (b) are diagrams showing an example of an Ambisonics microphone and (c) an example of a multi-microphone according to an embodiment. It is a top view which shows an example of the arrangement of the multi-microphone which concerns on embodiment. It is a block diagram which shows an example of the structure of the video conference system which concerns on 2nd Embodiment.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. In the video conference system according to the prior art, it was often unclear which of the participants displayed in the video was speaking, and it lacked a sense of realism. You can get a sense of realism as if you were having a meeting in the same meeting room. In the present embodiment, an Ambisonics system is used for the audio transmission system in order to realize a video conference system having a high sense of presence with a simple configuration and at low cost. The Ambisonics method is a recording technology that can record, mix, and reproduce audio in all directions of 360 degrees in a three-dimensional space. One of the features of the Ambisonics method is that audio can be recorded by a method independent of the speaker layout during playback, and can be decoded according to the speaker layout in the playback environment. Furthermore, a major feature of the Ambisonics method is that the sound field can be rotated freely and without deterioration by signal processing before reproduction.

[First Embodiment]
The video conference system 10 according to the present embodiment will be described with reference to FIGS. 1 to 4. In the present embodiment, the video conference system 10 is arranged in the conference rooms 12A and 12B (hereinafter, collectively referred to as "conference room 12"), and the conference is held with the participants 32A (one or more people) in the conference room 12A. An example of a case where a conference using audio and video as a medium is held with a participant 32B (one or more people) in the room 12B will be described as an example. Hereinafter, when the participants 32A and 32B are collectively referred to, they are referred to as "participant 32". The arrangement of the participants is not particularly limited, but in the present embodiment, as shown in FIG. 1, it is assumed that the participants are arranged in a ring around the table 34A or 34B (hereinafter, collectively referred to as “table 34”).

As shown in FIG. 1, the video conference system 10 includes audio transmission devices 40A and 40B (hereinafter, collectively referred to as “audio transmission device 40”), and video transmission devices 42A and 42B (hereinafter, collectively referred to as “video”). It is configured to include a transmission device 42 "). The voice transmission device 40A is a device that transmits the voice in the conference room 12A from the conference room 12A to 12B, and the voice transmission device 40B is a device that transmits the voice in the conference room 12B from the conference room 12B to 12A. Further, the video transmission device 42A is a device that transmits the video in the conference room 12A from the conference room 12A to 12B, and the video transmission device 42B is a device that transmits the video in the conference room 12B from the conference room 12B to 12A. .. Therefore, since the audio transmission device 40 and the video transmission device 42 are symmetrical with respect to the conference rooms 12A and 12B, the audio transmission device 40A and the video transmission device 42A on one side will be described below.

As shown in FIG. 1, the voice transmission device 40A includes a microphone 24A, a voice transmission unit 16A, a voice reception unit 18A, a plurality of speakers 30B (20 cases are exemplified in FIG. 1), and a control unit 80A. ing. On the other hand, the video transmission device 42A includes a video camera 26A, a video transmission unit 20A, a video reception unit 22A, and a screen 28B.

Here, in the present embodiment, the audio transmission unit 16A and the audio reception unit 18A, and the video transmission unit 20A and the video reception unit 22A are connected via the network 70. The network 70 is not particularly limited, for example, LAN (Local Area Network), WAN (Wide Area Network), Internet, ISDN (Integrated Services Digital Network), etc., and may be classified into wireless and wired. In the present embodiment, since the conference rooms 12A and 12B are exemplified as remote locations from each other, the embodiment in which the network 70 intervenes will be described as an example. However, the conference rooms 12A and 12B are adjacent to each other. When located at a short distance, the audio transmission unit 16A and the audio reception unit 18A, and the video transmission unit 20A and the video reception unit 22A may be directly connected without interposing the network 70.

The microphone 24A collects the sound in the conference room 12A. As described above, since the ambisonics method is used as the audio localization method in the present embodiment, the microphone 24A also uses an ambisonics-compatible microphone. Details of the microphone 24A will be described later. As will be described later, the microphone 24A (24B) according to the present embodiment is configured to include a plurality of individual microphones 36.

The speaker 30B is arranged so as to surround the participant 32B along the wall surface in the conference room 12B, and the voice in the conference room 12A transmitted via the voice transmission device 40A is loudened in the conference room 12B. In this embodiment, an embodiment in which 20 speakers 30B are arranged will be described as an example, but the present invention is not limited to this, and an appropriate number is selected in consideration of the accuracy of identification of the speaker (speaker) and the like. good.
Further, it does not necessarily have to be arranged on the wall surface, and may be, for example, on the ceiling or the table 34. In the following, when the speakers 30A and 30B are generically referred to, they are referred to as "speaker 30".

The voice transmission unit 16A receives the voice signal picked up by the microphone 24A, performs predetermined processing, and sends it out to the network 70. On the other hand, the voice receiving unit 18A receives the voice signal from the network 70, performs a predetermined process, and outputs the voice signal to the speaker 30B.

The video camera 26A is a camera that captures images centered on the participants in the conference room 12A, and is not particularly limited in this embodiment, and a general video camera can be used. The video transmission unit 20A transmits a video signal from the video camera 26A to the network 70. The video receiving unit 22A receives a video signal from the network 70 and displays it on the screen 28B.

The control units 80A and 80B are configured to include a CPU, ROM, RAM and the like (not shown), respectively. The control unit 80A controls the audio transmission device 40A and the video transmission device 42A in an integrated manner, and the control unit 80B controls the audio transmission device 40B and It controls the video transmission device 42B in an integrated manner. More specifically, for example, the encoding method and the decoding method in the encoder 54 and the decoder 60 described below are set and changed.

With reference to FIG. 2, audio transmitting units 16A and 16B (hereinafter, collectively referred to as “audio transmitting unit 16”), audio receiving units 18A, 18B (hereinafter, collectively referred to as “audio receiving unit 18”), video. The transmission units 20A and 20B (hereinafter, collectively referred to as “video transmission unit 20”) and the video reception units 22A and 22B (hereinafter, collectively referred to as “video reception unit 22”) will be described in more detail.

As shown in FIG. 2A, the audio transmitter 16A includes an amplifier 50, an A / D (analog-to-digital) converter 52, an encoder (denoted as “ENC” in FIG. 2A) 54, and a transmitter 56 (FIG. 2A). In 2 (a), it is configured with "TX"). The A / D converter 52 may include an antialiasing filter. Since the configuration of the voice transmission unit 16B is the same as that of the voice transmission unit 16A, the description thereof will be omitted.

The amplifier 50 is an amplifier that amplifies the audio signal from the microphone 24A. However, the amplifier 50 may be provided when necessary according to the level of the audio signal and the like. The A / D converter 52 converts an audio signal from the microphone 24A, which is an analog signal, into a digital signal. The encoder 54 encodes a digital audio signal from the A / D converter 52 into an ambisonics signal (ambisonics signal) to generate an ambisonics signal. The transmitter 56 sends the ambisonics signal received from the encoder 54 to the network 70. The transmitter 56 may transmit the ambisonics signal after converting it into a code suitable for the transmission format in the network 70.

On the other hand, as shown in FIG. 2 (b), the audio receiver 18A includes a receiver 58 (denoted as “RX” in FIG. 2 (b)), a decoder 60 (denoted as “DEC” in FIG. 2 (b)), and D. It includes a / A (digital-to-analog) converter 62 and an amplifier 64. The decoder 60 may include an antialiasing filter. Since the configuration of the voice receiving unit 18B is the same as that of the voice receiving unit 18A, the description thereof will be omitted.

The receiver 58 receives an audio signal (ambisonics signal) transmitted via the network 70. If the audio signal is encoded by the transmitter 56, the audio signal is decoded. The decoder 60 decodes the ambisonics signal received by the receiver 58 and reproduces the audio signal. In this decoding, decoding is performed in consideration of the number, arrangement, and the like of the speakers 30 in the conference room 12. The D / A converter 62 converts an audio signal, which is a digital signal reproduced by the decoder 60, into an analog signal. The amplifier 64 amplifies the audio signal from the D / A converter 62, and drives the speaker 30B in the subsequent stage by the audio signal. However, the amplifier 64 may be provided when necessary according to the level of the audio signal and the like. The speaker 30B amplifies the audio signal toward the participants in the conference room 12B.

On the other hand, as shown in FIG. 2C, the video transmission unit 20A includes the transmitter 66. The transmitter 66 transmits a video signal including the participants in the conference room 12A received from the video camera 26A to the network 70. The transmitter 66 may convert the video signal into a code suitable for the transmission format of the network 70. The video transmission unit 20B has the same configuration as the video transmission unit 20A.

On the other hand, as shown in FIG. 2D, the video receiving unit 22A includes the receiver 68. The receiver 68 receives a video signal from the network 70 including the participants in the conference room 12A. If the video signal is encoded by the transmitter 66, it is decoded. The video receiving unit 22B has the same configuration as the video receiving unit 22A.

In the video conference system 10 according to the present embodiment, the microphones 24A and 24B (hereinafter, collectively referred to as “microphone 24”) and the video cameras 26A and 26B (hereinafter, collectively referred to as “video camera 26”) are arranged. The position is not particularly limited, but as an example, as shown in FIG. 1, it may be arranged in front of the participants arranged in a ring shape (in other words, between the table 34 and the screen 28). At that time, it is preferable that the microphone 24 and the video camera 26 are arranged close to each other.

By the operation of the audio transmission device 40A and the video transmission device 42A as described above, the audio signal and the video signal in the conference room 12A are transmitted to the conference room 12B. Similarly, by the operation of the audio transmission device 40B and the video transmission device 42B, the audio signal and the video signal in the conference room 12B are transmitted to the conference room 12A, and a video conference between the conference rooms 12A and 12B is performed.

In this embodiment, a mode in which one video camera 26A and one 26B are used in the conference rooms 12A and 12B, respectively, has been described as an example, but the present invention is not limited to this. It may be transmitted as a 3D video signal by using two video cameras each as the video cameras 26A and 26B. In that case, it is possible to listen as a 3D image by using a 3D encoder on the receiving side.

Next, the microphone 24 according to the present embodiment will be described in more detail with reference to FIG. FIG. 3 (a) is the most basic ambisonics microphone, which includes four individual microphones (microphone capsules) 36. The direction of each individual microphone 36 is the normal direction of the four faces of the regular tetrahedron. The entire signal picked up by these four microphone capsules is called an A format signal. In the Ambisonics method, this A format signal is further converted into a B format signal for use. The B format signal is a signal represented by a case where the A format signal is picked up with a predetermined sound pick-up directivity characteristic.

For example, the 0th-order sound collection directivity of B format corresponds to the sound collection signal when sound is picked up by an omnidirectional microphone, and the 1st-order sound collection directivity is a bidirectional microphone that is spatially orthogonal to each other. Corresponds to the sound pick-up signal when sound is picked up. That is, in the Ambisonic microphone, the amount corresponding to the sound pressure and the sound pressure vector can be obtained. FIG. 3B synthesizes and shows first-order directivity characteristics in the X, Y, and Z directions. Further, the circle indicated by the dotted line with the reference numeral “W” in FIG. 3 (b) indicates the 0th-order sound collection directivity.

The microphone 24 according to the present embodiment is not limited to the ambisonics microphone shown in FIG. 3 (a), but is a multi-microphone in which a large number of individual microphones 36 are arranged on a spherical surface as shown in FIG. 3 (c). It may be a microphone 24. Alternatively, as shown in FIG. 4 (a), a large number of individual pieces (11 cases are exemplified in FIG. 4 (a)) arranged at arbitrary positions in the vicinity of the distance d / 2 from the center. It may be the microphone 24 composed of the microphone 36.

Further, in the present embodiment, only the horizontal localization of the sound image needs to be reproduced. Therefore, as shown in FIGS. 4B to 4D, the microphone 24 having the individual microphones 36 arranged on the horizontal plane may be used. .. As the number of individual microphones 36 increases, the order of Ambisonics increases, and the localization accuracy of the sound image improves. 4 (b) is a first-order example using four individual microphones 36, FIG. 4 (c) is a sixth-order example using 14 individual microphones 36, and FIG. 4 (d) is nine. The following fourth-order examples using the individual microphones 36 of the above are shown. The individual microphone 36 in FIG. 4 is an omnidirectional microphone as an example.

Next, a method of generating and reproducing an ambisonics signal in the present embodiment will be described in detail.

で平面波が作る音圧ｐを球面調和展開すると音圧ｐは、下記（１）式で表される。 When a plane wave of amplitude Q arrives from the ψ and φ directions (ψ is the azimuth and φ is the elevation angle with 0 ° directly above).

When the sound pressure p created by the plane wave is expanded in spherical harmonics, the sound pressure p is expressed by the following equation (1).

である。 In the above equation (1),

Is.

When the above equation (1) is cut off at the nth order, expressed in a matrix, and the plane wave of the sound source (mainly the voice from the speaker in the present embodiment) is multiplied by the spherical harmonic function, the following equation (2) is obtained.

上記（３）式における左辺は平面波の音場の指向性を表し、中辺は任意の

におけるマイクロホン２４の

から導出できる音場の指向性を表す。右辺の太文字のＢは平面波のアンビソニックス信号と称され、１次の場合は一般にＷ、Ｘ、Ｙ、Ｚの4つのアルファベットで表わされる。すなわち、本実施の形態に係る音声伝送装置４０のエンコーダ５４は、複数位置における個別マイクロホン３６により収音したマイクロホン信号（音声信号）からＢフォーマット信号を作成する。 The pseudo-inverse matrix can be calculated by appropriately arranging the columns of the sound receiving positions indicated by the bold letters r (r, θ, φ) so that they are independent. By multiplying both sides of the above equation (2) by the pseudo-inverse matrix of _XYr in bold letters from the left side, the following equation (3) is obtained.

The left side in the above equation (3) represents the directivity of the sound field of a plane wave, and the middle side is arbitrary.

Of the microphone 24 in

Represents the directivity of the sound field that can be derived from. The bold letter B on the right side is called a plane wave ambisonics signal, and in the case of the first order, it is generally represented by four alphabets W, X, Y, and Z. That is, the encoder 54 of the voice transmission device 40 according to the present embodiment creates a B format signal from the microphone signals (voice signals) picked up by the individual microphones 36 at a plurality of positions.

Next, the reproduction of the sound field by the nth-order ambisonics signal generated as described above will be described. In the present embodiment, the reproduction of the sound field is performed by the decoder 60 of the audio transmission device 40. The decoder 60 decodes the ambisonics signal and generates a drive signal for the speaker 30. Hereinafter, the method of decoding the ambisonics signal in the present embodiment will be described in detail.

なお、上記（４）式において、（θ_ｌ，φ_ｌ）は、原点から見たスピーカ３０の方向、ａ_ｌは、各スピーカ３０の入力信号（駆動信号）である。 When L speakers 30 are installed on the same spherical surface equidistant from the center, it is assumed that the sound waves radiated from these speakers 30 are plane waves, and the sound pressures produced by these speakers are expanded in spherical harmony, the following (4) ) Is obtained.

In the above equation (4), (θ _l , φ _l ) is the direction of the speaker 30 as seen from the origin, and a _is the input signal (drive signal) of each speaker 30.

Here, assuming that the sound pressure created by the plane wave and the sound pressure created by the L speakers 30 are equal (Equation (1) = Eq. (4)), the expansion is expressed as a truncated matrix by the nth order, and the orthogonality of the spherical harmonics is expressed. The following equation (5) can be obtained by using.

Therefore, the drive signal for driving the speaker 30 is derived by the above equation (5). The derivation at this time is performed in consideration of the number, arrangement, and the like of the speakers 30 in the conference room 12. The decoder 60 outputs the drive signal derived by the above equation (5) to each of the speakers 30. The speaker 30 is driven by the drive signal generated as described above, and the voice is loudened. By listening to the loudened voice signal through the speaker 30, the participants of the conference can feel as if the voice was emitted from the position of the speaker projected on the screen, which makes the speaker comfortable and easy. Can be identified in.

Further, according to the video conference system 10 according to the present embodiment, it is possible to maintain a state in which the directions of the visual sense and the voice are matched even if the speaker moves. Moreover, the video conference system 10 according to the present embodiment does not need to use additional equipment such as a headset. The inverse pseudo matrix (inverse filter) defined by the above equation can be calculated in advance by deciding the arrangement of the speakers 30, and by doing so, it is possible to reproduce in real time. Become.

The direction and distance of the speaker as seen from the microphone 24, which is an ambisonic microphone, and the direction and distance of the speaker felt by the conference participants may not always match the actual direction and distance. Although there may be a shift, the visual information is superior to the auditory information regarding the position of the sound image, so that the sound image is actually seen in the direction and distance of the visual. As a result, the participants in the conference room 12B can feel as if they are having a conference together with the conference room 12A by the image projected on the screen 28B and the aurally localized sound image. By transmitting signals from the conference room 12B to 12A in the same manner, the participants in the conference room 12A can feel as if they are having a conference together with the conference room 12B.

[Second Embodiment]
The video conference system 100 according to the present embodiment will be described with reference to FIG. The present embodiment is a mode in which the arrangement of the speakers 30 is changed in the video conference system 10 according to the above embodiment, and more specifically, the number of the speakers 30 is reduced. Therefore, the same reference numerals are given to the same configurations as those of the video conference system 10, and detailed description thereof will be omitted.

In the video conference system 100, the arrangement of the speakers 30A in the conference room 12A and the arrangement of the speakers 30B in the conference room 12B are performed as shown in FIG. In FIG. 5, a mode in which eight speakers 30 are arranged are exemplified, but the number may be appropriately changed in consideration of the distinctiveness of the speaker and the like, and the number of the speakers 30 may be changed as appropriate in the conference rooms 12A and 12B. It does not have to be the same number.

Here, a feature of the video conference system according to the present invention is that a plurality of different conference rooms are integrated to recognize video and audio as if they were one room. Basically, since the voice can be heard only from the direction of the speaker in the other conference room, the voice does not reach from the direction of the speaker having the difference between FIGS. 1 and 5 (hereinafter, “surplus speaker”). Taking this into consideration, it is possible to reduce the number of redundant speakers.

When obtaining the input signal of the speaker 30 represented by the formula (5), since the speaker 30 must be arranged so as to surround the listener in principle of Ambisonics, the formula (including the position of the surplus speaker) is included. 5) is calculated. After that, when actually constructing the system, only the speaker 30 shown in FIG. 5 is arranged without installing the surplus speaker in the direction of the speaker. This makes it possible to localize the sound image in the direction of the speaker while reducing the number of speakers 30. In other words, the plurality of speakers 30 are not arranged at positions where it is predicted that none of the voices of the plurality of speakers will arrive.

The signal processing method of the video conference system 100 according to the present embodiment will be described. The processing up to the formula (5) is the same as that of the video conference system 10. To both ears acquired in advance at the angle of the speaker 30 seen from the listener for each listener with respect to the speaker input signal a _obtained for the arrangement of each speaker 30 (θ _l , φ _l ). By convolving the head-related transfer functions _HL (θ _l , φ _l ) and _HR (θ _l , φ _l ) of, and adding them to the left and right, they are converted into binaural signals for each listener. If the position of the prepared head-related transfer function and the position of the listener are different, the one with the closest angle is used.

In each of the above embodiments, the microphone 24 is exemplified and described by arranging the microphone 24 between the table 34 and the screen 28 in which the participants of the conference are located in a ring shape, but the present invention is not limited to this. For example, the microphone 24 may be arranged in the center of the participants (the center of the gusset, the table 34) in which the microphone 24 is arranged in a ring shape. In this case, since the original ambisonics signal assumes the case where the microphone 24 is arranged at the position of the video camera 26, the audio signal picked up by the microphone 24 may be coordinate-converted to the position of the video camera 26. The coordinate conversion at that time may be performed before encoding with the encoder 54 shown in FIG. 2 (a).

10,100 Video conferencing system 12, 12A, 12B Conference room 16, 16A, 16B Audio transmission unit 18, 18A, 18B Audio reception unit 20, 20A, 20B Video transmission unit 22, 22A, 22B Video reception unit 24, 24A, 24B Microphone 26, 26A, 26B Video camera 28, 28A, 28B Screen 30, 30A, 30B Speaker 32, 32A, 32B Participant 34, 34A, 34B Table 36 Individual microphone 40, 40A, 40B Audio transmission device 42, 42A, 42B Video Transmission device 50 Amplifier 52 A / D converter 54 Encoder 56 Transmitter 58 Receiver 60 Decoder 62 D / A converter 64 Amplifier 66 Transmitter 68 Receiver 70 Network 80A, 80B Control unit

Claims (6)

Multiple microphones that pick up the speaker's voice,
A generation unit that generates an ambisonics signal based on the audio signals picked up by the plurality of microphones.
A reproduction unit that reproduces an audio signal from the ambisonics signal received from the generation unit, and a reproduction unit.
A plurality of speakers that enhance the audio signal received from the playback unit, and
An image pickup unit that outputs an image pickup signal that captures an image including the speaker, and an image pickup unit.
Includes a display unit that displays an image pickup signal received from the image pickup unit.
The direction of the sound from the plurality of speakers perceived by the participants participating in the conference while watching the video of the display unit and the sound from the plurality of speakers coincides with the direction of the speaker displayed in the video. It is a video conference system that
The imaging unit is arranged between each of the plurality of participants located in a ring shape and the display unit.
The plurality of microphones are arranged in the center of the plurality of speakers located in an annular shape.
The generation unit generates the ambisonics signal after coordinate conversion so that the audio signal picked up at the positions of the plurality of microphones becomes the audio signal picked up at the positions of the imaging unit.
Video conference system . The video conference system according to claim 1 , wherein the plurality of speakers are arranged so as to surround at least a part of the periphery of the plurality of speakers. The video conference system according to claim 2 , wherein the plurality of speakers are not arranged at a position where it is predicted that none of the voices of the plurality of speakers will arrive. The generator includes a transmitter amplifier circuit that amplifies the audio signal from the plurality of microphones, an analog-to-digital conversion circuit that converts the output of the transmitter amplifier circuit, which is an analog signal, into a digital signal, and an analog-to-digital conversion circuit. Equipped with a coding circuit that encodes the output of the ambisonics signal.
The reproduction unit amplifies the output of the decoding circuit that decodes the ambisonic signal, the digital-to-analog conversion circuit that converts the output of the decoding circuit that is a digital signal into an analog signal, and the output of the digital-to-analog conversion circuit. The video conferencing system according to any one of claims 1 to 3 , further comprising a receiving side amplifier circuit for sending to a plurality of speakers. Further, a network for connecting between the generation unit and the reproduction unit and between the image pickup unit and the display unit by a transmission line is further provided.
The coding circuit includes a transmission circuit that outputs the ambisonics signal to the transmission line.
The video conference system according to claim 4 , wherein the decoding circuit includes a receiving circuit that receives the ambisonics signal from a transmission line. The video conference system according to claim 4 or 5 , wherein the decoding circuit decodes into the ambisonics signal according to the arrangement of the plurality of speakers.

Images