JP2006339869A - Apparatus for integrating video signal and voice signal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus for integrally controlling a video signal and a voice signal capable of creating a space with a feeling of real life wherein a receiver side can eliminate deviation between a talker at a transmission side and voice or sound uttered by the talker, so that the apparatus can reproduce a state of the transmission side talker uttering the voice or sound as it is. <P>SOLUTION: A face region detection section 108 calculates face region positional information of a person from video information. A sound receiving direction determining section 105 prescribes an existing direction of a transmission side conference participant on the basis of the face region positional information and a zoom magnification of a camera 103 and the direction of the camera. A sound receiving section 104 acquires sound information from the prescribed existence direction. A sound receiving reproducing section 109 forms an image of the sound information around a face region of the conference participant at the transmission side displayed on a display apparatus 107 of a receiver side terminal on the basis of the face region positional information. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention transmits video signals and sound signals such as sounds and voices from spatially separated points such as videophones, and integrates the video signals and sound signals that create a realistic space using these signals. It relates to the device which controls automatically.

In recent years, with the spread of broadband networks, videophones using VoIP technology are rapidly spreading. Specifically, videophones using personal computers, videophones connected to televisions, and the like have begun to permeate.

Because such a videophone is a form of use in which you talk while watching the screen,
It is a hands-free call. However, since the echo canceller corresponding to the stereo call is difficult and expensive, the call voice is monaural. For this reason, as long as the conversation between the sending side and the receiving side is a one-to-one conversation, a good conversation is possible, but one-to-multiple, or multiple-to-multiple In the case of conversation, it is difficult for the receiving side to understand which speaker on the transmitting side is speaking only by video and voice. Further, when a plurality of people speak at the transmitting side at the same time, those voices are transmitted as monaural to the receiving side, so that the comprehension of the contents of the speech is significantly deteriorated at the receiving side.

On the other hand, some of the commercial videophone systems that are compatible with stereo calls using a stereo echo canceller have been realized to realize more realistic videophones. When the call voice is stereo, the atmosphere of the sending side is easier to communicate compared to monaural, and even if multiple speakers speak at the same time on the sending side, it is easy to understand the content of each speaker. is there. Furthermore, it is possible to realize a more realistic videophone by expanding the bandwidth of the call voice.

  However, there is a tendency that a gap is easily generated between the video displayed on the television screen and the sound reproduced from the speaker, which may be unnatural. For example, when the video is zoomed, a video in which a specific area is enlarged is displayed on the TV screen, but there is no change in the audio reproduced from the speaker, and the receiver side has a difference between the video and the audio. The gap becomes even larger, which makes the call feel unnatural.

As a method for eliminating such an unnatural feeling of communication, methods described in the following patent documents have been proposed. In this patent document 1, by analyzing the video, it is estimated what kind of space the other party is talking in, and the parameter for processing the acoustic signal is changed according to the estimation result. . For example, when the other party is talking in a large room, a realistic videophone is realized by selecting parameters that add reverberation and the like.
JP-A-7-131770

  However, even if such a process is performed, the problem that the video and the audio are shifted has not been solved.

Therefore, the present invention eliminates the difference between the image of the speaker on the transmitting side and the voice or sound emitted by the speaker on the receiver side, and the situation where the speaker on the transmitting side emits sound or sound as much as possible. It is an object of the present invention to provide an apparatus for controlling video signals and audio signals in an integrated manner that can be faithfully reproduced and can create a more realistic space.

  The video signal and sound signal integration apparatus according to the present invention includes a video acquisition unit that acquires a video signal, a sound acquisition unit that acquires a sound signal, the acoustic signal acquired by the sound acquisition unit, and the video acquisition unit. Transmitting means for transmitting the acquired video signal; receiving means for receiving the video signal and the acoustic signal transmitted by the transmitting means; and video display means for displaying the video signal received by the receiving means; An apparatus for integrating a video signal and an acoustic signal, comprising: an acoustic control unit that controls the acoustic signal received by the receiving unit; and an acoustic output unit that outputs the acoustic signal controlled by the acoustic control unit. A face position detecting means for detecting the face position of one or more persons on the video signal from the video signal acquired by the video acquiring means, and the face position detecting means. Person direction specifying means for specifying the presence direction of each person based on the face position of each person detected in the above and the video acquisition condition of the video acquisition means, and the sound acquisition means is provided by the person specifying means. Acquire acoustic signals from the identified direction of each person, and the acoustic control means corresponds to each person near the face position of each person on the video signal displayed on the display means. Each acoustic signal is imaged.

  The video signal and sound signal integration apparatus according to the present invention includes a video acquisition unit that acquires a video signal, a sound acquisition unit that acquires a sound signal, the sound signal acquired by the sound acquisition unit, and the video acquisition. Transmitting means for transmitting the video signal acquired by the means, receiving means for receiving the video signal and the acoustic signal transmitted by the transmitting means, and video display for displaying the video signal received by the receiving means A video signal and sound signal integrating device comprising: a sound control means for controlling the sound signal received by the receiving means; and a sound output means for outputting the sound signal controlled by the sound control means. A face position detecting means for detecting a face position of one or more persons on the video signal from the video signal acquired by the video acquiring means, and the face position detecting hand And a person direction specifying means for specifying the presence direction of each person based on the face position of each person detected by the image acquisition condition and the image acquisition condition of the image acquisition means, and the sound acquisition means includes sound from all directions. The sound control means obtains a sound signal from the direction of existence of each person based on information about the direction of existence of each person specified by the person specifying means from the sound signals from all directions. Are generated, and each of the generated acoustic signals corresponding to each person is imaged in the vicinity of the face position of each person on the video signal displayed on the display means.

  According to these inventions, it is possible to create a space full of realism that a speaker displayed on a display device is actually speaking on the spot.

  In the video signal and audio signal integration apparatus according to the present invention, the video acquisition condition of the video acquisition unit includes a zoom magnification of the video acquisition unit.

  In the video signal and audio signal integration apparatus according to the present invention, the video acquisition condition of the video acquisition means includes direction information that the video acquisition means is directed to.

  In the video signal and sound signal integration device according to the present invention, the sound acquisition means increases or decreases the volume level of the sound signal acquired according to the zoom magnification of the image acquisition conditions.

  Further, in the video signal and audio signal integration device according to the present invention, the audio control means increases or decreases the volume level of the audio signal output by the audio output means according to the zoom magnification of the video acquisition conditions. It is characterized by.

  According to the video signal and sound signal integration device according to the above two inventions, by increasing / decreasing the zoom magnification of the video acquisition means, the size of the person etc. displayed on the video display means is also increased / decreased, and the sound output is accordingly performed The volume level of the person's sound signal output by the means is also increased or decreased. Therefore, a more realistic space will be created.

According to the present invention, the receiver side eliminates the difference between the image of the speaker on the transmitter side and the voice or sound emitted by the speaker, and the situation where the speaker on the transmitter side emits voice or sound can be as much as possible. It is possible to provide an apparatus for integrated control of video and audio signals that can be faithfully reproduced and can create a more realistic space.

  The significance or effect of the present invention will become more apparent from the following description of embodiments.

  However, the following embodiment is merely one embodiment of the present invention, and the meaning of the term of the present invention or each constituent element is not limited to that described in the following embodiment. Absent.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention implemented in a hands-free videophone device will be described below with reference to the drawings.

  FIG. 1 is a diagram showing an outline of a configuration when a hands-free videophone which is one embodiment of the present invention is used for a video conference.

  In FIG. 1, in front of a hands-free videophone device on the transmission side (hereinafter referred to as a transmission-side terminal), a speaker utters voice or sound (hereinafter referred to as sound information) by utterance or the like. Then, the video of the speaker and the sound information uttered by the speaker are transmitted directly or via a network to a hands-free videophone device (hereinafter referred to as a receiver terminal) on the receiver side.

  The receiver terminal reproduces the video of the speaker on the transmitter side on the screen of the display device and outputs sound information emitted by the speaker from the vicinity of the face area of the speaker on the screen. Further, when the video on the transmission side is displayed with zoom-in, the volume level of the sound information to be output is also increased. By performing such processing, the receiving terminal can create a space full of realism that makes it feel as if the transmitting speaker is actually speaking on the screen. Conversely, on the transmitting side, a space full of realism similar to the receiving side is created. Thereby, it is easy to conduct a conversation and a more realistic video conference can be performed.

  FIG. 2 is a diagram showing a configuration of a hands-free videophone device which is one embodiment of the present invention.

  In FIG. 2, a hands-free videophone device 100 includes a transmission processing unit 101 that transmits video information and sound information to a receiving terminal, and a receiving processing unit 102 that reproduces video signals and sound information from the receiving terminal. The

  The transmission processing unit 101 includes acquisition and transmission of video information, a camera 103 that transmits camera information such as a zoom magnification of the camera and a direction in which the camera is facing, a sound receiving unit 104 that acquires and transmits sound information, and a sound reception The unit 104 includes a sound receiving direction determining unit 105 that determines at what volume level the sound information in which direction is acquired. For example, the sound receiving unit 104 is configured by arranging a plurality of small microphones having high directivity, and can acquire sound information in a desired direction at a desired volume level.

  The reception processing unit 102 includes a speaker 106 that outputs sound information transmitted from the transmission side terminal, a display device 107 that displays video information transmitted from the transmission side terminal, and video information transmitted from the transmission side terminal. A face area detection unit 108 that detects the face area of a person, and the sound information output from the speaker 108 is displayed on the display device 107 based on the position information of the face area detected by the face area detection unit 108. The sound receiving signal reproducing unit 109 is controlled so as to be output from the vicinity of the area.

  FIG. 3 is a diagram showing a configuration and a data flow when a video conference is performed using two hands-free video phone devices 100.

  In FIG. 3, the camera 103 of the transmitting terminal captures a meeting scene including a person who is attending a video conference on the transmitting side, and uses the captured video information for the face area detection unit 108 and the display device of the receiving terminal. It transmits to 107 via a network. In addition, the camera 103 transmits camera information such as the zoom magnification of the camera and the direction in which the camera is facing to the sound receiving direction determination unit 105 described later.

  The face area detection unit 108 of the receiving terminal detects the face area of the person from the video information transmitted from the camera 103, calculates the position information of the detected face area of the person, The data is transmitted to the sound receiving direction determination unit 105 on the talk terminal side.

  FIG. 4 is a diagram showing face area position information calculated by the face area detection unit 108.

As shown in FIG. 4, when the face area detection unit 108 acquires video information of a predetermined size from the camera 103, the lower left corner of the video information is the origin, and the minimum value is 0 ° and the maximum value is 180 in the vertical and horizontal directions. A coordinate system of ° is set, and the center coordinates of the detected face area are calculated as face area position information. In Figure 4, the face area detection unit 108 detects a human face region of the two from the video information, as the face area position information for each person _{(Θ px1 °, Θ py1 °} ), (Θ px2 °, Θ py2 _° )
Is calculated.

  For detecting a human face area from video information, for example, a method disclosed in Japanese Patent No. 3490910 “Face Area Detection Device” can be used.

The sound receiving direction determination unit 105 identifies the direction in which the video conference attendee on the transmission side exists based on the face region position information transmitted from the face region detection unit 108 and the camera information transmitted from the camera 103, and receives the receiving direction. The sound information in the direction specified by the sound unit 104 is acquired.
When there are a plurality of TV conference attendees on the transmission side, the sound reception direction determination unit 105 identifies the presence direction of each attendee, and the sound for each attendance direction specified by the sound reception unit 104 is present. Get information.

  The sound receiving direction determining unit 105 specifies the sound receiving direction based on the position information in the horizontal direction among the face area position information transmitted from the face region detecting unit 108. In order to specify the sound receiving direction, It is necessary to consider the camera information, for example, the zoom magnification of the camera and the direction in which the camera is facing.

  FIG. 5 is a diagram illustrating that the presence direction of the person changes according to the zoom magnification of the camera 103 even when the face area position information of the person calculated by the face area detection unit 108 is the same.

  As shown in FIG. 5, even if the horizontal position of the face area of the person detected by the face area detection unit 108 from the video information is the position a: b of the horizontal length of the video data size, the camera When the camera 103 is zoomed up, even if the direction in which the person exists is calculated as Θz ° with respect to the front direction of the camera 103, the direction in which the person exists is Θw ° when the camera is shooting wide. In this case, it is clear that Θz °> Θw °. That is, the direction in which the person exists differs between when the camera 103 is shooting with zoom-in and when shooting with a wide angle.

  Therefore, when the person's presence direction is specified from the face area position information of the person detected from the video information, it is necessary to correct the face area position information according to the zoom magnification of the camera 103.

  FIG. 6 shows a flow from when the sound receiving direction determination unit 105 determines the sound receiving direction based on the face area position information transmitted from the face area detection unit 108 in consideration of the zoom magnification of the camera 103 and the direction of the camera 103. FIG.

In FIG. 6, if the position information in the horizontal direction is Θpx1 ° and Θpx2 ° among the face region position information of two persons transmitted from the face region detection unit 108 _, the zoom magnification of _the camera 103 is λ. In this case, the sound receiving direction determination unit 105 calculates the first corrected sound receiving directions Θc1 ° and Θc2 ° by the following equations (1) and (2).

Θc1 = Θpx1 · f (λ) (1)
Θc2 = Θpx2 · f (λ) (2)
However, f (λ) is a correction function when the zoom magnification λ of the camera 103 is input. Next, the direction in which the camera 103 faces is, for example, a sound receiving unit composed of a plurality of microphones as shown in FIG. When the front direction of 104 is the reference (0 °) and Θd °, the sound receiving direction determination unit 105 calculates the second corrected sound receiving directions Θt1 ° and Θt2 ° by the following equations (3) and (4). To do.

Θt1 = Θc1 + Θd (3)
Θt2 = Θc2 + Θd (4)
The sound receiving direction determining unit 105 determines the sound receiving directions Θt1 ° and Θt2 ° after the second correction as the existence directions of the persons corresponding to the detected areas, and the sound receiving unit 104 receives sound information from the directions. Get it.

  The sound receiving direction determination unit 105 determines the volume level of sound information to be acquired according to the zoom magnification of the camera, and causes the sound receiving unit 104 to acquire sound information at the volume level.

  The sound receiving unit 104 acquires the sound information of each video conference attendee on the transmission side from each direction specified by the sound receiving direction determining unit 105 at the volume level determined by the sound receiving direction determining unit 105, It transmits to the received sound signal reproduction unit 109 of the side terminal.

  Based on the face area position information transmitted from the face area detection unit 108 and the sound information transmitted from the sound reception unit 104, the sound reception signal reproduction unit 109 transmits each attendee on the transmission side displayed on the display device 107. The sound information is controlled so that sound information corresponding to each attendee is emitted from the vicinity of the face area, and the sound information is output from the speaker 106.

  FIG. 7 is a diagram showing a method for outputting sound information by the sound reception signal reproduction unit 109.

  The received sound signal reproduction unit 109 includes a signal processing unit 109a that controls received sound information and a transfer function database 109b that stores a plurality of transfer functions for convolution with the sound information. The transfer function database 109b stores a plurality of transfer functions (for example, transfer function 1, transfer function 2, transfer function 3,...) For controlling the imaging position of received sound information. As shown in FIG. 7, the screen of the display device 107 is divided in advance for each predetermined area (1, 2, 3,... In FIG. 7), and each transfer function corresponds to each divided area. The signal processing unit 109a can image sound information in a region corresponding to each transfer function by convolving each transfer function with the received sound information.

  The signal processing unit 109a determines a region where the received sound information is to be imaged from the received face region position information, and selects a transfer function corresponding to the region. Next, the selected transfer function is convolved with the received sound information and reproduced from the speaker 106. Thereby, the sound information of each attendee can be imaged in the vicinity of the face area of each attendee on the transmission side displayed on the display device 107.

  As a result, the receiving-side video conference participant can feel as if each transmitting-side attendee is actually speaking from the display device 107 of the receiving-side terminal.

  Since humans have a pair of left and right ears, it is relatively easy for humans to distinguish which direction the sound information is output in the horizontal direction (left and right direction). It is not easy to tell from which direction the sound information is output in the direction (vertical direction).

  Therefore, the horizontal position of each attendee's face area on the display device 107 is specified based on the horizontal position information of the face area position information transmitted from the face area detection unit 108 to the sound reception signal reproduction unit 109. The sound information transmitted from the sound receiving unit 104 may be controlled so that sound information corresponding to each attendee is emitted from the vicinity of the horizontal position. In this case, the vertical position of the face area is fixed in advance to an appropriate position.

  Specifically, as shown in FIG. 8, the transfer function database 109b of the received sound signal reproduction unit 109 has a plurality of transfer functions (for example, transfer function 1, (Transfer function 2, transfer function 3,...) Are stored, and as shown in FIG. 8, the screen of the display device 107 is preliminarily arranged in predetermined horizontal regions (1, 2, 3,... Each transfer function corresponds to each divided area. In this case, the signal processing unit 109a can image sound information in the vicinity of the horizontal position of the area corresponding to each transfer function, that is, the face area, by convolving each transfer function with the received sound information.

  In the above embodiment, sound information is controlled using two speakers 106 as shown in FIG. 2 or FIG. 3, but a flat panel speaker 110 is used instead of the speaker 106 as shown in FIG. However, it may be arranged on the back surface of the display device 109. According to such a configuration, sound information can be actually output from the vicinity of the face area of each attendee on the transmission side displayed on the display device 107, so that a more realistic space can be created.

  Further, in the configuration of the hands-free videophone device 100 shown in FIG. 2, the sound receiving direction determining unit 105 of the transmission processing unit 101 may be eliminated and the configuration shown in FIG.

In such a configuration, the sound receiving unit 104 has, for example, almost no directivity, so-called
The microphone array includes a plurality of omnidirectional microphones, and each microphone acquires sound information of the entire range. When a video conference is performed using two hands-free videophone devices 100 having the above configuration, as shown in FIG. 11, the sound receiving unit 104 on the transmitting terminal side receives the sound information acquired by each microphone from the receiving terminal. It transmits to the sound signal reproduction unit 109. The camera 103 of the transmitting terminal transmits the camera information to the sound reception signal reproducing unit 109 of the receiving terminal.

The sound reception signal reproducing unit 109 of the receiving terminal determines the direction in which the video conference attendee on the transmitting side exists based on the face area position information transmitted from the face area detecting unit 108 and the camera information transmitted from the camera 103. The sound information corresponding to the specified direction is extracted or generated from the sound information for each microphone identified and transmitted from the sound receiving unit 104.

  The method of specifying the direction of presence by the sound reception signal reproduction unit 109 is the same as the method of specifying the direction of presence by the sound reception direction determination unit 105 in FIG. 2 or FIG. 3 described above.

  The received sound signal reproduction unit 109 is based on the face area position information transmitted by the face area detection unit 108, and the sound information extracted or generated is displayed near the face area of each attendee on the transmission side displayed on the display device 107. The sound information is controlled so that sound information corresponding to each attendee is emitted, and the sound information is output from the speaker 106.

  Each part structure which concerns on embodiment of this invention is not restricted to the above-mentioned embodiment, A various deformation | transformation is possible within the technical scope as described in a claim. For example, in the above-described embodiment, the sound reception direction determination unit 105 is provided in the transmission processing unit 101, and the face area detection unit 106 and the sound reception signal reproduction unit 107 are provided in the reception processing unit 102. It is possible to appropriately select which of the transmission processing unit 101 and the reception processing unit 102 includes these.

Furthermore, the hands-free videophone 10 in the embodiment according to the present invention described above.
The sound receiving direction determining unit 105, the face area detecting unit 108, and the sound receiving signal reproducing unit 109 constituting 0 can be realized in hardware by a CPU, memory, other LSI, etc. of any computer, and in software Can also be realized by a program loaded in a memory. Needless to say, it can also be realized by combining hardware and software.

  As described above, in the above-described hands-free videophone device 100, the face area detection unit 108 obtains the face area position information of the person participating in the conference from the video information of the transmission-side conference scene photographed by the camera 103. calculate.

  When the hands-free videophone device 100 has the configuration shown in FIG. 2, the sound receiving direction determination unit 105 determines whether the conference participant on the transmission side is based on the face area position information, the camera zoom magnification, and the camera direction. The presence direction is specified, and the sound receiving unit 104 acquires sound information from the direction. At this time, sound information is acquired by increasing or decreasing the volume level according to the zoom magnification of the camera 103. The received sound signal reproduction unit 109 forms an image of the acquired sound information in the vicinity of the face position of the transmitting conference participant displayed on the display device 107 based on the face area position information.

  When the hands-free videophone device 100 has the configuration shown in FIG. 10, the sound receiving unit 104 acquires sound information from all directions by a plurality of omnidirectional microphones. The sound reception signal reproduction unit 109 identifies the direction of presence of the conference participant on the transmission side based on the face area position information, the zoom magnification of the camera 103, and the direction of the camera 103. Next, the received sound signal reproduction unit 109 processes the acquired sound information to generate sound information from the direction in which each conference participant exists. Next, the received sound signal reproduction unit 109 images the generated sound information corresponding to each participant in the vicinity of the face position of each transmitting conference attendee displayed on the display device 107. At this time, the sound information is imaged by increasing / decreasing the volume level according to the zoom magnification of the camera 103.

  Thereby, the meeting attendee on the receiving side can feel that the meeting attendee on the sending side emits sound information from the display device on the receiving side. That is, according to the hands-free videophone device, on the receiving side, the situation where the conference participant on the transmitting side is producing voice and sound can be reproduced as faithfully as possible, and a more realistic space can be created.

It is a figure which shows the outline of a structure in the case of using the hands-free video telephone concerning embodiment for a video conference. It is a figure which shows the structure of the hands-free video telephone apparatus which concerns on embodiment. It is a figure which shows the structure and data flow in the case of performing a video conference using the hands-free video telephone apparatus which concerns on embodiment. It is a figure which shows the face area position information which the face area detection part with which the hands-free video telephone apparatus concerning an embodiment is provided calculates. In the hands-free videophone device according to the embodiment, it is a diagram for explaining the relationship between the face area position information of the person, the zoom magnification of the camera, and the direction of the person. In the hands-free videophone device according to the embodiment, it is a diagram showing a flow from the face area position information to the determination of the sound receiving direction in consideration of the zoom magnification and direction of the camera. It is a figure for demonstrating the control method of sound information in the hands-free video telephone apparatus which concerns on embodiment. It is a figure for demonstrating the control method of sound information in the hands-free video telephone apparatus which concerns on embodiment. It is a figure which shows the 2nd structure of the hands-free video telephone apparatus which concerns on embodiment. It is a figure which shows the 3rd structure of the hands-free video telephone apparatus which concerns on embodiment. It is a figure which shows the structure and data flow in the case of performing a video conference using the hands-free video telephone apparatus of the 3rd structure which concerns on embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Hands-free video telephone apparatus 101 Transmission processing part 102 Reception processing part 103 Camera
DESCRIPTION OF SYMBOLS 104 Sound receiving part 105 Sound receiving direction determination part 106 Speaker 107 Display apparatus 108 Face area detection part 109 Sound receiving signal reproduction part

Claims (6)

Video acquisition means for acquiring a video signal;
Sound acquisition means for acquiring an acoustic signal;
Transmitting means for transmitting the audio signal acquired by the audio acquisition means and the video signal acquired by the video acquisition means;
Receiving means for receiving the video signal and the audio signal transmitted by the transmitting means;
Video display means for displaying the video signal received by the receiving means;
Acoustic control means for controlling the acoustic signal received by the receiving means;
Acoustic output means for outputting an acoustic signal controlled by the acoustic control means;
A video signal and audio signal integration device comprising:
Face position detection means for detecting the face position of one or more persons on the video signal from the video signal acquired by the video acquisition means;
A person direction specifying means for specifying the presence direction of each person based on the face position of each person detected by the face position detection means and the image acquisition condition of the image acquisition means;
The acoustic acquisition means acquires an acoustic signal from the direction of existence of each person specified by the person specifying means,
The sound control means images each sound signal corresponding to each person near the face position of each person on the image signal displayed on the display means. Integrated device. Video acquisition means for acquiring a video signal;
Sound acquisition means for acquiring an acoustic signal;
Transmitting means for transmitting the audio signal acquired by the audio acquisition means and the video signal acquired by the video acquisition means;
Receiving means for receiving the video signal and the audio signal transmitted by the transmitting means;
Video display means for displaying the video signal received by the receiving means;
Acoustic control means for controlling the acoustic signal received by the receiving means;
Acoustic output means for outputting an acoustic signal controlled by the acoustic control means;
A video signal and audio signal integration device comprising:
Face position detection means for detecting the face position of one or more persons on the video signal from the video signal acquired by the video acquisition means;
A person direction specifying means for specifying the presence direction of each person based on the face position of each person detected by the face position detection means and the image acquisition condition of the image acquisition means;
The sound acquisition means acquires sound signals from all directions,
The acoustic control unit generates an acoustic signal from the direction of existence of each person based on information on the direction of existence of each person identified by the person identification unit from acoustic signals from all directions, An apparatus for integrating a video signal and an acoustic signal, wherein the generated acoustic signal corresponding to each person is imaged in the vicinity of the face position of each person on the video signal displayed on the display means. The video signal and sound signal integration apparatus according to claim 1, wherein the video acquisition condition of the video acquisition unit includes a zoom magnification of the video acquisition unit. The video signal and sound signal integration apparatus according to any one of claims 1 to 3, wherein the video acquisition condition of the video acquisition unit includes direction information of the video acquisition unit. 5. The video signal and sound signal integration apparatus according to claim 3, wherein the sound acquisition unit increases or decreases a volume level of the sound signal acquired according to the zoom magnification of the image acquisition conditions. . 5. The video signal according to claim 3, wherein the sound control unit increases or decreases a volume level of the sound signal output from the sound output unit according to the zoom magnification in the image acquisition condition. Sound signal integration device.

Images