JP2008113164A - Communication apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology to efficiently transmit necessary video information within a limited communication band in data transfer between conference terminals. <P>SOLUTION: The communication apparatus includes a plurality of Web cameras 107 which are so set up that their shooting areas may be different and generate a video signal, a microphone array 106 which collects the sound emitted from a sound source and generates a sound signal, and a sound source direction information generating means which generates sound source direction information which indicates the direction of the sound source based on the sound signal generated by the microphone array 106. The communication apparatus takes the image of the speaker by the Web camera 107 whose shooting area includes the direction indicated by the sound source direction information and outputs the voice and image of the speaker toward the network. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a communication apparatus that transmits an image together with sound.

In recent years, a conference system that conducts a conference using a plurality of conference terminals connected via a communication network has become widespread. Patent Document 1 discloses a technology for supporting the operation of a remote video conference that is performed with participants in remote locations. The system disclosed in this document includes a plurality of video conference terminals and a multipoint video conference relay device that mediates exchange of audio information and video information at each terminal. Then, the relay device appropriately superimposes the conference operation information such as the reference page of the material used in the conference and the remaining time until the conference ends on the audio information and video information passing through the conference device.
Japanese Patent Laid-Open No. 05-145918

  By the way, the video information that the participant needs regarding the other party varies depending on the aspect of the conference. For example, immediately after the start of the conference, it is necessary to know the state of the other party's conference room and the positional relationship of the participants, and it is necessary to know who is speaking with what expression during the discussion. There will be. However, if the technique of Patent Document 1 is used to transmit the necessary video information in detail to the other party, the amount of data to be transmitted becomes enormous and cannot be covered with a limited communication bandwidth.

  The present invention has been made in response to the above problems, and an object of the present invention is to provide a technique for efficiently transmitting necessary video information, for example, in data transfer between conference terminals.

  In the first embodiment of the communication apparatus according to the present invention, a shooting range is set, a plurality of shooting units each outputting a video signal indicating a video in each shooting range, and an audio signal of the collected sound A sound signal generating means for generating, a sound source direction specifying means for specifying the direction of the sound source based on the sound signal generated by the sound signal generating means, and outputting sound source direction information indicating the specified direction; and the sound source direction information Selection means for selecting the corresponding photographing means, and output means for outputting the video signal output by the photographing means selected by the selecting means and the audio signal generated by the audio signal generating means to another terminal device via the network It is characterized by comprising.

  In the second embodiment of the communication apparatus according to the present invention, a plurality of photographing means each for which a photographing range is set and each outputs a video signal indicating a video in each photographing range, and the sound of the collected sound A sound signal generating means for generating a signal, a sound source direction specifying means for specifying a sound source direction based on the sound signal generated by the sound signal generating means, and outputting sound source direction information indicating the specified direction; and the sound source direction Selection means for selecting the photographing means corresponding to the information; audio extraction means for extracting the audio signal corresponding to the sound source direction information from the audio signals generated by the audio signal generation means; and the imaging selected by the selection means And a video signal output by the means and an output means for outputting the audio signal extracted by the audio extraction means to another terminal device via a network.

  The communication device according to a third embodiment of the present invention is the communication device according to the second embodiment, wherein the sound signal generation means has a microphone array in which a plurality of microphones are arranged, and the sound extraction means is the microphone. A sound collection direction control device for controlling the sound collection direction of the array, wherein the sound source direction specifying means determines the direction from the sound collection direction controlled by the sound collection direction control device and the volume level of the audio signal output from the microphone array. It is characterized by having a direction specifying device for specifying.

  The communication device according to a fourth embodiment of the present invention is the communication device according to the first or second embodiment, wherein the audio signal generation means includes a plurality of microphones whose positions can be individually set, and the sound source direction. The specifying unit selects one microphone recognized as a sound source direction from the volume level of the audio signal output by each microphone, outputs information indicating the selected microphone as the sound source direction information, and the audio signal extraction unit includes The microphone audio signal selected by the sound source direction specifying means is extracted.

  According to a fifth embodiment of the communication apparatus of the present invention, in the embodiment described in any one of the first to fourth aspects, the output unit captures a specific area among all areas constituting one screen. It is set as a display area of the video signal output by the imaging means selected by the means selection means, and the synthesized video signal is output to the other area so that another video signal is displayed.

  In addition, the sixth embodiment of the communication device according to the present invention, in the fifth embodiment, how to divide all areas constituting one screen of the video signal output by the output means. A plurality of segment data for setting the specific area in the segmented area, and segment data selection means for selecting the segment data, and the output unit selects the segment selected by the segment data selection unit One screen is divided according to the data, and an area indicated by the divided data is recognized as the specific area.

  Further, in a seventh embodiment of the communication device according to the present invention, in the embodiment described in any one of the first to sixth, a plurality of pattern images showing a plurality of modes according to the sound source position of the region to be photographed are provided. The pattern image selecting means having a table storing a correspondence relationship between a sound source position in each pattern image and a photographing range of each photographing means, and a pattern image selecting means for selecting the pattern image. And a shooting range control unit that determines each shooting range corresponding to the selected template with reference to the table and controls each shooting unit so as to match the determined shooting range.

  According to the conference terminal according to the present invention, it is possible to provide a technique for efficiently transmitting video information necessary for data transfer between conference terminals.

Hereinafter, a conference terminal according to an embodiment of the present invention will be described with reference to the drawings.
(A: Configuration)
FIG. 1 is a block diagram showing a configuration of a conference system 1 including a conference terminal according to an embodiment of the present invention. The conference system 1 includes a conference terminal 10A, a conference terminal 10B, and a communication network 20, and the conference terminal 10A and the conference terminal 10B are connected to the communication network 20 by wire. The conference terminal 10A and the conference terminal 10B have the same configuration, and in the following, when there is no need to distinguish between the conference terminal 10A and the conference terminal 10B, they are collectively referred to as the conference terminal 10. Although the case where it is connected to the network 20 is illustrated, three or more conference terminals may be connected.

In this embodiment, each communication protocol described below is used as a communication protocol. That is, Real-time Transport Protocol (hereinafter “RTP”) is used for transferring audio data and video data as a communication protocol in the application layer. RTP is a communication protocol for providing a communication service for transmitting and receiving audio data and video data in end-to-end in real time, and details thereof are defined in RFC1889. In RTP, data is exchanged between communication terminals by generating and transmitting / receiving RTP packets. On the other hand, HTTP (Hypertext Transfer Protocol) is used for transferring image data. Further, UDP is used as a transport layer communication protocol, and IP is used as a network layer communication protocol. Each of the conference terminal 10A and the conference terminal 10B is assigned an IP address, and is uniquely identified on the network.
Since HTTP, UDP, and IP are communication protocols that are generally widely used, description thereof is omitted.

  Next, the hardware configuration of the conference terminal 10 will be described with reference to FIG. In the following description, when it is necessary to distinguish which conference terminal the configuration of the conference terminal 10 listed above belongs to, for example, the control unit 101 of the conference terminal 10A is changed to the control unit 101A or the like. As shown, the alphabet is used.

  A control unit 101 shown in the figure is, for example, a CPU (Central Processing Unit), and controls the operation of each unit of the conference terminal 10 by executing various programs stored in a storage unit 103 described later.

The storage unit 103 includes a ROM (Read Only Memory) 103a and a RAM (Random Access Memory) 103b.
The ROM 103a stores data and a control program for causing the control unit 101 to realize functions characteristic of the present invention. Examples of the data include a Web camera selection table, test data, a transmission rate management table, and the like. These data will be described in detail later.
The RAM 103b is used as a work area by the control unit 101 operating according to various programs, and stores audio data / video data received from the microphone array 106 and the web camera 107.

  The operation unit 104 is, for example, a keyboard or a mouse. When the operator of the conference terminal 10 operates the operation unit 104 to perform some input operation, data representing the operation content is transmitted to the control unit 101.

The microphone array 106 includes a plurality (eight in this embodiment) of microphone units (not shown) and an analog / digital (hereinafter abbreviated as “A / D”) converter. The microphone array 106 has a function as a directional microphone, and has a function of collecting sound while scanning a direction in which the sound is collected. The control unit 101 analyzes the voice data generated from the voices from various directions as described above, and specifies the direction in which the volume level is high as the direction of the sound source (that is, the speaker if the received voice is a human voice). To do. In addition, when a plurality of participants speak at the same time, and the sound is input from a plurality of directions at the same time, the control unit 101 compares the sound volume levels from the plurality of directions, and the sound volume level is the highest. The higher direction is the direction of the sound source.
FIG. 5 is a diagram showing an example of the relative arrangement of the microphone array 106 and the participant 2 in the conference room. Specifically, the information regarding the sound source azimuth specified by the microphone array 106 is generated as the direction of the sound source viewed from the center of the microphone array 106 (polar angle Φ in polar coordinates), and is output to the RAM 103b together with the audio data. In FIG. 5, the speaker is the participant 2a, and the sound source orientation is π / 6.

The Web camera 107 outputs the input from the C-MOS image sensor as a Motion-JPEG moving image. Note that Motion-JPEG is a moving image data generation method in which video for each captured frame is JPEG compressed and recorded continuously. The Web camera 107 captures a video with a predetermined image size and the number of frames per unit time (fps; frames per second), performs image compression with JPEG, and outputs the image to the RAM 103b. The resolution uses a value preset in the Web camera, and the number of frames per unit time is appropriately controlled by the control unit 101. The image compression rate can be set under the control of the control unit 101 within the range of JPEG image compression (compression rate is 1/5 to 1/60).
In the present embodiment, five web cameras 107 are attached to the conference terminal 10, and the participant 2 can manually change their orientation.

  Data output from the Web camera 107 is temporarily written in the RAM 103b, and the control unit 101 generates an RTP packet from the data written in the RAM 103b. As shown in FIG. 2, the RTP packet is configured by adding a header portion to the payload portion, similarly to a packet that is a data transfer unit in IP and a segment that is a data transfer unit in TCP.

In the payload portion, for example, audio / video data for a predetermined time (20 milliseconds in the present embodiment) is written in a data transmission message. In the reception notification message, the sequence number of the received packet is written.
Four types of data including a time stamp, a payload type, a sequence number, and partition information are set in the header portion. Here, the time stamp is data indicating the time (the time elapsed since the start of voice communication was instructed). The payload type is data for identifying the type of communication message to the destination of the communication message. There are five message types used in the present embodiment: a speaker video data transmission message, a conference room video data transmission message, a voice data transmission message, a material data transmission message, and a reception notification message. In these messages, five types of numbers “1”, “2”, “3”, “4”, and “5” are written in the payload type, respectively. The sequence number is an identifier for uniquely identifying each packet. For example, when one voice data is divided into a plurality of RTP packets and transmitted, 1, 2, 3,. Thus, a sequence number is assigned. The section information is information that defines in which area of the video display unit 105, which will be described later, an image included in the RTP packet. For example, image position information is written in coordinates, such as (0, 0)-(256, 192).

  The communication IF unit 102 is, for example, a NIC (Network Interface Card) and is connected to the communication network 20. The communication IF unit 102 sends IP packets obtained by sequentially encapsulating RTP packets delivered from the control unit 101 in accordance with a lower layer communication protocol to the communication network 20. Encapsulation means generating a UDP segment in which the RTP packet is written in the payload portion, and further generating an IP packet in which the UDP segment is written in the payload portion. Further, the communication IF unit 102 receives an IP packet via the communication network 20 and performs a process reverse to the encapsulation, thereby reading out the RTP packet encapsulated in the IP packet and controlling the control unit 101. Delivered to.

  The video display unit 105 is a 1024 pixel × 768 pixel monitor. An image or material is displayed based on the data received via the communication IF unit 102.

The speaker 108 reproduces sound represented by the sound data delivered from the control unit 101, and includes a speaker unit and a D / A converter. The D / A converter converts the audio data delivered from the control unit 101 into an analog audio signal by performing D / A conversion, and outputs the analog audio signal to the speaker unit. Then, the speaker unit reproduces sound corresponding to the sound signal received from the D / A converter.
The hardware configuration of the conference terminal 10 has been described above.

  Here, an installation mode of the conference terminal 10 in the conference room will be described. FIG. 6A is a diagram (corresponding to FIG. 5) showing a state of the conference room in which the conference terminal 10 is installed. A desk 3 is installed in the conference room, and participants 2a, 2b, 2c, and 2d participating in the conference are seated on chairs installed around the desk. A conference terminal 10 is installed beside the desk, and a video display unit 105, a microphone array 106, five Web cameras 107 installed, and a speaker 108 face the participants. In the following, when it is necessary to distinguish and indicate each of the five web cameras 107, they are represented as web cameras 107a, 107b, 107c, 107d, and 107e, respectively, as shown in FIG.

  The video display unit 105 is arranged at a place where all the participants can see. The microphone array 106 is disposed below the video display unit 105 so that a plurality of microphone units are horizontally aligned. The speakers 108 are arranged at two locations on the conference terminal 10 so as to sandwich the microphone array 106.

  The Web cameras 107 are arranged below the video display unit 105 so as to be arranged in a horizontal line, and are connected to the conference terminal 10 via lines numbered respectively. Specifically, the web camera 107 a is connected to line 1, the web camera 107 b is connected to line 2, the web camera 107 c is connected to line 3, the web camera 107 d is connected to line 4, and the web camera 107 e is connected to line 5.

  Here, FIG. 6B shows a state where the conference room shown in FIG. 6A is viewed from the conference terminal 10 side. In FIG. 6B, areas a, b, c, d, and e indicate areas that are photographed by the Web cameras 107a, 107b, 107c, 107d, and 107e shown in FIG. 6A, respectively. The shooting range of each Web camera 107 is set so that the area e includes the entire conference room, and the areas a, b, c, and d include the participants 2a, 2b, 2c, and 2d, respectively.

  Next, characteristic functions of the conference terminal 10 will be described. These are roughly divided into three types: (1) video selection function, (2) usable bandwidth measurement function, and (3) data transmission rate control function. The above functions are realized by the control unit 101 executing a control program stored in the ROM 103a.

First, the video selection function will be described. The video selection function is a function of selecting the Web camera 107 directed to the speaker who is speaking at that time and displaying the video captured by the selected Web camera 107 on the other party conference terminal 10. .
Here, the Web camera selection table stored in the ROM 103a will be described. FIG. 7 is a diagram illustrating an example of the Web camera selection table. In the Web camera selection table, the range of the declination that is the sound source direction information is associated with the line number to which each Web camera 107 is connected.

  The control unit 101 reads the sound source azimuth information written in the RAM 103b and identifies the line number by comparing the declination value, which is the sound source azimuth information, with the Web camera selection table. For example, as shown in FIG. 5, when the participant 2a with the deflection angle Φ = π / 6 speaks, the Web camera selection table corresponds to the deflection angle Φ = 0 to π / 4 of the sound source direction information. The web camera 107 connected to the line 1, that is, the web camera 107a is selected. Subsequently, the control unit 101 outputs a signal for generating and outputting video data to the Web camera 107 connected to the selected line number. The Web camera 107 that has received the signal from the control unit 101 generates video data and outputs it to the RAM 103b.

  The control unit 101 transmits the video data generated by the Web camera 107e toward the counterpart conference terminal 10 at the same time as the video data in which the speaker is photographed as described above. At that time, the control unit 101 writes “1” in the payload type of the RTP packet including the video data in which the speaker is photographed, and “(0,0)-(512, 384)” in the section information. On the other hand, “2” is written in the payload type of the video data generated by the Web camera 107e, and “(0, 384)-(512, 768)” is written in the section information. In addition, the document data is transmitted in parallel with the video data. “4” is written in the payload type of the document data, and “(512, 0) − (1024, 768)” is written in the section information.

  The counterpart conference terminal 10 receives both video data. The video display unit 105 reads out the section information of the received data, and displays the data contents in a corresponding area of the video display unit 105. As shown in FIG. 4, the display surface of the video display unit 105 has coordinates set with the upper left in the drawing as the origin. For example, the section information “(0,0)-(512,384)” is surrounded by four points of the origin (0,0), (512,0), (512,384), (0,384). Represents an area. Accordingly, the video data of the Web camera 107a is displayed on the speaker display unit 105a, the video data of the Web camera 107e is displayed on the conference room display unit 105b, and the material data is displayed on the material display unit 105c.

  Next, the available bandwidth measurement function will be described with reference to the flowchart shown in FIG. The available bandwidth measurement function is a function of measuring the maximum communication bandwidth that can be used in the communication network 20 when performing data communication with the other party conference terminal via the communication network 20. First, the control unit 101 determines a transmission interval when transmitting a packet (step SC100). When performing the available bandwidth measurement process for the first time, a predetermined transmission interval is selected. Next, the control unit 101 generates a plurality of packets from the test data stored in the ROM 103a and transmits the packets to the partner conference terminal at the selected transmission interval (step SC110). At this time, the control unit 101 writes the sequence number of each transmitted packet in the RAM 103b. The test data is Motion-JPEG video data, similar to the video data generated by the Web camera 107 in advance, and the video content may be anything.

  The control unit 101 of the partner conference terminal 10 receives the test data, writes the sequence number of each received packet in a reception notification message, and returns the reception notification message to the transmission conference terminal. The control unit 101 of the transmitting conference terminal 10 receives the reception notification message returned from the other conference terminal (step SC120), and generates the packet loss occurrence rate (number of packets not received / The number of transmitted packets) is calculated (step SC130).

When the test data is transmitted at the predetermined transmission interval and no packet loss occurs (step SC130; “No”), the control unit 101 performs the processes after step SC100 again. At that time, in step SC100, a transmission interval shorter than the transmission interval used in the previous processing of step SC100 to step SC130 by a predetermined rate is selected. When the transmission interval is shortened, the transmission data amount per unit time, that is, the transmission rate is increased. In general, the rate of packet loss is low when the transmission rate is small compared to the available bandwidth, and the rate of packet loss increases as the transmission rate increases relative to the available bandwidth. It has been known. Therefore, if a packet loss occurs during the transmission of test data, it means that the transmission rate used at that time exceeds the available communication bandwidth.
Therefore, the control unit 101 repeatedly shortens the transmission interval of each packet and repeats steps SC100 to SC130. When packet loss occurs (step SC130; “Yes”), the test data is stored one time before that. The transmission rate (data amount of test data / time taken for transmission) at the time of transmission is calculated as the available bandwidth (unit: BPS; Byte / second) (step SC140).
The above is the available bandwidth measurement process.

Next, the data transmission rate control function will be described.
The data transmission rate control function is a function for executing processing for controlling the data transmission rate of audio data and video data to be output according to the available communication bandwidth measured by the available bandwidth measurement function.

  FIG. 9 is a diagram showing an example of a transmission rate management table. In the transmission rate management table, the number of frames per unit time (frames per second) set in the Web camera 107 and the compression rate of the JPEG image in the Motion-JPEG system are defined in correspondence with the available communication bandwidth. Has been. The control unit 101 compares the available communication bandwidth measured by the above-described available bandwidth measurement function with the transmission rate management table stored in the ROM 103a, so that a unit time corresponding to the available bandwidth is obtained. Select the number of frames per frame and the JPEG image compression rate.

The control unit 101 sets the number of frames per unit time and the compression rate of the JPEG image to the selected values in all the Web cameras 107. When data communication is started, the Web camera 107 generates video data with the set number of frames per unit time, and the control unit 101 compresses the generated video data at the compression rate of the selected JPEG image.
The above is the data transmission rate control function.

(B: Operation)
Next, an operation performed by the conference terminal 10 when a participant on the conference terminal 10A side and a participant on the conference terminal 10B conduct a remote conference will be described. It is assumed that each of the Web cameras 107 is already installed so that the area shown in FIG. 6B can be taken at the start of the remote conference. In the Web camera selection table, it is assumed that the direction of the sound source and the line number to which each Web camera 107 is connected are written in association with each other as shown in FIG.

  First, the control unit 101A performs parameter adjustment processing immediately after the start of data communication and at regular intervals. FIG. 10 is a flowchart showing the flow of parameter adjustment processing performed by the data transmission side (conference terminal 10A).

First, the control unit 101A performs an available bandwidth measurement process (step SA100). Next, the control unit 101 compares the measurement value of the available bandwidth measurement process with the transmission rate management table (see FIG. 9) stored in the ROM 103a, and uses the maximum utilization among the smaller values in the table. Select the number of frames associated with the possible bandwidth and the compression rate of the JPEG image. The control unit 101A sets the number of frames taken per unit time of all the Web cameras 107 to the selected value (step SA110) and sets the JPEG image compression rate to the selected value (step SA120). In subsequent step SA130, it is determined whether or not a fixed time has elapsed since the parameter adjustment process was started. If the determination result in step SA130 is “No”, the process in step SA130 is repeated until a predetermined time has elapsed. When the predetermined time has elapsed, the determination result in step SA130 is “Yes”, and step SA140 is performed. In step SA140, control unit 101A determines whether data communication has ended. If the determination result in step SA140 is “No”, the processes in and after step SA100 are performed again. If the determination result in step SA140 is “Yes”, the control unit 101 ends the parameter adjustment process.
From the above processing, the control unit 101 performs an available bandwidth measurement process at regular intervals after the start of data communication, and various parameters related to data transmission are set each time according to the measured available bandwidth. It will be. As a result, data can be transmitted according to the available communication bandwidth that changes from moment to moment, and data can be transmitted efficiently and without hindrance.

Hereinafter, an operation performed by the conference terminal 10A when the participant 2a on the conference terminal 10A side speaks during the remote conference and the participant on the conference terminal 10B side listens to the speech will be described.
FIG. 11 is a flowchart showing a flow of processing performed by the conference terminal 10 when performing data communication. When the participant 2a speaks, the microphone array 106A picks up the sound and generates sound data (step SB100). The control unit 101A generates sound source direction information based on the generated audio data (step SB110). In this case, since the participant 2a is located at a position as shown in FIG. 5, the sound source orientation information is π / 6, which is the orientation (deflection angle) of the participant 2a. The audio data and sound source direction information generated by the microphone array 106A are written into the RAM 103bA.

  The control unit 101A identifies one of the line numbers to which the web camera 107 is connected (in this case, line 1) by comparing the sound source direction information written in the RAM 103bA with the web camera management table (step SB120). . Next, the control unit 101A causes the Web camera 107aA specified in step SB120 to generate video data representing a detailed video of the participant 2a (step SB130).

  The control unit 101A transmits the speaker video data generated by the Web camera 107aA, the conference room video data generated by the Web camera 107eA, audio data, and material data to the conference terminal 10B (Step SB140).

  The conference terminal 10B receives video data, audio data, and material data representing a speaker video / conference room video. The audio data is reproduced from the speaker 108B. For the video data and the material data, the control unit 101B reads the section information included in each video data. In the section information, an area of the video display unit 105B where each data is to be displayed is written in coordinates. The video display unit 105B displays each data content in a predetermined area based on the section information.

Through the above processing, the participant who uses the conference terminal 10B on the other side can listen to the voice of the speaker who uses the conference terminal 10A from the speaker 108B, while viewing the document data presented by the speaker and the facial expression of the speaker. It can be visually recognized.
(C: Modification)
As mentioned above, although embodiment of this invention was described, this invention can be implemented with the various form described below.

(1) In the above embodiment, the video selection function, the available bandwidth measurement function, and the data transmission rate control function are provided for the conference terminal. However, the installation target is not limited to the conference terminal. For example, the present invention may be applied to a server device that generates video data and audio data and provides them to the client device.

(2) In the above embodiment, a case has been described in which the functions characteristic of the conference terminal according to the present invention are implemented by software modules. However, the conference terminal according to the present invention is combined with the hardware modules responsible for the above functions. You may make it comprise.

(3) In the above-described embodiment, the case where RTP is used as the communication protocol of the application layer for communication of video data and audio data has been described, but it is needless to say that other communication protocols may be used. In short, any communication protocol may be used as long as it is a communication protocol in which video data or audio data is written and transmitted for a predetermined time in a payload portion of a data block having a predetermined header portion and a payload portion. . In the above-described embodiment, the case where UDP is used as the transport layer communication protocol has been described. However, TCP may be used. Similarly, the network layer communication protocol is not limited to IP.

(4) In the above-described embodiment, the case where transmission / reception of video data, audio data, and material data is performed in parallel has been described. However, the types of data are not limited to these data types. There is no need to send and receive document data.

(5) In the above embodiment, the case where the conference terminal 10A and the conference terminal 10B are wired to the communication network 20 has been described. However, the communication network 20 is a wireless packet communication network such as a wireless local area network (LAN), for example. Yes, the conference terminal 10A and the conference terminal 10B may of course be connected to this wireless packet communication network. Moreover, although the case where the communication network 20 is the Internet was demonstrated in the said embodiment, of course, LAN (Local Area Network) may be sufficient. In short, any communication network may be used as long as it has a function of mediating communication performed in accordance with a predetermined communication protocol.

(6) In the above embodiment, a case has been described in which a control program for causing the control unit 101 to realize functions characteristic of the communication apparatus according to the present invention is written in the ROM 103a in advance. The control program may be recorded and distributed on a computer-readable recording medium, or the control program may be distributed by downloading via a telecommunication line such as the Internet.

(7) In the above embodiment, the case where the video data of the entire conference room is controlled by the same resolution, the number of frames per unit time, and the compression rate of the JPEG image as the video data of the speaker's video has been described. The video of the entire conference room may be sent out by reducing the number of frames per unit time or increasing the compression rate of the JPEG image as necessary, or still images may be sent out at regular intervals. .

(8) In the above embodiment, the case where the video data of the speaker is a moving image has been described. However, a still image may be transmitted as necessary or according to the line status of the communication network.

(9) In the above embodiment, the case where the speaker is specified by using the microphone array has been described. However, the speaker identification method is not limited to the microphone array. For example, a plurality of microphones (preferably the same number as the number of participants) are installed in front of the participants in the conference room, and the audio data with the highest volume level is selected from the sounds collected by the plurality of microphones. It may be determined that the participant corresponding to the microphone that has generated the selected audio data is a speaker, and a camera directed to the identified speaker may be selected.
Further, as many buttons as the number of participants may be provided as the operation unit 104, and each participant may make a statement after pressing the button. In that case, the control unit 101 may receive the signal output from the pressed button, identify the speaker, and select the Web camera 107 directed to the speaker.

(10) In the above embodiment, the Web camera generates the video data by the Motion-JPEG method. However, the video recording method is not limited to Motion-JPEG, and other methods such as MPEG may be used.

(11) In the above embodiment, the case where five Web cameras 107 are installed has been described, but the number of Web cameras 107 is not limited to five. What is necessary is just to determine suitably according to the number of participants. The same applies to the number of lines (terminals) for connecting the web cameras 107 provided in the conference terminal 10 as long as the number is larger than the number of installed web cameras 107.

(12) In the above embodiment, the case where the participant manually adjusts the shooting area of the Web camera 107 has been described. However, the imaging area of the Web camera 107 may be adjusted under the control of the control unit 101. In that case, an embodiment as described below is possible.
The conference terminal 10 is provided with drive means for adjusting the shooting area by rotating the Web camera horizontally and vertically, and the drive means can be controlled by the control unit 101. In addition, an “arrangement template” is stored in the ROM 103a in advance. FIG. 12 is a diagram illustrating an example of an arrangement template. A plurality of patterns A, B, and C are set in the arrangement template. For each pattern, the position of the participant and the shooting direction of the Web camera are defined in association with each other. Here, the position of the participant is a diagram schematically showing the relative position of each participant with reference to the microphone array 106, while the web camera shooting direction is the web camera 107 connected to each line. The shooting direction. Note that the shooting direction is represented as a polar angle in the same polar coordinate as that used when the microphone array generates the sound source direction information.
When starting the conference, the video display unit 105 displays the template image written in the “participant position” item of the arrangement table. A participant who uses the conference terminal selects a pattern that matches the number of participants and seat positions in the conference from a plurality of patterns displayed on the video display unit 105. When a specific pattern is selected, the control unit 101 drives the drive unit so that each of the Web cameras 107 connected to each line is directed in the Web camera shooting direction defined in the selected pattern.
According to the embodiment described above, the participant creates a template according to the installation position after installing the conference terminal 10 in the conference room, so that the position of the Web camera is manually set every time a subsequent conference is held. The trouble of reworking can be saved.

(13) In the above embodiment, the case where the display surface of the video display unit 105 is partitioned as shown in FIG. 4 has been described. However, the section of the video display unit 105 is not limited to the above mode. For example, the partition template shown in FIG. 13 is stored in the ROM 103a. In the division template, a plurality of templates schematically showing the division method of the display surface of the video display unit 105 are defined for the number of participants in the conference. In each template, a section in which a number is written displays a video (image of each participant) of the Web camera 107 connected to the line number of the number. For example, in the section displayed as 1/2/3, an image of one of the Web cameras 107 connected to the line numbers 1 to 3 is displayed. In the section where “conference room” is written, the video of the web camera 107e connected to the line 5 is displayed in the same manner as the conference room display unit 105b in the above embodiment. The section in which “Material” is written is the same as the material display portion 105 c in the above embodiment.
Now, when the conference starts, the participant selects a specific template according to the number of participants in the conference. When a specific template is selected, the video display unit 105 is partitioned as defined by the selected template, and the video and materials of each Web camera 107 are displayed in each partition as defined by the template. The
As described above, information required by the participant can be displayed on the video display unit 105 in a layout according to the situation at that time.
Further, after selecting the default template as described above, the template may be switched for a short period when the speaker speaks further. That is, when a speaker switching template is separately selected and the speaker is switched, the control unit 101 switches the partitioning method of the video display unit 105 from the default template to the speaker switching template for a predetermined time, and the predetermined time has elapsed. Then it returns to the default template. For example, when there are three participants, template A is selected as the default template, and template C is selected as the speaker switching template. For example, when participant A starts speaking, template C will be displayed. The video of participant A who is a speaker is displayed in the center of the screen in the center of the video display unit 105, and then the video of all participants, the video of the entire conference room, and materials are displayed as shown in template A. . Similarly, when Participant B speaks, the video of Participant B is displayed large in the center of the screen for a predetermined time after starting speaking. As described above, by switching the template during the conference, it is possible to notify that the speaker has changed or to notify the participant which participant has started speaking.

(14) In the above-described embodiment, the case where the speaker / conference room video data and the material data are displayed in the division as shown in FIG. 4 on the video display unit 105 has been described. However, the partitioning of the video display unit 105 is not limited to the above-described mode, and may be performed in various other modes such as the size of each data display area and the positional relationship.

(15) In the above-described embodiment, a case has been described in which an area in which each video data is displayed on the video display unit 105 is specified by including partition information in the header part of the RTP packet. However, the conference terminal 10 on the video data receiving side may control which area of the video display unit 105 displays each video data. In that case, the counterpart conference terminal 10 divides the video display unit 105 into a plurality of areas in advance and associates video types (speaker video and conference room video) with each of the zones. When video data is received, the payload type of the packet is read out, the type of video is determined from the payload type, and the video data is assigned to the corresponding video display unit 105 and output.

(16) In the above embodiment, the case where the display surface of the display unit 105 is partitioned and a plurality of data is displayed has been described. However, for example, when the video of the speaker is displayed on the entire display surface, the display surface of the video display unit 105 may not be partitioned.

It is a block diagram which shows the structure of the conference system containing the conference terminal which concerns on this invention. It is a figure which shows the structure of a RTP packet. 2 is a block diagram showing a configuration of a conference terminal 10. FIG. It is the figure which showed an example of the video display part. It is a figure explaining the detection of the sound source direction by a microphone array. It is a figure which shows the positional relationship of the conference terminal and participant in a conference room. It is a figure which shows the positional relationship of the participant seen from the conference terminal side. It is a figure which shows an example of a web camera selection table. It is a flowchart which shows the flow of an available bandwidth measurement process. It is a figure which shows an example of a transmission rate management table. It is a flowchart which shows the flow of a parameter adjustment process. It is a flowchart which shows the flow of data transfer. It is a figure which shows an example of an arrangement | positioning template. It is a figure which shows an example of a division template.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Conference system, 2 ... Participant, 3 ... Desk, 10, 10A, 10B ... Conference terminal, 20 ... Communication network, 101 ... Control part, 102 ... Communication IF part, 103 ... Memory | storage part (103a; ROM, 103b; (RAM), 104 ... operation unit, 105 ... video display unit (105a ... speaker display unit, 105b ... conference room display unit, 105c ... document display unit), 106 ... microphone array, 107 ... web camera, 108 ... speaker, 109 …bus

Claims (7)

A plurality of photographing means each for which a photographing range is set and outputting a video signal indicating an image within each photographing range;
Audio signal generation means for generating an audio signal of the collected sound;
Sound source direction specifying means for specifying the direction of the sound source based on the sound signal generated by the sound signal generating means, and outputting sound source direction information indicating the specified direction;
Selecting means for selecting the photographing means corresponding to the sound source direction information;
A communication apparatus comprising: a video signal output by the photographing unit selected by the selection unit; and an output unit that outputs the audio signal generated by the audio signal generation unit to another terminal device via a network. A plurality of photographing means each for which a photographing range is set and outputting a video signal indicating an image within each photographing range;
Audio signal generation means for generating an audio signal of the collected sound;
Sound source direction specifying means for specifying the direction of the sound source based on the sound signal generated by the sound signal generating means, and outputting sound source direction information indicating the specified direction;
Selecting means for selecting the photographing means corresponding to the sound source direction information;
A voice extraction unit that extracts a voice signal corresponding to the sound source direction information from the voice signal generated by the voice signal generation unit;
A communication apparatus comprising: a video signal output by the photographing means selected by the selection means; and an output means for outputting the audio signal extracted by the audio extraction means to another terminal device via a network.   The sound signal generation means includes a microphone array in which a plurality of microphones are arranged, the sound extraction means includes a sound collection direction control device that controls a sound collection direction of the microphone array, and the sound source direction specifying means includes the sound source direction specifying means The communication apparatus according to claim 2, further comprising: a direction specifying device that specifies the direction from a sound collection direction controlled by a sound collection direction control device and a volume level of an audio signal output from the microphone array. .   The sound signal generating means has a plurality of microphones whose positions can be individually set, and the sound source direction specifying means has one microphone recognized as a sound source direction from the volume level of the sound signal output from each microphone. 3. The information indicating the selected microphone is output as the sound source direction information, and the sound signal extracting means extracts the sound signal of the microphone selected by the sound source direction specifying means. The communication apparatus as described in.   The output means sets a specific area among all areas constituting one screen as a display area of a video signal output by the photographing means selected by the photographing means selection means, and other video signals are displayed in the other areas. 5. The communication apparatus according to claim 1, wherein a video signal synthesized so as to be displayed is output. A plurality of pieces of division data indicating how to divide all areas constituting one screen of the video signal output by the output means, and setting the specific area in the divided areas;
Section data selection means for selecting the section data,
6. The communication apparatus according to claim 5, wherein the output unit classifies one screen according to the segment data selected by the segment data selection unit and recognizes an area indicated by the segment data as the specific area. A table storing a plurality of pattern images showing a plurality of modes according to the sound source position of the region to be imaged, and storing a correspondence relationship between the sound source position in each pattern image and the imaging range of each imaging means;
Pattern image selection means for selecting the pattern image,
A shooting range control unit that determines each shooting range corresponding to the template selected by the pattern image selection unit with reference to the table, and controls each shooting unit so as to match the determined shooting range; The communication device according to any one of claims 1 to 6.

Images