JPH11177953A - Conference terminal device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a conference terminal which can always process the preferential information with no special operation to be consciously performed by a user during a conference even when a large quantity of video data cannot be processed due to the low calculation capability. SOLUTION: This device is connected to a network 10 and transmits the information on the voices, images relative to a conference through the network 10. If the specific information is contained in a transmitting signal, the priority information is added (13) to the transmitting signal and sends it to a receiving terminal to request the preferential processing of the transmitting signal. The receiving terminal detects the priority information that is added to the transmitting signal and processes the information with preference in response to the request of the transmitter side.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a conference terminal device applicable to a multipoint multimedia conference system using a personal computer or a workstation.

[0002]

2. Description of the Related Art In recent years, remote multipoint communication is performed by transmitting conference-related information such as video and audio (hereinafter, information such as video and audio is referred to as multimedia data) via a network. Multimedia conferencing systems are being used.

FIG. 19 shows an outline of an example of a conventional multipoint multimedia conference system. In FIG. 19, reference numerals 1a to 1d denote conference terminal devices participating in a multipoint multimedia conference system. These conference terminal devices 1a to 1d are remote points A and B, respectively. , C, and D, but are connected to each other through the network 2. The network 2 is further connected to a multipoint control device (hereinafter referred to as MCU) 3.

In the multipoint multimedia conference system shown in FIG. 19, each of the conference terminal devices 1a to 1d is usually used.
Transmits, to the MCU 3 via the network 2, information such as a video captured by a camera, a material video prepared in advance, and the voice of the user when he / she is speaking. MC
U3 collects these multimedia data such as video and audio, synthesizes the data, and
To each of the conference terminal devices 1a to 1d.

[0005] In this case, the MCU 3 is connected to each conference terminal device 1.
For the video information from a to 1d, each of them is classified and in each of the conference terminal devices 1a to 1d,
Compositing is performed so that the terminal can determine which terminal the video information is from. That is, for example, as shown in FIG. 20, a video window 5 including divided screens 5a to 5d of the number of conference terminal devices participating in the conference is displayed on the screen of the display 4 of a certain conference terminal device. Then, the MCU 3 synthesizes the video information.

[0006] In general, a plurality of people do not speak at the same time in a conference, and each conference participant can display a video of one speaker (speaker) or a video provided by the speaker. Focus on it. Therefore, it is desirable that at least the video of the speaker or only the video provided by the speaker be displayed with high quality. This is because the expression of the speaker, the movement of the lips, or the image provided by the speaker helps the listener to deepen the understanding of the listener.

However, in the multipoint multimedia conference system using the MCU 3 described above, the MCU 3 handles the video information from each of the conference terminal devices 1a to 1d equally and sets the video window 5 for the divided image synthesis. In order to generate the image, it is not possible to improve the image quality by giving priority only to the image from the terminal to which attention is paid.

[0008] Further, data from a certain conference terminal device is M
If the arrival at the CU 3 is delayed, the MCU 3 waits for the arrival of the delayed data, which adversely affects the video display of all the other terminals. For a conference terminal device whose data arrives late, it is possible to display the image with reduced video quality, but in such a case, the terminal using the terminal whose data arrives late is used for the conference. If the person who is participating is a speaker, the conference must proceed while watching low-quality images, making the conference very difficult.

Further, at this stage, the MCU 3 is very expensive, and it is ideal that a multipoint multimedia conference can be performed without using the MCU 3 if possible. In order to satisfy the requirement not to use the MCU 3, there is an approach of using multicast for multimedia data distribution to multiple points as shown in FIG.

In the case of this system using multicast, the conference terminal devices 1a to 1e participating in the conference are used.
Transmit data to each other by multicast. Then, each of the conference terminal devices 1a to 1e receives the multimedia data of the number of the other conference terminal devices participating in the conference, and processes each of the received data at its own terminal. Then, as shown in FIG. 22, images from other terminals participating in the conference are displayed individually and independently.

FIG. 22 shows a display screen of the display 4 of the conference terminal device 1e at the point E in FIG.
6a to 6d are terminals 1a to 1d at points A to D in FIG.
This is an independent video window for displaying a restored image of video information transmitted from the video window.

However, in the case of a system using this multicast, individual conference terminal devices are required to process information having a large data amount, such as video, for all terminals participating in the conference. Insufficient ability, and even if dedicated hardware is used, it is very difficult to smoothly display images from all terminals. In particular, as the number of terminals participating in a multimedia conference increases, it becomes more difficult.

It is conceivable to prepare a decoder for the video to be transmitted in each conference terminal device, but this is disadvantageous in terms of cost and hardware scale and is impractical.

As one means for solving the above problems, for example, Japanese Patent Application Laid-Open No. 8-163556 discloses a receiving terminal (for receiving conference-related information among a plurality of conference terminal devices connected to a network). The same applies to a conference terminal device, the same applies hereinafter) provided with a frame rate conversion unit, so that the user sets the frame rates of a plurality of videos according to the computer processing capacity of the receiving terminal so as to reduce the frame rates of videos that do not need to be noticed It has been proposed to. According to this, it is possible to prevent computer power from being wasted when displaying an image, and to prevent deterioration in quality of the image of interest.

[0015] For example, Japanese Patent Application Laid-Open No. 9-74548 discloses means for solving the above-mentioned problems. That is, in the system described in Japanese Patent Application Laid-Open No. 9-74548, the number of valid video frames transmitted to the receiving terminal is measured, and the result is measured. It transmits the conference-related information among the plurality of connected conference terminal devices, or transmits it to the transmitting conference terminal device (hereinafter the same). The transmitting terminal controls the frame rate of the moving image to be transmitted according to the information. With the above configuration, the receiving terminal can always process the moving image data within the range of the calculation processing capability.

[0016]

However, the prior art described in the above publication has the following problems.

That is, in the case of the invention disclosed in Japanese Patent Application Laid-Open No. 8-163556, the frame rate must be set by the user himself / herself at the receiving terminal every time, which is very troublesome. There is a problem that there is. For example, if the setting is made so that the frame rate from the terminal of the speaker is increased each time the speaker changes, it is very troublesome and the user cannot concentrate on the conference.

In the example disclosed in Japanese Patent Application Laid-Open No. 9-74548, when the processing capability differs among a plurality of receiving terminals, it is difficult for the transmitting terminal to deal with each of them. In general, the video to be focused on is the video of the speaker or a special video provided by the speaker. If the video is not clearly visible, the conference will not proceed smoothly and it will stress the participants of the conference . However, in the example of Japanese Patent Application Laid-Open No. 9-74548, there is a problem that the receiving terminal cannot always give priority to the specific video and display it with high quality.

In view of the above points, the present invention provides information that should always be given priority without special conscious operation by the user during a conference even when a large amount of video data cannot be processed due to low computational capability. It is an object of the present invention to provide a conference terminal device for a multipoint multimedia conference, which can preferentially process.

[0020]

According to a first aspect of the present invention, there is provided a conference which is connected to a network and which transmits conference-related information such as audio and video through the network. In the terminal device,
An input unit for receiving an input of the conference-related information; a specific information presence / absence detection unit for detecting whether or not predetermined specific information is included in the information from the input unit; In response to the result, the priority information for giving priority to the processing at the receiving conference terminal based on the presence or absence of the specific information is generated, and the conference-related information from the input unit is converted into a signal to be transmitted to the network. Information processing means for generating a transmission signal by adding the priority information and the identifier of the terminal, and data transmission means for transmitting a transmission signal from the information processing means to a receiving terminal through the network And the following.

According to the second aspect of the present invention, a voice information input unit for receiving input of voice information, another information input unit for receiving input of conference-related information other than voice, and Voice information presence / absence detecting means for determining the voice level of voice information and detecting the presence / absence of input voice information; and processing at the receiving conference terminal based on the presence / absence of the voice information in response to the detection result of the voice information presence / absence detecting means. While generating priority information to give priority to, the audio information from the audio information input unit and other conference related information from the other information input unit is converted into a signal to be sent to the network, Information processing means for generating a transmission signal by adding the priority information and the identifier of the own terminal; and transmitting the transmission signal from the information processing means to the receiving terminal through the network. A data transmission means for, characterized in that it comprises a.

According to the third aspect of the present invention, a voice and a voice are connected through a network.
In a conference terminal device for transmitting conference-related information such as video, an input unit that receives the input of the conference-related information, and a process of converting the conference-related information from the input unit into a signal to be transmitted to the network Also, with reference to the user's instruction to prioritize the processing at the receiving terminal, when the conference-related information from the input unit should prioritize the processing at the receiving terminal, the priority indicating that Information, and to the signal to be sent to the network, the priority information and the identifier of its own terminal is added,
Information processing means for generating a transmission signal, and data transmission means for transmitting a transmission signal from the information processing means to a receiving terminal through the network.

According to a fourth aspect of the present invention, in the conference terminal device connected to a network for transmitting conference-related information such as audio and video through the network, an input unit for receiving the input of the conference-related information. With respect to the conference-related information, a condition for prioritizing processing at the receiving terminal, priority condition setting means for setting in advance in response to an instruction from a user, and for the conference-related information from the input unit, A priority condition confirmation unit for confirming whether or not a condition set by the priority condition setting unit is met; and a process of converting conference-related information from the input unit into a signal to be transmitted to the network, When the result of the confirmation by the condition confirmation means indicates that the processing related to the conference should be prioritized in the receiving terminal, the priority information indicating that Information processing means for generating a transmission signal by adding the priority information and the identifier of the terminal itself to a signal to be transmitted to the network, and transmitting the transmission signal from the information processing means through the network. Data transmitting means for transmitting to the receiving terminal.

According to a fifth aspect of the present invention, in the conference terminal device according to the fourth aspect, the conference-related information is audio,
Information including a plurality of pieces of information such as video, wherein the priority condition setting means sets information selected by the user's instruction input from the conference-related information to information for which processing at the receiving terminal is to be prioritized. When the priority condition confirmation means includes the information selected by the user's instruction input set by the priority condition setting means, the processing of the conference-related information at the receiving terminal is performed. It is characterized in that it is determined that priority should be given.

According to a sixth aspect of the present invention, there is provided a conference terminal device connected to a network for transmitting conference-related information such as audio and video through the network. And a condition for prioritizing the processing at the receiving terminal with respect to the conference-related information, as a condition for including any of the conference-related information when the processing is performed at the receiving terminal. The condition information, along with the identifier of the terminal itself,
Priority condition transmitting means for transmitting to the receiving terminal in advance, and processing for converting the conference-related information from the input unit into a signal to be transmitted to the network, and determining the identifier of the own terminal and the priority condition on the receiving side Information processing means for generating a transmission signal by adding information to be used to perform the transmission, and data transmission means for transmitting a transmission signal from the information processing means to a receiving terminal through the network. It is characterized by.

According to a seventh aspect of the present invention, in any one of the first to seventh aspects, the information processing means packetizes each of a plurality of pieces of information constituting the conference-related information, and multiplexes the packets. It is characterized in that it includes means for converting.

The invention according to claim 8 is a conference terminal device connected to a network and receiving transmission signals from a plurality of other conference terminal devices through the network, wherein the transmission signal transmitted through the network is transmitted. A data receiving unit for receiving a signal, recognizing a terminal identifier included in the transmission signal received by the data receiving unit, and storing the received data in a buffer memory for each conference terminal device corresponding to the terminal identifier. Buffer means for separating and storing; decoding means for performing a decoding process on received data stored in the buffer means; and specific information which is predetermined in the conference-related information received by the data receiving means. Is recognized, and for the received data containing the specific information, the decoding unit gives priority to the received data. A priority control means for executing the decoding, characterized in that it comprises a.

According to a ninth aspect of the present invention, in the conference terminal device according to the eighth aspect, the specific information is audio information.

According to a tenth aspect of the present invention, there is provided a conference terminal device connected to a network for receiving transmission signals from a plurality of other conference terminal devices through the network, wherein the transmission terminal transmits the transmission signal through the network. A data receiving unit for receiving a signal, recognizing a terminal identifier included in the transmission signal received by the data receiving unit, and storing the received data in a buffer memory for each conference terminal device corresponding to the terminal identifier. Buffer means for storing divided data, decoding means for decoding received data stored in the buffer means, and recognition of priority information included in the transmission signal received by the data receiving means. For the reception data designated as priority, a priority control means for controlling the decoding means to execute decoding preferentially. Characterized in that it comprises a and.

An eleventh aspect of the present invention is a conference terminal device connected to a network and receiving transmission signals from a plurality of other conference terminal devices through the network, wherein the transmission signal transmitted through the network is transmitted. A data receiving unit for receiving a signal, recognizing a terminal identifier included in the transmission signal received by the data receiving unit, and storing the received data in a buffer memory for each conference terminal device corresponding to the terminal identifier. A buffer means for storing divided data, a decoding means for decoding received data stored in the buffer means, and a decoding processing by the decoding means for the conference-related information is prioritized. Priority condition for determining which of the conference related information should be given priority to the decoding process Information, together with the identifier of the transmitting side terminal, in advance, priority condition storage means, and among the priority condition information stored in the priority condition storage means, the priority condition information relating to the terminal identified by the terminal identifier With reference to the above, it is recognized whether the conference-related information received by the data receiving unit matches the priority condition information, and for the reception data matching the priority condition information, the decoding unit And priority control means for controlling to execute decoding preferentially.

According to a twelfth aspect of the present invention, in any one of the eighth to eleventh aspects, the transmission signal of the conference-related information is obtained by packetizing and multiplexing information such as video and audio. The apparatus further includes a separating unit that separates the multiplexed packets of the conference-related information received by the data receiving unit for each information.

[0032]

According to a first aspect of the present invention, there is provided a conference terminal device on the transmitting side, wherein specific information predetermined between the transmitting terminal and the receiving terminal is included in the conference related information to be transmitted. , Priority information is added to the transmission signal. In the receiving terminal, the transmission signal to be prioritized is processed with priority over the transmission signal from another conference terminal device and decoded with high quality, based on the priority information included in the transmission signal.

According to a second aspect of the present invention, there is provided a conference terminal device on the transmitting side. In particular, when voice information is included in the conference-related information to be transmitted, priority information is added to the transmission signal.
That is, when the user using the conference terminal device is the speaker, the priority information is added to the transmission signal, and the reception terminal transmits the transmission signal from another conference terminal device. The signal is processed prior to the signal and decoded with high quality.

As described above, in the conference, attention is paid to the video of the speaker (speaker) or the video provided by the speaker, so that at least the video of the speaker or
It is desirable that only the video provided by the speaker is displayed with high quality, but according to the invention of claim 1, the receiving terminal uses the conference terminal device used by the user who is the speaker. Is always processed with priority and is decoded with high quality.

Also, the invention of claim 3 is a conference terminal device on the transmitting side. In the conference terminal device of claim 3, whether the priority information is included in the transmission signal by referring to the instruction from the user is described. It is determined whether or not the transmission signal is to be transmitted to the receiving terminal. Therefore, the priority information is added to the transmission signal according to the user's designation on the transmitting side, not only when the voice information is present, but also when necessary, and the processing is preferentially performed by the receiving terminal. Is done.

Also in the case of the third aspect, since the receiving terminal processes the transmission signal to which the priority information is added in preference to other transmission signals, the user is particularly conscious. There is an effect that information that should always be prioritized can be processed preferentially without performing any operation.

The invention of claim 4 is also a conference terminal device on the transmitting side. In the conference terminal device of claim 4, conditions for allowing the user to prioritize the processing on the receiving terminal in advance. (Priority condition) is set and entered,
It is set by the priority condition setting means. Then, the priority condition confirmation means determines and confirms whether or not the conference-related information input from the input unit matches the set priority condition. When the result of the check indicates that the priority condition is met, the information processing means causes the transmission signal to include the priority information and the identifier of the terminal itself as additional information.

Upon receiving this through the network, the receiving terminal detects the presence of the priority information,
It is automatically determined whether or not the received transmission signal is to be processed with priority, and a decoding process is performed.

In the case of the present invention, since the user can set the priority condition, for example, information that the user needs to reproduce in high quality is reproduced as intended by the user. Will be able to do it.

In the case of the invention of claim 5, as the priority condition in claim 4, it is possible to set that the receiving terminal performs priority processing when any information is included in the transmission signal. . For example, the user can set such that only when the voice of the speaker is included in the transmission signal, or only when the video information subjected to the specific compression is included in the transmission signal. For example, a conference video material is MPEG
When it is determined that the compressed data is used, when the MPEG-compressed video data is included in the transmission signal, priority information is automatically added to the transmission signal, and the receiving terminal reliably performs priority processing. You can do so.

The invention according to claim 6 is also a conference terminal device on the transmitting side. In the case of the conference terminal device according to claim 5, the priority condition information is transmitted by the priority condition transmitting means together with the identifier of the terminal. , To the receiving-side conference terminal device. When the priority condition information includes any information of multimedia information such as video and audio,
This is information on whether the process is prioritized on the receiving side.

Therefore, in the receiving terminal, the priority conditions are stored in advance for each conference information terminal of each transmitting side, so that when a transmission signal is received, any conference terminal is identified by the identifier contained therein. A conference terminal that recognizes whether the signal is a transmission signal from an apparatus, determines what information is included as multimedia information constituting the transmission signal, and transmits the transmission signal stored in advance. By referring to the priority information about the device,
It can be automatically determined whether or not to give priority to this.

According to a seventh aspect of the present invention, there is also provided a conference terminal device on the transmitting side. In the conference terminal device according to the seventh aspect, the multimedia information as the conference-related information includes a packet for each information such as video and audio. And the plurality of packets are multiplexed to form a transmission signal. Therefore, the reception-side conference terminal device can easily know what information is being transmitted as the transmission signal only by detecting the packet structure of the transmission signal, and can also determine each information such as video and audio. Is easy to separate.

The invention of claim 8 is a conference terminal device on the receiving side. In the conference terminal device of claim 8, the received transmission signal is transmitted to any of the other conference terminal devices by using an identifier contained therein. It recognizes whether or not the signal is a transmission signal. Then, the transmission signal is temporarily stored in the buffer memory corresponding to the identified conference terminal device. And
The priority control means recognizes whether or not the received signal includes predetermined specific information, for example, audio information. If the specific information includes, for example, audio information, the received information indicates Performs the decoding process prior to information from other terminals.

Therefore, in the conference terminal device on the receiving side, it is possible to preferentially decode, for example, video information to be displayed with high quality without any troublesome operation such as selecting information to be prioritized by the user. Can be.

In particular, when the specific information is audio information as described in claim 9, as described above, priority is given to the image such as the hand gesture and gesture of the speaker and the information provided by the speaker. High quality decoding is possible.

According to a tenth aspect of the present invention, there is provided a conference terminal device on the receiving side, wherein the priority information is included in the transmission signal received by the data receiving means. In such a case, the received data is preferentially subjected to decoding processing by the decoding means. Therefore, in the case of the tenth aspect, it is possible to easily perform the priority decoding process simply by recognizing the priority information included in the transmission signal.

The invention of claim 11 relates to a conference terminal device on the receiving side, wherein in the conference terminal device of claim 11, another conference terminal device connected to the network is stored in the priority condition storage means. Priority conditions for each are accumulated. The priority condition information is information indicating what information of the conference related information should be included in the priority of the decoding process.

When receiving the transmission signal, the priority control means refers to the priority condition information for the conference terminal device which has transmitted the transmission signal, and determines whether or not the transmission signal matches the priority condition information. recognize. Then, control is performed such that the decoding unit preferentially executes decoding of the matching received data.

According to the eleventh aspect of the present invention, different priority conditions are set for each conference terminal device by storing priority condition information for each conference terminal device in the conference terminal device on the receiving side in advance. Even so, priority processing corresponding to that can be performed.

According to the twelfth aspect of the present invention, the eighth to eighth aspects are described.
According to the eleventh aspect of the present invention, since the conference-related information is packeted for each piece of information, the processing for each piece of information is easy. For example, it is possible to realize a priority process for only video information with a simple configuration. it can.

[0052]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Several embodiments of a conference terminal device according to the present invention will be described below with reference to the drawings. The conference terminal device according to the embodiment described below is applied to the multipoint multimedia conference system as described above. In the multipoint multimedia conference system in this case, as shown in FIG. 21, a plurality of conference terminal devices installed at multipoints are connected to each other through a network, and mutually transmit and receive information in a multicast format. This is the case when you want to do it.

[First Embodiment] In the first embodiment, during a multipoint multimedia conference, the user should always prioritize images to be noticed without any special conscious operation. Transmission including packetization means for packetizing multimedia data by adding priority information when transmitting data including information to be processed with priority in order to enable high quality display. The receiving side terminal and the receiving side terminal provided with priority terminal selecting means for detecting the priority information included in the receiving information and selecting data from which transmitting side terminal should be processed with priority. Build a point multimedia conference system.

In the first embodiment, when predetermined priority data is included in a transmission signal, priority information is additionally inserted into a transmission signal from a transmitting terminal. In the example to be described, the priority data is voice information so that transmission information from the conference terminal device used by the speaker (speaker) is processed with priority.

Each of the conference terminal devices connected to the network has the above-described configuration of the receiving terminal and the configuration of the transmitting terminal.

<Configuration of the transmitting side of the conference terminal device> (corresponding to claims 1, 2 and 7) FIG. 1 shows an example of the configuration of the transmitting side of the conference terminal device of the first embodiment. 11v is an input terminal for video information,
11a is an input terminal for audio information. The portion surrounded by a dotted line is a portion constituting the information processing means.

In this case, the video signal output terminal of the video camera 21 is connected to the video information input terminal 11v. The video camera 21 captures images of surroundings including conference participants who are users of the conference terminal device and explanatory materials, and supplies the captured output signal to the video information input terminal 11v.

The audio information input terminal 11a is supplied with an audio signal of a microphone for mainly picking up a speech of a user of the conference terminal device as a conference participant.

The video signal input through the input terminal 11v and the audio signal input through the input terminal 11a are supplied to the encoding unit 12, where they are converted into digital signals and encoded. For simplicity of explanation,
In the example of FIG. 1, the encoding unit 12 collectively shows a video signal encoding unit and an audio signal encoding unit.

Regarding the codec system of the video information and the audio information in this embodiment, the ITU which is generally used in such a multimedia conference system is used.
−T (International Telecommunication Union-Teleco
mmunication) is used. That is, for the video signal, H.264 is used. 261
And H. H.263 is used. 711, G.R. 722 and G. 728 or the like is used.

Which one of the plurality of codec systems should be selected depends on the network to which the conference terminal is connected. For example, when connecting to the Ethernet, the video and audio codecs are respectively set to H. . 263, G.R. 711 is used.

Of course, MPEG or a completely different codec may be used, or a plurality of codecs may be prepared and selected and used. In addition,
When a plurality of codecs are selected and used, for example, a video codec often uses DCT (Discrete Cosine Transform) in many cases.
It is desirable that only the module that executes DCT be in a form that can be commonly used regardless of the codec format.

The video data and audio data encoded by the encoding unit 12 are supplied to a packetizing unit 13.

The audio signal input from the microphone 22 is also supplied to the level detector 15, and the audio signal level is detected. The level detector 15 samples the input analog audio signal as shown in FIG. 2 at appropriate time intervals, and A / D converts it into an 8-bit digital signal as shown in FIG. Are detected in time series.

The audio signal levels detected in time series from the level detecting section 15 are sequentially sent to a level comparing section 16 and compared with a preset reference level to determine whether or not the user has made a statement. That is, the presence or absence of the utterance voice is detected. The reference level is set to a value at which noise and speech uttered by the user can be identified and detected. Since this reference level may vary depending on the microphone or sound source board used, each conference terminal device is first set, and the reference level is determined from the noise level empirically obtained. Therefore, the reference level may be different for each conference terminal device.

In the level comparing section 16, the level detecting section 15
If the level of the audio signal detected in step (1) is higher than the reference level, it is determined that audio information is included as multimedia data transmitted from the transmitting terminal. That is, it is determined that the user of the transmitting terminal is speaking.

When the level of the audio signal detected by the level detecting section 15 is lower than the reference level, the level comparing section 16 includes the audio as multimedia data transmitted from the transmitting terminal. Not, ie
It is determined that the user of the transmitting terminal has not spoken.

Then, the level comparing section 16 supplies information on the result of the judgment to the packetizing section 13.

The packetizing section 13 packetizes the multimedia data encoded by the encoding section 12. In addition, such a multimedia conference system often has a function of sharing application software between terminals in many cases, so that the packetizing unit 13 sends the packet from the CPU 14 through the main bus.
Data other than video and audio related to such a function is similarly packetized.

When packetizing the multimedia data, the packetizer 13 refers to the determination result of the presence / absence of the audio signal from the level comparing section 16 and adds audio to the data to be transmitted. When it is recognized that a signal is included, priority information is added to all data to be transmitted in synchronization with the recognized voice information. According to the first embodiment, the priority information indicates that the data transmitted from the transmitting terminal includes voice in the receiving terminal and that the data is to be processed with priority. It is for notifying that.

Therefore, in the first embodiment, the packetizing section 13 includes the priority information in the header of each packet. FIG. 4 schematically shows the structure of a packet in the case of this embodiment. Each packet is composed of a header part and a data part.

In the header portion, a packet identifier PID (packet ID) indicating which multimedia data packet such as video or audio is included,
Time stamp TS indicating the time when the data occurred
In addition to the identifier TID (terminal ID) of the terminal itself,
The priority information PR is included. By referring to the time stamp TS, it is possible to detect information to be synchronized, that is, information in the same frame. Also,
As the identifier ID of the terminal, for example, the address of the terminal on the network 10 is used. Of this information,
For example, in IP (Internet Protocol), since it is included in the header of the data or the like, it can be used.

The priority information PR is, for example, 1-bit information. For example, when audio information is included in the transmission signal, the bit of the priority information PR is set to “1”, and the audio signal is included in the transmission signal. Otherwise, the bit of the priority information PR is set to "0". Therefore, when voice information is included in the transmission signal, as described above, the bits of the priority information PR in the header portion of the voice information packet and all the packets transmitted in synchronization with the voice information packet are set to “1”. ". This information can be incorporated by assigning, for example, user bits prepared in the above-described IP header.

Each data packetized in this way is supplied to the multiplexing unit 17. The multiplexing unit 17 performs multiplexing as shown in FIG. 5 so that a plurality of multimedia data, for example, video and audio can be synchronized when separated at the receiving terminal. In FIG. 5, video packets and audio packets in one frame are data generated at the same time. Each may not always fit in one packet, and may be a plurality of packets. In FIG. 5, the header of each packet is indicated by hatching at the head of each packet.

As described above, the packet data of the multimedia data multiplexed by the multiplexing unit 17 is passed to the data transmitting unit 18 and sent out to the network 10,
The message is transmitted to the conference terminal device on the receiving side.

FIGS. 6 and 7 show flowcharts of the operation on the transmitting side of the conference terminal device described above. That is, when the conference terminal device receives an input of multimedia data such as video and audio (step S1),
The multimedia data is encoded (step S2), and the detection of the presence or absence of the input voice is checked (step S3).

The flowchart of FIG. 7 shows the flow of the detection check of the presence or absence of the input voice. That is, an analog audio signal from the input terminal 11a is sampled (step S11), and the sampled value is D / A converted to detect an audio level (step S12). Then, each D / A conversion value is compared with a reference level (step S).
13) As a result of the comparison, it is determined whether or not the D / A conversion value exceeds the reference level (step S14). If the D / A conversion value exceeds the reference level, information indicating that there is a voice is output as a detection check result (step S14). S15) If it does not exceed the reference level, information indicating no sound is output as a detection check result (step S16).

When the result of the detection check for the presence or absence of the input voice is obtained in step S3 as described above, it is determined whether or not the input voice is present (step S4). When it is determined that there is an input voice, that is, when it is determined that the user of the conference terminal device is speaking, the process proceeds to step S5, and the priority information to be added to the header portion of each packet of the multimedia data is set to “priority information”. (The bit of the priority information PR is set to "1"), and when it is determined that there is no input voice, the process proceeds to step S6, where the priority information added to the header portion of each packet is set to "non-priority". (The bit of the priority information PR is “0”).

Next, each packet of the multimedia data is multiplexed (step S7), and the multiplexed data is transmitted to the network 10 (step S8).

<Configuration of Receiving Side of Conference Terminal> (Corresponding to Claims 10 and 12) Next, the receiving side of the conference terminal will be described. FIG.
FIG. 2 is an example of a configuration diagram of a data reception and decoding processing part of the multipoint multimedia conference terminal device according to the present invention. In FIG. 8, a portion surrounded by a dotted line is a portion constituting information processing means.

The data receiving section 31 receives the multiplexed multimedia data packet transmitted through the network 10. These data are sent to the demultiplexing unit 32, and the multiplexed packets are converted into video packets,
Separate for each media such as voice packet and application packet.

The packets for each medium transmitted from the transmitting terminal separated by the separating unit 32 are decomposed by the packet decomposing unit 33. As described above, since the identifier ID of the transmitting terminal (address of the transmitting terminal) is added to each packet, it is determined here from which transmitting terminal the multimedia data has been transmitted. Is done.

The video and audio data extracted by decomposing the packet are stored in the buffer memory corresponding to the transmitting terminal among the buffer memories 34a to 34d from the packet decomposing unit 33. Buffer memories 34a to 34
“d” is prepared for several conference terminals connected to the network 10. The example of FIG.
1 is a configuration example of the receiving side of the conference terminal device at the point E in FIG.
Buffer memories 34a to 34d are provided,
The multimedia data is sorted for each transmitting terminal and temporarily stored. Further, the buffer memories 34a to
In each of 34d, information is stored for each medium such as video and audio.

The video and audio data from the buffer memories 34a to 34d are sent to the decoding unit 35 in a data unit of one frame and are decoded. Decoding unit 35
In this embodiment, the buffer memories 34a to 34a
This is not a configuration in which all the data from 4d are processed in parallel at the same time, but one frame of data from any one of the buffer memories 34a to 34d.

When one frame of data is stored in any of the buffer memories 34a to 34d, the data is transmitted from the buffer memory to the decoding unit 35. In the decoding unit 35, 1 is stored in the buffer memories 34a to 34d.
When the notification that the data for the frame has been accumulated arrives, it is recognized that the decoding process of the data for one frame in the buffer memory to which the notification has arrived is enabled. Then, one frame of data in the buffer memory is decoded.

In this case, when there is no priority processing as will be described later, one frame of data is decoded so that the order of decoding is even among the four buffer memories. Therefore, at this time, there is no difference in the image quality of the images from the four conference terminal devices 1a to 1d (in this example, whether or not there is frame skipping).

In FIG. 8, for the sake of simplicity, the decoding section 35 integrally shows the video and audio decoding sections, but actually, the decoding section 35 for video information and the decoding section for video information are combined. And a decoding unit for audio information. This decoding unit 35
Corresponds to the encoding method adopted in the encoding unit 12 of the transmitting terminal. Of course, in a case where a plurality of codecs are selected and used as the encoding method, the decoding unit 35 is configured to have the functions of a plurality of decoders correspondingly. In this case, similarly to the transmitting side, the inverse DCT is used in most cases regardless of the codec format. Therefore, it is preferable that only the inverse DCT module be used regardless of the codec.

Also, in actuality, video decoding takes more time to process than audio decoding. For example, a delay circuit is inserted to eliminate the delay difference between the video and audio processing delays, and the Are synchronized, but this point is also omitted in FIG.

As mentioned in the background section of the prior art,
There is no problem if the processing capacity of the decoding unit 35 is very high and the data stored in the buffer memories 34a to 34d can be processed without delay, but if the number of terminals participating in the multimedia conference is large, the decoding unit 35 The amount of data that needs to be processed also increases, and cannot be processed in practice.

Therefore, in this embodiment, based on the priority information included in each packet of the transmission signal,
When the information transmitted from the transmitting terminal includes data to be subjected to priority processing in the decoding unit 35, the decoding unit 35
The data is controlled so as to be processed preferentially.

That is, the packet disassembly unit 33 extracts the priority information PR inserted and added to the header of each packet of the received transmission signal, and sends the extracted priority information to the priority terminal selection unit 36 together with the terminal identifier ID. The degree information PR is sent. The priority terminal selection unit 36 determines the priority information P
R is analyzed to determine whether the received transmission signal from the transmitting terminal is to be processed with priority. That is,
When the bit of the priority information PR is “1”, it is determined that the priority processing should be performed, and when it is “0”, it is determined that the priority processing is not necessary. And the information of the judgment result is
This is sent to the decoding unit 35.

In the case of this embodiment, the priority terminal selection unit 3
The information of the determination result from No. 6 relates to the priority processing instruction for the four buffer memories 34a to 34d in the decoding unit 35 and is 4-bit information. Each bit of the 4-bit priority processing instruction information indicates whether information transmitted from each of the conference terminal devices at the four points A to D is subjected to priority processing. In this example, the bit is " Priority processing of information from a terminal having a value of "1" and non-priority processing of information from a terminal having a bit of "0".

FIG. 9 is a diagram for explaining an example of the 4-bit priority processing instruction information. Priority terminal selection unit 36
Analyzes the priority information from the packet decomposer 33, and manages the priority and non-priority of the information from each buffer memory in the format shown in the table of FIG. In FIG. 9, こ と means that the transmitting terminal is given priority, and × means that there is no need to give priority. The 4-bit priority processing instruction information is set to “1” for the former (○) and to “1” for the latter
Is generated by associating "0" with.

In the example of FIG. 9, at times t1, t2, and t3, information from the terminal at the point A is processed with priority. Also indicates that the non-priority information is in a non-priority situation. At time tn, the transmission information from the conference terminal device at the point B and the transmission information from the conference terminal device at the point C are to be processed with priority.

In the decoding section 35, if the 4-bit priority processing instruction information from the priority terminal selection section 36 indicates that information from any terminal is to be processed with priority, the decoding section 35 Even if a notification that data for one frame has been accumulated from the buffer memory for the terminal of
In accordance with the instruction of the priority processing instruction information from the priority terminal selection unit 36, the decoding processing of the information from the terminal to be prioritized is continuously performed.

At times t1, t2, and t3 in FIG. 9, the information from the buffer memory 34a is given priority and the decoding unit 35
Is subjected to priority decoding processing.

When there are a plurality of terminals for which priority processing is to be performed as at time point tn in FIG. 9, the decoding processing is performed alternately between these terminals. When the 4-bit priority processing instruction information is “0000”, there is no need to perform priority processing, so the decoding unit 35 performs equal processing on the data in the four buffer memories 34 a to 34 b in a predetermined order. Perform the decryption process.

The video and audio data decoded by the decoding unit 35 as described above are output from the output terminal 38, respectively.
Output to the display 41 and the speaker 42 through v and 38a for reproduction.

In the packet disassembly unit 33, data that needs to be calculated and that is extracted from the packet disassembly and related to a function such as sharing of application software between terminals passes through the main bus and passes through the CPU 37.
Sent to.

FIG. 10 shows a flowchart of the operation on the receiving side of the conference terminal device according to the first embodiment described above.

That is, the conference terminal device fetches multiplexed data, which is a transmission signal from any terminal, from the network 10 (step S21), and separates the multiplexed data thus obtained for each packet (step S2).
2). Next, each packet is disassembled (step S2).
3) The packet ID and the terminal ID of the additional information in the header are checked (step S24), and the audio data and the video data are separated and stored in the buffer memory corresponding to the transmitting terminal (step S25).

With the storage in the buffer memory, the priority information PR in the header is checked (step S).
26), a bit number corresponding to the number of other conference terminal devices other than the self, that is, a 4-bit signal in this example, is generated and supplied to the decoding unit 35 (step S27).

At the time of decoding, the state of each bit of the 4-bit signal is checked, and it is determined whether or not there is transmission data from the transmitting terminal to be prioritized (step S28).
When it is determined that there is data from the transmitting terminal to be prioritized, decoding of the information to be prioritized is performed prior to information from another terminal (step S29), and the output data is output. The video and audio are reproduced (step S33).

When it is determined that there is no data to be transmitted from the transmitting terminal which is prioritized, it is determined whether or not decoding is in progress. If decoding is in progress, the process in the decoding unit 35 is terminated. When waiting (step S31) and the processing in the decoding unit 35 is completed, the data from the four buffer memories 34a to 34d is decoded evenly at the same time (step S32). Then, the decoding result is output (step S33), and the video and audio are reproduced.

If the multipoint multimedia conference terminal having the above configuration is used, even when the conference terminal device has a low calculation function and cannot process a large amount of data such as video, the user is particularly conscious during the conference. Without operating
An image that should always be noticed can be preferentially displayed with high quality, so that the conference can proceed smoothly. That is, almost all transmitted images and sounds of interest are decoded and reproduced.

On the other hand, information from a terminal that is not designated as priority cannot be accumulated in the corresponding buffer memory even if it is transmitted because information from other terminals is processed with priority. Data with the information may be discarded. Then, when there is no terminal designated as priority, decoding of the video information of the non-priority terminal is performed. Therefore, the video output image of the non-priority terminal is in a state of so-called dropped frames. However, as described above, originally, the image information from the non-priority terminal is the degree to which the user pays attention to the output image. Since there are few meetings, there is almost no trouble in the meeting.

Particularly, in the above-described example of the first embodiment, the presence or absence of voice information is determined, and the voice information is included in the transmission signal, that is, the information from the user who speaks in the conference. Is given priority processing, it is possible to prevent the content of the utterance of the speaker from being interrupted during reproduction on the receiving side terminal, and prevent the operation of the utterer or the quality of the video of the material to be presented from being reduced.

In the configuration of the transmitting terminal shown in FIG. 1, the level detecting section 15 and the level comparing section 16
Since it is only necessary to be able to confirm the presence or absence of an audio signal by removing the influence of noise, it is preferable that the presence or absence of the audio signal is detected by another means other than the level detector 15 and the level comparator 16. Of course you can.

Further, the priority information PR can be added to a portion that can be distinguished from information such as video information and audio information, such as a head portion of the data portion of each packet, instead of the header portion.

Further, the priority information PR does not need to be one bit, but may be composed of a plurality of bits. In addition, the priority information PR is not always inserted and added to the header and data parts, but rather the audio information packet and
Only the packet transmitted in synchronization with the audio information packet may be inserted and added. In that case,
The receiving terminal simply determines whether or not the priority information exists in the header portion of the packet or the data portion of the packet,
It can be determined whether the information is to be given priority.

Further, in the above description, the priority is determined based on the presence or absence of the voice.
The specific image information, for example, the information to be submitted as a material to the conference, if the MPEG compression image information is included in the transmission signal when the MPEG compression image information is included in the transmission signal, the priority information as described above. May be included in the transmission signal and transmitted. That is, whether or not to give priority is not limited to voice, and can be determined based on whether or not predetermined information is included in the transmission signal.

[Second Embodiment] In the above-described first embodiment, when the multimedia data transmitted from the transmitting terminal includes specific information such as voice, the priority is lowered. The embodiment has been described in which information is added to a packet, and the receiving terminal recognizes the packet and determines the priority for processing.

However, the criterion for prioritizing the processing is not always based on specific information common to all conference terminal devices in advance, and there are cases where the user of each conference terminal device wants to change it on a case-by-case basis. Specifically, for example, when a user wants to explain an image taken at an exhibition or the like to each receiving terminal while presenting the image to each receiving terminal, at least the taken image may have a frame drop. There is a request to display with no high quality.

In view of this point, the second embodiment described below allows a user to set conditions for priority designation. As an example of this priority condition, in the following description, it is assumed that when specific information specified by a user is included in a transmission signal, a priority is given to processing on the receiving side. As the priority condition, for example, when a video of a specific source is included, or when a speech sound is included,
Specified by the user.

In the second embodiment, the priority condition is stored on the transmitting side, and when the signal to be transmitted matches the priority condition, the information on the priority condition is included in the transmission signal. In addition, it is transmitted to the receiving terminal.

Further, the transmitting terminal transmits the information of the set priority conditions at an appropriate time, for example, before the start of the conference.
It is transmitted to the receiving terminal in advance. The receiving terminal receives the priority condition information and saves it for each transmitting terminal. Then, when the receiving terminal receives the transmission signal from the transmitting terminal, it compares the priority condition information included in the received transmission signal with the priority condition information received and stored in advance, and Is checked to determine whether the processing should be performed with priority, and the processing is performed with priority.

<Configuration of transmitting side of conference terminal apparatus> (corresponding to claims 3, 4, 5, and 7) FIG. 11 shows a transmitting side terminal portion of the multipoint multimedia conference terminal apparatus according to the second embodiment. That is, this is an example of a configuration diagram of an encoding process and a data transmission portion. The part surrounded by the dotted line is a part constituting the information processing means.

In the configuration on the transmitting side according to the second embodiment, the level detection unit 15 and the level comparison unit 16 according to the first embodiment are not provided. 51, a priority condition setting unit 52, a priority condition confirmation unit 53, and an external memory 54 are provided. Other configurations are the same as those in the case of FIG. 1 of the first embodiment.

[0119] The external memory 54 stores image information to be provided as conference materials. In this example, the image information is stored as MPEG compressed data.

The setting operation section 51 is for setting and inputting conditions to be preferentially processed in the receiving terminal. In this example, information on what data should be transmitted from its own terminal and that the receiving side terminal wants to be processed with special priority is set and input.

In the case of this embodiment, information to be transmitted includes imaged video data of a conference participant from camera 21, audio data (speech) from microphone 22,
The data from the CPU 14 and the MP from the external memory 54
Four types of data including material video data in the EG compressed data format are assumed. The priority condition can be set so that when any of the four types of data is included, the receiving terminal wants to process the data with priority.

For example, if priority is desired when audio information is included, "audio information" is set and input. If priority is desired when the above-described MPEG-compressed video information is included, , And “MPEG information”.

The setting operation section 51 is composed of an input device such as a keyboard, a mouse, or a pen tablet, and is used to select and set the information as a priority condition. For example, as a simplest and easy-to-use mode, a method is adopted in which a pull-down menu MN as shown in FIG. 12 is projected on the screen and the user selects from the pull-down menu MN.

The information (hereinafter, referred to as priority condition information) about the data that the reception side preferentially processes on the receiving side, which is selectively set by the setting operation unit 51, is set in the priority condition setting unit 52. The priority condition setting section 52 has a memory, and the priority condition information is stored in the memory. The setting of the priority condition is performed by the user of the conference terminal before the conference starts.

The priority condition information set by the user is transmitted to a plurality of receiving terminals via the network 10 in the form of e-mail, for example, and the priority condition information of each receiving terminal is stored as described later. And the terminal ID together with the terminal ID. The priority condition information stored in the memory of the priority condition setting unit 52 is also supplied to the priority condition confirmation unit 53.

The priority condition confirmation section 53 receives all pieces of information that may be transmitted. That is, video data and audio data from input terminals 11v and 11a, MPEG data from external memory 54, CPU
And the data from 14 are input.

When transmitting a transmission signal from the own terminal to another terminal through the network 10 during a conference, the priority condition confirmation unit 53 detects the presence or absence of the four types of input signals, thereby detecting the transmission signal. Recognizes what multimedia information contains, as a result of referring to the priority condition set as described above, the recognized transmission signal is a multimedia data type that matches the priority condition, The priority condition information is transmitted to the packetizing unit 13.
And requests the packetizing unit 13 to add it to the transmission signal and transmit it to the network.

Also, in this example, the priority condition checking unit 53
When the recognized transmission signal does not match the set priority condition, it sends priority condition information without priority designation to the packetizing unit 13, adds it to the transmission signal, and sends it to the network. , To the packetizing unit 13.

In the second embodiment, the packet section 13 is used to packetize video data and audio data from the encoding section 12 and to share data from the CPU 14, for example, application software between terminals. Packetization of related data, MP from external memory 54
EG data is packetized.

Then, at the time of packetization, priority information is additionally inserted into each packet in the same manner as in the first embodiment. The priority information in the case of the second embodiment is information used when the receiving terminal determines whether or not the data should be processed with priority, and in the second embodiment, The priority condition information sent from the priority condition confirmation unit 53 is used as it is.

That is, when the transmission signal conforms to the set priority condition, the packetizing unit 13 transmits information equal to the set priority condition information from the priority condition confirmation unit 53 to the priority information. As described above, additional insertion is performed in the header of each packet in the same manner as in the first embodiment. If the transmission signal does not meet the set priority condition, the priority condition checking unit 5
The information without priority designation from No. 3 is similarly added and inserted into the header part of each packet as priority information to be added.

In this embodiment, the priority condition information is represented as a type of transmission multimedia data including the set information. That is, FIG. 13 shows the correspondence between the multimedia data type and the priority condition information. In this example, since four types of multimedia data types are assumed, the case where the priority condition information is represented by 2 bits is shown. Priority condition is 4
If more than two types can be set, the priority condition information is, of course, configured as information of three bits or more.

As shown in the table of FIG. 13, in this example, when the setting operation section 51 does not designate a priority condition, as shown in the table of FIG. There are four types of media data types: priority is not specified, MPEG video is included, utterance voice is included, Priority is MPEG video and utterance audio is included.

When the multimedia data type has no priority condition specified, the 2-bit priority condition information is “00”. When the multimedia data type includes MPEG video, the 2-bit priority condition information is “00”. 0
1 ", when the multimedia data type includes a speech voice, the 2-bit priority condition information is" 10 ", and the MPE
In the case of the multimedia data type including the G video and the speech sound, the 2-bit priority condition information is set to “11”.

In the memory of the priority condition setting unit 52, 2-bit priority condition information corresponding to the setting made by the setting operation unit 51 is written, and the same information is transmitted to the receiving terminal through the network 10 and the priority information is transmitted. It is stored in the condition information storage unit.

FIG. 14 is a flowchart of the operation of the priority condition setting unit 52 on the transmitting side of each conference terminal device described above.

That is, first, the user's priority condition setting / selection input via the setting operation section 51 is received (step S41). Then, the set priority condition is determined (step S42), and the 2-bit priority condition information is stored in the memory (step S43). Next, the two-bit priority condition information is transmitted to all other receiving terminals via the network, for example, in an e-mail format.
This is the end of the setting of the priority condition.

For example, when MPEG data is selected and set by the user, "01" is stored as a priority condition in the memory of the priority condition setting section 52 as the priority condition information, and this 2-bit information is stored in another memory. The data is sent to all of the receiving terminals via the network 10 and is stored and held in each of the receiving terminals.

After that, when the conference starts, as described above, the priority condition confirmation unit 53 transmits a transmission signal from the own terminal to another terminal through the network during the conference.
It recognizes what kind of multimedia information the transmission signal contains, and in accordance with the result of the recognition by the priority condition checking unit 53, the above-mentioned priority information is added to the packetizing unit and the receiving terminal is notified. Sent out.

The operation of the conference terminal device according to the second embodiment after the start of the conference on the transmitting side will be described with reference to the flowchart of FIG.

That is, when the conference starts, the first
Similarly to the embodiment, when video data and audio data are input from the camera 21 and the microphone 22 attached to the transmitting terminal (step S51), the multimedia data is encoded (step S5).
2).

In parallel with this encoding, the priority condition checking unit 5
In 3, by checking the presence or absence of four types of input signals,
The transmission signal is checked (step 53), and it is determined whether the priority condition set in the priority condition setting unit 52 is satisfied (step S54).

When it is determined that the set priority condition is met, the priority condition confirmation unit 53
The priority condition information stored in the memory of the priority condition setting unit 52 is sent to the packetizing unit 13, and the packetizing unit 13 adds the priority condition information to the header of each packet.
A packet of each multimedia data is generated (step S55).

For example, if the priority is set when the MPEG data is included in the transmission signal, and the information from the external memory 54, that is, the input of the MPEG data is confirmed, the transmission signal matches the priority condition. Then, the 2-bit information "01" is added to the header of each packet as priority information to generate each data packet.

On the other hand, in the judgment at step S54,
If it is determined that the priority condition does not match the set priority condition, in this example, the priority condition confirmation unit 53 sends “00” meaning “no designation” to the packetizing unit 13. , The 2-bit information “0
"0" is added to the header of each packet as priority information, and each data packet is generated (step S5).
6).

Next, each packet of the multimedia data is multiplexed (step S57), and the multiplexed data is transmitted to the network 10 (step S58). In the case of this example, a plurality of multimedia data when separated at the receiving terminal, for example, in this case,
Multiplexing can be performed so that animation can be synchronized in addition to video and audio.

<Configuration of Reception Side of Conference Terminal> (Response to Claims 11 and 12) Next, the reception side of the conference terminal will be described. FIG.
FIG. 2 is an example of a configuration diagram of a data reception and decoding processing part of the multipoint multimedia conference terminal device according to the present invention. In FIG. 16, a portion surrounded by a dotted line is a portion constituting information processing means.

The configuration of FIG. 16 is different from the configuration of the data reception and decoding processing part of the conference terminal device in the first embodiment shown in FIG. 8 in the second embodiment. In that the priority condition information storage unit 61 is provided and the operation of the priority terminal selection unit 36 is different. Other configurations are almost the same as those in the case of FIG. 8 of the first embodiment, and a detailed description thereof will be omitted.

That is, in the second embodiment, prior to the start of the conference, the priority condition information storage unit 61 sends each of the information transmitted in the form of e-mail via the network 10 and the data reception unit 31, for example. The priority condition information set in the transmitting terminal is stored for each transmitting terminal. As described above, the packet header includes
Although the address on the network is used as the terminal ID, the e-mail address is the address on the network itself. Therefore, the priority condition information storage unit 61 uses the terminal ID as an index and sets the priority condition of each transmitting terminal. The information is stored.

The priority condition information stored in the priority condition information storage section 61 is supplied to the priority terminal selection section 36. That is, the priority terminal selection unit 36 converts the terminal ID and the priority information obtained by decomposing the packet extracted from the transmission signal by the packet
3 and determines whether or not the signal arriving from the terminal should be subjected to priority processing by referring to the priority condition information of the terminal stored in the priority condition information storage unit 61. Then, based on the determination result, the priority terminal selection unit 36 generates 4-bit priority processing instruction information to be supplied to the decoding unit 35, as in the case of the above-described first embodiment.

FIG. 17 is a diagram for explaining an example in which the priority terminal selection section 36 generates 4-bit priority processing instruction information in the second embodiment. The priority terminal selection unit 36 receives the priority information from the packet decomposition unit 33,
By referring to and analyzing the priority condition information from the priority condition information storage unit 61, the priority and non-priority management of information from each buffer memory is performed in a format as shown in the table of FIG. Also in this case, as in the first embodiment, a terminal whose 4-bit priority processing instruction information bit is "1" is a terminal to be given priority, and a terminal whose bit is "0" is a terminal that does not need to be given priority. It means each.

The example of FIG. 17 is a case of the conference terminal device at the point E, and the conference terminal devices at the points A, B, and D do not specify the priority condition. The stored value of each piece of priority condition information of the terminal device is “00”, and the conference terminal device at the point C wants to perform the priority processing when the MPEG video data is included in the transmission signal. Is set, only the stored value of the priority condition information of the conference terminal device C is “01”.

In this case, at time t1, t2, the transmission signal arrives from the terminal at any point without the priority designation. In this case, the 4-bit priority processing instruction from the priority terminal selection unit 36 The information is “0000”, and the decoding unit 35 decodes the data from the four buffer memories in order and evenly.

At time ti, since the transmission signal from the conference terminal device at the point C has “01” added as priority information, this is detected by the priority terminal selection unit 36 and the decoding unit The 4-bit priority processing instruction information supplied to 35 is “0010”. That is, if the stored value of the priority condition information is “00”, no priority is given, and the corresponding bit of the conference terminal is
If it is always "0" and is other than "00",
"1" only when it matches the added priority information
Becomes As a result, the decoding unit 35 performs the decoding process with priority given to the information from the conference terminal device in the buffer memory 35c.

In the second embodiment, the decoding unit 35
In the case where the 4-bit priority processing instruction information from the priority terminal selection unit 36 indicates that information from any terminal is to be processed with priority, if the buffer memory for another terminal is used, Even if the notification that the data for one frame has been accumulated arrives, the decoding process of the information from the terminal to be prioritized is continuously performed according to the instruction of the priority processing instruction information from the priority terminal selection unit 36. To

When there are a plurality of terminals for which priority processing is to be performed, decoding processing is performed alternately between these terminals. The 4-bit priority processing instruction information is "000
When it is “0”, there is no need to perform processing with priority, so the decoding unit 35 uses the four buffer memories 34 a to 34 a to
It is the same as in the first embodiment that the decoding process is uniformly performed on the data of 34b in a predetermined order.

FIG. 18 shows a flowchart of the operation on the receiving side of the conference terminal device according to the second embodiment described above. This flowchart is substantially the same as the flowchart of the operation on the receiving side of the conference terminal device according to the first embodiment, except that the method of checking the priority information is different.

That is, steps S61 to S65 and steps S68 to S73 are the same as those in FIG. 10 of the first embodiment.
Correspond to steps S21 to S25 and steps S28 to S33 of the flowchart of FIG. The second embodiment is different from the first embodiment in the processing steps from step S66 and the checking of the priority information in step S67 to the formation of the 4-bit priority processing instruction information.

That is, in the second embodiment, as described above, when a transmission signal is received from the transmitting terminal, the processing up to step S65 stores the audio data and the like in the buffer memory corresponding to the transmitting terminal. The video data is separated and stored, and the terminal ID and the priority information obtained from the packet decomposing unit 33 are stored in the priority terminal selecting unit 3.
6 receives.

Then, the priority terminal selection unit 36 reads the priority condition information of the transmitting terminal indicated by the terminal ID stored in the priority condition information storage unit 61, and the read priority condition information is other than “00”. When the packet decomposing unit 33
Is compared with the priority condition information, and if they match, the bit corresponding to the terminal in the 4-bit priority processing instruction information is set to “1”. If the comparison results do not match, the bit corresponding to the terminal in the 4-bit priority processing instruction information remains "0". If the read priority condition information is “00”, no priority is given, and the corresponding bit is always “0”.

As a result, if the priority information matches the priority condition information, the information from the terminal is preferentially decoded in step S69. Otherwise, the decoding is performed in the four buffer memories 34a to 34a. The data from 34d is evenly decoded.

With the use of the conference terminal device of the second embodiment described above, as in the first embodiment, even if the terminal has low computational capability and cannot process a large amount of video data, the conference terminal can be used. An image that should always be noticed can be preferentially displayed with high quality without the user being particularly conscious of the operation during the operation, so that the conference can proceed smoothly.

In the case of the second embodiment, it is possible for the user to determine in advance the information to be processed preferentially on the receiving side. Since the receiving side is given priority processing to prevent the quality from deteriorating, the conference proceeds smoothly.

Note that the method of sending the priority condition information to a plurality of receiving terminals prior to the conference is not limited to the method using e-mail, but may be a packet transmission method or other various transmission methods. Needless to say, it can be used.

The priority information added to the conference-related information transmitted during the conference is not a header part but a part that can be distinguished from information such as video information and audio information such as the head of the data part of each packet. May be added.

Further, as described above, the priority information, that is, the priority condition information does not need to be 2 bits, and may be composed of 3 bits or more. Also, the priority information may not always be inserted and added to the header portion and the data portion, but may be inserted and added only to the packet of the transmission signal that matches the priority condition at the transmitting terminal. In that case, the receiving terminal can determine whether or not the priority information exists by simply determining whether or not the priority information exists in the packet header or the data portion of the packet. By comparing the received priority information with the priority condition information in the priority condition information storage unit, priority processing of information to be prioritized can be reliably performed without error.

Further, in the above description of the second embodiment, the case where the priority condition is merely set at the beginning of the meeting has been described. By operating, it is also possible to change and update the priority condition information flexibly.

[Other Embodiments] In the second embodiment, the priority condition checking means checks whether or not a transmission signal meets the priority condition.
Although transmission is performed with priority information equal to the priority condition information added, the multimedia data type of the transmission signal is determined in advance on the transmission side and the reception side, and the transmission signal packet is always included in the transmission signal packet. By adding the multimedia data type, it is possible to eliminate the need for the transmission side to check the priority condition.

That is, the priority condition checking unit 53 in FIG. 11 simply checks which of the four types of input signals and which are included in the transmission signal, and determines the multimedia data type according to the included information. Is checked. Then, the information of the multimedia data type is sent to the packetizer 13.

On the receiving side, the multimedia data type of the received transmission signal is always determined, and it is checked whether or not it matches the pre-stored priority condition information. It can be determined whether the data should be.

Thus, it is not necessary to supply the priority condition information from the priority condition setting unit 52 to the priority condition confirmation unit 53, and the priority condition confirmation unit 53 determines whether the transmission signal matches the priority condition. Need not be determined.

Further, in the second embodiment, the priority conditions set by the user in each conference terminal device are notified to the receiving terminal in advance, and each receiving terminal has a plurality of transmitting terminals. The priority condition information set in the above is collected and stored, and priority processing is performed based on whether or not the priority information included in the transmission signal matches the stored priority condition information of the transmitting terminal. Whether or not to set.

However, the priority information PR as described in the first embodiment is inserted only when the transmitting terminal transmits a transmission signal matching the preset priority condition to another receiving terminal. It can also be done.
In this case, the receiving terminal only needs to have the configuration shown in FIG. 8 in the case of the first embodiment, and notifies the receiving terminal of the priority conditions set in advance by the user in each conference terminal device. It is not necessary that the receiving side terminal store the priority condition information for each transmitting side terminal.

Also, in the first embodiment, the mode in which the transmitting terminal notifies the receiving terminal of the priority information based on the presence or absence of voice has been described. However, it is determined that the priority information is based only on the presence or absence of voice. If so, the same effect can be obtained without relying on the transmitting terminal, by performing the process of checking for the presence or absence of voice by the receiving terminal. In this case, it is not necessary to add the priority information to the transmission signal.

[Other Modifications] In the first embodiment and the second embodiment, both images input from outside,
A case has been described where the processing contents of the processing means for processing a plurality of multimedia data such as audio and the processing means for processing the multimedia data transmitted from the transmitting terminal include encoding and decoding, respectively. However, the present invention is not limited to this. For example, in the processing on the receiving side, the effects of the present invention described above can be obtained even if the processing is applied only to the display processing on the display.

In both the first embodiment and the second embodiment, priority information to be processed is sent to the receiving terminal, and the receiving terminal selects a terminal to be processed with priority based on the information. It is a form. However, if the processing power at all receiving terminals is obviously insufficient,
If it is known in advance, in addition to the priority selection processing on the receiving side, if priority does not have to be given, the amount of data to be transmitted can be reduced and transmitted.

For example, in the case of the first embodiment, voice is recognized, and in the case of the second embodiment, data set by the priority condition setting unit 51 is recognized by the level comparison unit and the priority condition confirmation unit 53, respectively. If not, the data amount may be reduced by a data amount reduction unit such as a frame rate conversion unit or a resolution conversion unit, and then encoded, and then packetized and transmitted in a packetization unit. Alternatively, instead of reducing the data amount, new video data may not be sent.

In this way, in addition to the effect of the present invention, an effect that the amount of data flowing on the network can be reduced can be obtained.

Further, in the first embodiment and the second embodiment, both the transmitting terminal and the receiving terminal are described separately, but as described above, in the multimedia conference, Since each conference terminal device performs both transmission and reception, it goes without saying that each conference terminal device has the functions of both the transmitting side and the receiving side described above.

[0180]

As described above, according to the present invention, in a multipoint multimedia conference, even if the conference terminal device has a low computational function and cannot process a large amount of video data, the user is not required to be able to process a large amount of video data. The user can always preferentially display a high-quality image without paying attention to the operation, and can smoothly proceed with the conference.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of a transmission side according to a first embodiment of this invention.

FIG. 2 is a diagram illustrating an example of an analog audio signal input from a microphone.

FIG. 3 is a diagram for explaining an example of a method for detecting the presence or absence of voice according to the first embodiment of the present invention.

FIG. 4 is a diagram for explaining an example of a form of a transmission signal according to the embodiment of the present invention.

FIG. 5 is a diagram for describing an example of a form of a transmission signal according to the embodiment of the present invention.

FIG. 6 is a flowchart for explaining an operation on the transmission side according to the first embodiment of this invention.

FIG. 7 is a flowchart illustrating an operation of detecting the presence or absence of a voice on the transmission side according to the first embodiment of this invention.

FIG. 8 is a block diagram illustrating a configuration example of a receiving side according to the first embodiment of this invention.

FIG. 9 is a diagram for explaining an operation of a priority terminal selection unit on the receiving side according to the first embodiment.

FIG. 10 is a flowchart for explaining the operation on the receiving side according to the first embodiment of this invention.

FIG. 11 is a block diagram illustrating a configuration example of a transmission side according to a second embodiment of this invention.

FIG. 12 is a diagram illustrating an example of a method of selecting and setting a priority condition according to the second embodiment.

FIG. 13 is a diagram illustrating an example of priority condition information according to the second embodiment.

FIG. 14 is a flowchart illustrating an example of a setting operation of a priority condition according to the second embodiment.

FIG. 15 is a flowchart illustrating an operation on the transmission side according to the second embodiment of this invention.

FIG. 16 is a block diagram illustrating a configuration example of a receiving side according to a second embodiment of this invention.

FIG. 17 is a diagram for explaining the operation of a priority terminal selection unit on the receiving side according to the second embodiment.

FIG. 18 is a flowchart for explaining the operation on the receiving side according to the second embodiment of this invention.

FIG. 19 is a diagram illustrating an example of a multipoint multimedia conference system.

20 is a diagram illustrating an example of a display screen of each terminal in the conference system of FIG. 19;

FIG. 21 is a diagram for explaining another example of the multipoint multimedia conference system.

FIG. 22 is a diagram illustrating an example of a display screen of each terminal in the conference system of FIG. 21;

[Explanation of symbols]

1a-1e Conference terminal device participating in multipoint multimedia conference 2,10 Network 3 MCU 4 Conference terminal device display 5a-5d Video window displayed on display 6a-6d Video window displayed on display 11v Video Information input terminal 11a Audio information input terminal 12 Encoding unit 13 Packetization unit 14, 37 CPU 15 Level detection unit 16 Level comparison unit 17 Multiplexing unit 18 Data transmission unit 31 Data reception unit 32 Separation unit 33 Packet decomposition unit 34a To 34d buffer memory 35 decoding unit 36 priority terminal selection unit 51 setting operation unit 52 priority condition setting unit 53 priority condition confirmation unit 54 external memory 61 priority condition information storage unit

Claims (12)

[Claims] 1. A conference terminal device connected to a network for transmitting conference-related information such as audio and video through the network, comprising: an input unit for receiving input of the conference-related information; Specific information presence / absence detecting means for detecting whether or not there is predetermined specific information in the information; processing at a receiving conference terminal based on the presence / absence of the specific information in response to a detection result of the specific information presence / absence detecting means; While generating priority information to give priority to, the conference-related information from the input unit is converted to a signal to be sent to the network,
Information processing means for generating a transmission signal by adding the priority information and the identifier of the terminal to the signal, and transmitting a transmission signal from the information processing means to a receiving terminal through the network A conference terminal device comprising: a data transmission unit. 2. A conference terminal device connected to a network for transmitting conference-related information such as audio and video through the network, comprising: a voice information input unit for receiving input of voice information; An other information input unit that receives input of conference-related information; a voice information presence / absence detecting unit that determines a voice level of voice information from the voice information input unit to detect presence / absence of input voice information; In response to the detection result of the means, while generating priority information that prioritizes the processing at the receiving conference terminal depending on the presence or absence of the voice information, the voice information from the voice information input unit and other information from the other information input unit Converting the conference related information into a signal to be sent to the network,
Information processing means for generating a transmission signal by adding the priority information and the identifier of the terminal to the signal, and transmitting a transmission signal from the information processing means to a receiving terminal through the network A conference terminal device comprising: a data transmission unit. 3. A conference terminal device connected to a network for transmitting conference-related information such as audio and video through the network, comprising: an input unit for receiving the input of the conference-related information; While performing the process of converting the conference-related information into a signal to be transmitted to the network, and referring to an instruction from a user to give priority to the process at the receiving terminal, the conference-related information from the input unit is used as the receiving terminal. When priority should be given to the process, priority information indicating the priority is generated, and a signal transmitted to the network includes:
Information processing means for adding the priority information and the identifier of the terminal itself to generate a transmission signal; and data transmission means for transmitting a transmission signal from the information processing means to a receiving terminal through the network. A conference terminal device comprising: 4. A conference terminal device connected to a network for transmitting conference-related information such as audio and video through the network, comprising: an input unit for receiving input of the conference-related information; A condition for prioritizing the process at the receiving terminal, a priority condition setting means for setting in advance in response to a user's instruction, and a conference-related information from the input unit, which is set by the priority condition setting means. Priority condition confirmation means for confirming whether or not the condition is met, and processing for converting the conference-related information from the input unit into a signal to be transmitted to the network, and a confirmation result by the priority condition confirmation means. Generates priority information indicating that the conference-related information should give priority to the processing at the receiving terminal, Information processing means for adding the priority information and the identifier of the own terminal to a signal to be sent to the work, and generating a transmission signal; and transmitting the transmission signal from the information processing means to a receiving terminal through the network. A conference terminal device comprising: 5. The conference terminal device according to claim 4, wherein the conference-related information includes a plurality of pieces of information such as audio and video, and wherein the priority condition setting unit includes: Setting the information selected by the user's instruction input as information for which processing at the receiving terminal is to be prioritized, wherein the priority condition confirming means includes a user information set by the priority condition setting means. A conference terminal device, wherein when the information including the information selected by the instruction input is included, it is determined that the processing related to the conference should be prioritized in the receiving terminal. 6. A conference terminal device connected to a network for transmitting conference-related information such as audio and video through the network, comprising: an input unit for receiving an input of the conference-related information; As a condition for giving priority to the processing at the receiving terminal, priority condition information indicating which of the conference related information should be given priority to the processing at the receiving terminal when including any information, Along with the identifier, in advance, priority condition transmitting means to transmit to the receiving terminal, and performs a process of converting the conference-related information from the input unit into a signal to be transmitted to the network, the identifier of its own terminal,
Information processing means for generating a transmission signal by adding information used for determining the priority condition on the reception side; and transmitting a transmission signal from the information processing means to the reception side terminal through the network. A conference terminal device comprising: a data transmission unit. 7. The information processing means according to claim 1, wherein said information processing means includes means for packetizing each of a plurality of pieces of information constituting said conference-related information and multiplexing said packets. A conference terminal device according to any one of the above. 8. A conference terminal device connected to a network and receiving transmission signals from a plurality of other conference terminal devices through the network, wherein the data receiving means receives a transmission signal transmitted through the network. Buffer means for recognizing a terminal identifier contained in the transmission signal received by the data receiving means, and storing the received data in a buffer memory for each conference terminal device corresponding to the terminal identifier Decoding means for decoding received data stored in the buffer means; and in the conference related information received by the data receiving means,
Recognizing in advance whether or not predetermined specific information is included, priority control is performed such that the decoding unit preferentially executes decoding with respect to received data that includes the specific information. A conference terminal device, comprising: control means. 9. The conference terminal device according to claim 8, wherein the specific information is audio information. 10. A conference terminal device connected to a network and receiving transmission signals from another plurality of conference terminal devices via the network, wherein the data receiving means receives a transmission signal transmitted through the network. Buffer means for recognizing a terminal identifier contained in the transmission signal received by the data receiving means, and storing the received data in a buffer memory for each conference terminal device corresponding to the terminal identifier Decoding means for decoding received data stored in the buffer means; and priority information included in the transmission signal received by the data receiving means, and reception designated by priority. For data, priority control means for controlling the decoding means to execute decoding preferentially, Conference terminal apparatus according to symptoms. 11. A conference terminal device connected to a network and receiving transmission signals from a plurality of other conference terminal devices through the network, wherein the data receiving means receives a transmission signal transmitted through the network. Buffer means for recognizing a terminal identifier contained in the transmission signal received by the data receiving means, and storing the received data in a buffer memory for each conference terminal device corresponding to the terminal identifier Decoding means for decoding received data stored in the buffer means; and, for the conference-related information, a condition for giving priority to the decoding processing by the decoding means. Priority condition information indicating which of the related information is to be given priority to the decoding process when the information is included is identified by the transmitting terminal. Along with the discriminator, in advance, the priority condition storage means to be stored, of the priority condition information stored in the priority condition storage means, with reference to the priority condition information regarding the terminal identified by the terminal identifier, Recognizing whether the conference-related information received by the data receiving means matches the priority condition information, and for the received data matching the priority condition information, the decoding means preferentially decodes the received data. And a priority control unit that controls the execution of the conference terminal. 12. The transmission signal of the conference-related information includes a video,
It is obtained by packetizing and multiplexing information such as voice, and is provided with a separating unit that separates the multiplexed conference-related information packet received by the data receiving unit for each information. The conference terminal device according to claim 8.