JP2007274369A - Communication system, transmission device and receiving device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable a receiving end to grasp the cause of occurrence of a silent section without modifying a current packet format. <P>SOLUTION: A communication system is equipped with a data converting means of converting the data value to a predetermined value close to it and outputting the predetermined value when the received sound data show silent and a transmitting means of forming data blocks which are formed by writing data outputted from the data conversion means in a payload unit by every prescribed number of data in order of output and transmitting the data blocks to a communication partner's device through a communication network at a prescribed time interval. The communication system generates and transmits data blocks which are formed by writing sound data showing silent in the payload unit for prescribed several minutes, if new sound data are not received until a time corresponding to the prescribed time interval has passed after transmitting the data blocks. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for performing data exchange of a storage and exchange method via a communication network, and more particularly to a technique for transmitting and receiving audio data and video data.

  For example, audio data by packet communication via a communication network, such as streaming distribution of music, or transmission of audio data corresponding to sound collected by a microphone on the transmission side to a reception side via a packet communication network such as the Internet. The technology for transmitting and receiving is generally popular.

  When audio data corresponding to the audio collected by the microphone is transmitted by packet communication, transmission of the packet is delayed until the audio data for one packet is accumulated. The amount of data differs depending on whether the sound data is sound data indicating sound or silence data indicating sound. For this reason, the transmission interval of a packet will differ in the sound area in which a certain sound is sounding, and a silence area. Specifically, it is common for packets to be transmitted at intervals of about 20 milliseconds for voiced intervals and packets at intervals of about 80 milliseconds for silent intervals.

  On the other hand, on the audio data receiving side, in order to avoid interruption of audio reproduction due to a delay in the communication network, packets sent from the transmitting side are accumulated in a buffer, and a sufficient number of packets are accumulated. In general, the audio data included in the packet is started to be reproduced at the time when the packet is received. However, if it is necessary to increase the buffer size on the reception side in order to cope with the delay for the silent period, it is not preferable because the manufacturing cost of the device on the reception side increases. Thus, various techniques for preventing transmission delay in the silent period have been proposed, and the technique disclosed in Patent Document 1 is an example.

In Patent Document 1, when silence data is input, a silence packet is added to the voice data immediately before the silence data to generate a transmission packet, while the first voice data subsequent to the silence data is added. By generating a transmission packet with the presence of a sound flag and not generating a transmission packet consisting only of silence data, the reception side can indicate the start and end of a silence period without transmitting a packet consisting only of silence data. A technique is disclosed that allows communication to be performed.
JP-A-9-64911

  However, in the technique disclosed in Patent Literature 1, since the start and end of the silent period are transmitted to the receiving side by the silent flag and the voiced flag, the receiving side interprets the meaning content indicated by these flags. If it is not possible (that is, if the receiving side does not support the packet format (packet data structure) to which such a flag is attached), there is a problem that sound reproduction cannot be performed correctly. .

  Further, although the technique disclosed in Patent Document 1 can transmit the start and end of a silent period, is the silent period caused by mute on the transmission side (that is, is it intended?) ) Or, the problem is that it is impossible for the receiving side to know whether it is caused by some trouble such as the microphone being disconnected on the transmitting side or by the delay of the communication network. There is also.

  The present invention has been made in view of the above problems, and has an object to provide a technique that enables the receiving side to grasp the cause of the silent period without modifying the existing packet format. Yes.

  In order to solve the above problems, the present invention receives data indicating video or audio, determines whether or not the data value is a first value indicating no video or sound, and uses the first value to When it is determined that there is a data conversion unit that rewrites and outputs a predetermined value in the vicinity of the first value, and a predetermined number of data sequentially output from the data conversion unit according to a predetermined time interval A transmission unit that generates data blocks written in the payload portion in order of output, and transmits the data blocks via the communication network at the predetermined time interval, and data transmitted from the transmission unit Receiving the block via the communication network, reading the data written in the payload portion of the data block and outputting the data, and the data output from the receiving unit And a control unit that controls playback of video or audio, and the transmission unit transmits the data block until a time corresponding to the predetermined time interval elapses. In addition, when new data is not received from the data conversion means, a data block in which the predetermined number of data having the data value is written in the payload portion is generated and transmitted, and the reception is performed. If the means does not receive a subsequent data block after the time corresponding to the predetermined time interval elapses after receiving the data block, the means sets the first value and the predetermined value to The predetermined number of data having a second value different from any of the data values as the data value is output, and the receiving device continuously receives the predetermined number of data whose data value is the first value. A first message indicating a failure of the transmitting device is notified, and the predetermined number of data whose data value is the second value is continuously output from the receiving means. If it is, a second message indicating that a failure has occurred in the communication network is notified, and data whose data value is the predetermined value is a number corresponding to a predetermined time interval. Provided is a communication system further comprising notification means for notifying a third message indicating that video or audio is muted when being continuously output from the receiving means. .

  In a more preferred aspect, the transmission device includes determination means for determining whether or not muting is performed at a data supply source indicating the video or audio, and the data conversion means has a data value of the first value. When the rewriting of the data having the value is determined by the determination means that the mute is being performed, the data is rewritten to a third value different from any of the first and second values, If it is determined by the determining means that the mute has not been performed, it is rewritten to a fourth value different from any of the first, second and third values, and the notifying means has a data value of the first value. The third message is notified only when data having a value of 3 is continuously output from the receiving means in a number corresponding to a predetermined time interval.

  In order to solve the above problem, the present invention receives data indicating video or audio, determines whether or not the data value is a first value indicating no video or sound, and determines the first value. The data conversion means for rewriting and outputting the predetermined value in the vicinity of the first value, and the data sequentially output from the data conversion means for a predetermined time interval corresponding to the predetermined time interval. A transmission means for generating data blocks written in the payload part in order of output in a number of times and transmitting the data blocks via the communication network at the predetermined time interval; and the transmission means includes the data blocks If no new data is received from the data conversion means before the time corresponding to the predetermined time interval elapses after transmission, the data whose data value is the first value is stored as the predetermined value. Providing a transmission apparatus, characterized by generating as many payload portion to the written data blocks of the transmission.

  Further, in order to solve the above-mentioned problem, the present invention is a transmitting apparatus that receives data indicating video or audio and transmits a data block in which the data is written in a payload portion at a predetermined time interval via a communication network. When the data value of the data is a first value indicating no video or sound, the data value is rewritten to a predetermined value different from the first value and written to the payload portion, while the data block is transmitted. If no new data is received before the time corresponding to the predetermined time interval elapses, the number of data whose data value is the first value is equal to the number corresponding to the predetermined time interval. In a receiving device that communicates via a communication network with a transmitting device that generates and transmits a data block written in a payload portion, the data block transmitted from the transmitting device is The data received via the communication network, the data written in the payload portion of the data block is read and output, and until the time corresponding to the predetermined time interval has elapsed since the data block was received, A receiving means for outputting the predetermined number of data having as a data value a second value different from any of the first value and the predetermined value when a subsequent data block is not received; Control means for performing reproduction control of video or audio according to data output from the receiving means, and the predetermined number of data whose data value is the first value are continuously output from the receiving means Has notified the first message indicating the failure of the transmitting device, and the predetermined number of data whose data value is the second value has been continuously output from the receiving means. A second message indicating that a failure has occurred in the communication network, and data having a data value of the predetermined value continues for a number corresponding to a predetermined time interval. And a notifying means for notifying a third message indicating that the video or audio is muted when output from the receiving means.

  In another aspect for solving the above problem, the computer device receives data indicating video or audio and determines whether or not the data value is a first value indicating no video or silence. When it is determined that the value is the first value, the data conversion means for rewriting and outputting the predetermined value in the vicinity of the first value, and the data sequentially output from the data conversion means for a predetermined time A data block written in the payload part in the order of output according to the interval is generated, and the data block is transmitted as a transmission unit via the communication network at the predetermined time interval. If no new data is received from the data conversion means after the time corresponding to the predetermined time interval has elapsed since the transmission of the data block, the data value is Serial it may provide a program characterized that the data is a first value generating and transmitting data blocks written only in the payload portion the predetermined number. If such a program is installed in a general computer device and the computer device is operated according to the program, the same function as the transmission device according to the present invention can be given to the general computer device. It becomes possible.

  In another aspect for solving the above problem, the computer device receives data indicating video or audio, and transmits a data block in which the data is written in the payload portion at a predetermined time interval via the communication network. When the data value of the data is a first value indicating no video or sound, the data is rewritten to a predetermined value different from the first value and written to the payload portion. If no new data is received after the time corresponding to the predetermined time interval has elapsed since the transmission of the data block, the data whose data value is the first value is transferred to the predetermined time. In the transmitting apparatus that generates and transmits data blocks written in the payload portion in a number corresponding to the interval, and the receiving apparatus that communicates via the communication network, the transmitting apparatus The data block transmitted from the data block is received via the communication network, and the data written in the payload portion of the data block is read out and output, and the data block is received and the predetermined time interval is received. If the subsequent data block is not received before the time elapses, the predetermined number of data having a second value different from either the first value or the predetermined value as the data value Receiving means for outputting only, control means for controlling reproduction of video or audio in accordance with data outputted from the receiving means, and the predetermined number of data whose data value is the first value are continuously transmitted When the data is output from the receiving means, a first message indicating a failure of the transmitting device is notified, and the predetermined number of data whose data value is the second value is continuously transmitted. When the data is output from the receiving means, a second message indicating that a failure has occurred in the communication network is notified, and data whose data value is the predetermined value is predetermined constant. When the number of times corresponding to the time interval is continuously output from the receiving means, it functions as an informing means for informing a third message indicating that the video or audio is muted. You may provide a program to do. If such a program is installed in a general computer device and the computer device is operated according to the program, the general computer device can be given the same function as the receiving device according to the present invention. It becomes possible.

  According to the present invention, there is an effect that the reception side can grasp the cause of the silent period without modifying the existing packet format.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(A: Configuration)
FIG. 1 is a block diagram showing a configuration example of a remote conference system 10 which is an embodiment of a communication system according to the present invention. As shown in FIG. 1, the remote conference system 10 includes communication terminals 100 </ b> A and 100 </ b> B that are wired to a communication network 200. Hereinafter, when it is not necessary to distinguish between the communication terminals 100A and 100B, they are simply referred to as “communication terminal 100”. Further, FIG. 1 illustrates the case where two communication terminals 100 are connected to the communication network 200, but three or more communication terminals 100 may be connected to the communication network 200. Of course.

  The communication network 200 is, for example, the Internet, and includes many network devices such as routers although detailed illustration is omitted. This communication network 200 is for mediating voice data communication performed according to a predetermined communication protocol between the communication terminal 100A and the communication terminal 100B. In the present embodiment, a case where the communication terminal 100 is wired to the communication network 200 will be described. However, the communication network 200 is a wireless packet communication network such as a wireless local area network (LAN). Of course, it may be a wireless communication terminal connected to this wireless packet communication network.

In this embodiment, each communication protocol described below is used as the communication protocol. That is, Real-time Transport Protocol (hereinafter referred to as “RTP”) is used as the communication protocol of the application layer, UDP is used as the communication protocol of the transport layer, and communication of the network layer is performed. IP is used as a protocol. Note that UDP and IP are communication protocols that are widely used in general, and therefore description thereof will be omitted, and RTP will be described.
RTP is a communication protocol for providing a communication service for transmitting and receiving audio data and video data having real-time characteristics end-to-end, and the details thereof are defined in RFC1889. In RTP, an RTP packet generated by adding an RTP header defined in RFC1889 to audio data and video data for a predetermined time (for example, 20 milliseconds) is delivered to a lower layer communication protocol. Audio data and video data are transmitted, and conversely, audio data and video data are received by cutting out the payload portion of the RTP packet received from the lower layer. The details are stipulated in RFC 1889, but in the RTP header, the payload type indicating the type of data written in the payload part and the output source and input destination of the data written in the payload part are shown. An identifier indicating the time, a time stamp indicating the time when the data written in the payload portion is sampled, and the like are written. In the present embodiment, the predetermined time is set to 20 milliseconds.

  The communication terminal 100 collects the voice of the user of the own device and transmits voice data corresponding to the voice to the partner device via the communication network 200 according to RTP, while voice data transmitted from the partner device according to RTP. And a function of reproducing sound according to the sound data. The communication terminal 100 receives the cause of the silent section without using a special packet format by performing data conversion characteristic to the present invention when transmitting the voice data to the counterpart device. It is possible to let the side grasp.

FIG. 2 is a block diagram illustrating an example of a hardware configuration of the communication terminal 100.
As illustrated in FIG. 2, the communication terminal 100 includes a voice input unit 110, a transmission data conversion unit 120, a packet communication unit 130, a state notification unit 140, and a voice output unit 150.

The voice input unit 110 includes, for example, a microphone 111 and an analog / digital (hereinafter abbreviated as “A / D”) converter 112.
The microphone 111 collects the user's voice and outputs a voice signal corresponding to the voice to the A / D converter 112.
The A / D converter 112 performs A / D conversion on the audio signal delivered from the microphone 111, and outputs the audio data as the conversion result to the transmission data conversion unit 120. More specifically, the A / D converter 112 samples the signal level of the audio signal delivered from the microphone 111 at a predetermined sampling period (for example, 44.1 kHz), and performs 16-bit quantization on the sampling result. As a result, the signal level (that is, the volume) of the audio signal is converted into a data string of audio data represented by 65536 levels of -32768 to 32767 and output. In the present embodiment, the case where the sampling frequency is 44.1 kHz and the 16-bit quantization is performed has been described, but it goes without saying that the sampling frequency and the number of quantization bits are not limited to the above values. Yes.

  The transmission data converter 120 determines whether or not the audio input unit 110 is muted, and the data value of the audio data delivered from the audio input unit 110 is a predetermined value indicating silence (in this embodiment, “0 ”), And if the sound data is muted or is a predetermined value indicating silence, a predetermined first value in the vicinity of the value indicating silence is determined. (In this embodiment, it is converted to “1”) and delivered to the packet communication unit 130. Note that the audio data having a data value of “1” is sound data indicating that a sound is being produced, but its volume is extremely low and does not affect hearing.

  As shown in FIG. 2, the packet communication unit 130 includes an RTP packet transmission unit 131 connected to the transmission data conversion unit 120, an RTP packet reception unit 132 connected to the status notification unit 140, and an RTP packet transmission unit. 131 and a communication interface (hereinafter referred to as “IF”) unit 133 connected to the RTP packet receiving unit 132.

The RTP packet transmission unit 131 has a buffer (not shown) that accumulates audio data delivered from the transmission data conversion unit 120, generates the RTP packet, and is delivered to the payload portion from the transmission data conversion unit 120. Audio data is written in the given order in the order of delivery and delivered to the communication IF unit 133.
In addition, the RTP packet transmission unit 131 determines a predetermined timing (for example, when the predetermined time elapses after the start of voice communication is instructed, or after delivering the RTP packet to the communication IF unit 133). If audio data for the predetermined time has not been received from the transmission data converter 120 at the time when the predetermined time elapses), the audio data whose data value is a value indicating silence (that is, “0”) is complemented. Then, the RTP packet in which the audio data is written in the payload part is transferred to the communication IF part 133. Note that complementing audio data having a data value of “0” means generating audio data having a data value of “0” by the number corresponding to the predetermined time.

  The RTP packet receiving unit 132 reads out the audio data for the predetermined time from the payload portion of the RTP packet delivered from the communication IF unit 133 and delivers it to the status notification unit 140 in order from the beginning. In addition, the RTP packet receiver 132 receives a predetermined timing (for example, when the predetermined time elapses after the start of voice communication is instructed, or after receiving the RTP packet from the communication IF unit 133). When a new RTP packet is not received from the communication IF unit 133 at the time when a predetermined time elapses), the audio data whose data value is “0” is complemented, and the audio data is delivered to the state notification unit 140 It is.

  The communication IF unit 133 is a NIC (Network Interface Card), for example, and is connected to the communication network 200 by wire. The communication IF unit 133 sequentially encapsulates the RTP packet delivered from the RTP packet transmission unit 131 according to a lower layer communication protocol (that is, generates a UDP segment in which the RTP packet is written in the payload unit, and The IP packet obtained by writing the UDP segment in the payload portion is generated) and sent to the communication network 200, while the IP packet received via the communication network 200 is processed in reverse to the above encapsulation. By doing this, the RTP packet encapsulated in the IP packet is read and delivered to the RTP packet receiving unit 132.

  Based on the data value of the voice data delivered from the packet communication unit 130, the status notification unit 140 determines the source of the voice data and the state of the communication network 200, and some trouble occurs on the transmission side or the communication network. When it is determined that the transmission is performed, or when it is determined that the transmission side is muted, the fact is notified and the audio data is delivered to the audio output unit 150. More specifically, the state notification unit 140 includes display means such as a liquid crystal display and its drive circuit, and displays a message indicating the determination result on the display means. In this embodiment, a case where the state of the transmission side or the communication network is notified by displaying a message on the display means will be described. However, a voice message indicating the state of the transmission side or the communication network is sent to the voice output unit 150. Of course, the state of the transmission side or the communication network may be notified by outputting.

The audio output unit 150 performs audio reproduction control according to audio data delivered from the state notification unit 140, and includes a D / A converter 151 and a speaker 152. The D / A converter 151 performs a process (that is, D / A conversion) opposite to that of the A / D converter 112 on the data sequence of the audio data delivered from the state notification unit 140 to generate an audio signal. This is converted and delivered to the speaker 152. The speaker 152 emits sound corresponding to the sound signal delivered from the D / A converter 151.
The above is the configuration of the communication terminal 100.

(B: Operation)
Next, operations performed by the communication terminal 100 will be described with reference to the drawings.
In the following, in a situation where a user of the communication terminal 100A (hereinafter referred to as “user A”) and a user of the communication terminal 100B (hereinafter referred to as “user B”) are conducting a remote conference, the user A speaks. The operation performed by the communication terminals 100A and 100B when the user B listens to the user A's speech will be described.
In the operation example described below, when it is necessary to distinguish the constituent elements of the communication terminal 100A and the constituent elements of the communication terminal 100B, they are expressed as follows. That is, the components of the communication terminal 100A are expressed as “voice input unit 110A”, “transmission data conversion unit 120A”, “packet communication unit 130A”, “status notification unit 140A”, and “voice output unit 150A”. The components of the communication terminal 100B are expressed as “voice input unit 110B”, “transmission data conversion unit 120B”, “packet communication unit 130B”, “status notification unit 140B”, and “voice output unit 150B”.
Further, the description is similarly applied when it is necessary to distinguish between the communication terminals 100 </ b> A and 100 </ b> B with respect to each component of the voice input unit 110, the packet communication unit 130, and the voice output unit 150. For example, the components of the audio input unit 110A are represented as “microphone 111A” and “A / D converter 112A”, and the components of the audio input unit 110B are “microphone 111B” and “A / D converter”. 112B ".

  When the user A speaks to the microphone 111A, an audio signal corresponding to the message is input from the microphone 111A to the A / D converter 112A, and a data string of audio data corresponding to the audio signal is input to the A / D converter 112A. To the transmission data conversion unit 120A. Hereinafter, an operation performed by the transmission data conversion unit 120A when these data strings are received will be described with reference to FIG.

  FIG. 3 is a flowchart showing a flow of data conversion processing performed by the transmission data conversion unit 120A. As shown in FIG. 3, the transmission data converter 120A first determines whether or not the microphone 111A is muted (step SA100).

When the determination result in step SA100 is “No” (that is, when it is determined that mute has not been performed), the transmission data conversion unit 120A performs each of the audio data sequentially delivered from the audio input unit 110A. It is determined whether the data value is “0” (that is, a value indicating silence) (step SA110).
When the determination result in step SA110 is “Yes”, the transmission data conversion unit 120A adds “1” to the audio data delivered from the audio input unit 110 (step SA120), and the audio data is added. The packet is delivered to the packet communication unit 130A (step SA130). Conversely, when the determination result in step SA110 is “No”, the process in step SA130 is executed without performing the process in step SA120.

  Returning to step SA100, if the determination result is “Yes” (that is, if it is determined that muting has been performed), the transmission data conversion unit 120A sequentially performs the processing of step SA120 and step SA130. Execute.

  Since such an operation is performed, in the communication terminal 100A according to the present embodiment, when the transmission data conversion unit 120A is operating normally, the audio data whose data value is “0” (value indicating silence). Is not delivered to the packet communication unit 130A. Hereinafter, the operation performed by the packet communication unit 130A when the voice data delivered from the transmission data conversion unit 120A is received will be described with reference to FIG.

  FIG. 4 is a flowchart showing a flow of RTP packet transmission processing performed by the RTP packet transmission unit 131A of the packet communication unit 130A. As shown in FIG. 4, RTP packet transmitter 131A first determines whether or not audio data for a predetermined time has been received from transmission data converter 120A at a predetermined correct timing (step SA140). Specifically, immediately after the RTP packet transmission unit 131A is instructed to start voice communication, whether or not the RTP packet transmission unit 131A has received voice data for a predetermined time from when the instruction is given until the predetermined time elapses, In addition, after the RTP packet has already been transmitted, it is determined whether or not audio data for the next predetermined time has been received before the predetermined time has elapsed since the RTP packet was delivered to the communication IF unit 133. If it has not been received, it is determined that audio data for a predetermined time has not been received at a predetermined correct timing.

When the determination result in step SA140 is “Yes” (that is, when it is determined that it has been received), the RTP packet transmission unit 131A receives a data sequence of audio data for a predetermined time received from the transmission data conversion unit 120A as a payload. The RTP packet written to the communication unit is generated and delivered to the communication IF unit 133 (step SA150).
On the other hand, if the determination result in step SA140 is “No”, the RTP packet transmitter 131A reports a failure of the transmission data converter (step SA160), and the data value is “0” (a value indicating silence). ) Is supplemented (step SA170), and the processing of step SA150 is executed for the speech data supplemented in step SA170.

  Thereafter, the communication IF unit 133A sequentially encapsulates the RTP packets delivered from the RTP packet transmission unit 131A to generate IP packets, and transmits the IP packets to the communication terminal 100B via the communication network 200. In this way, the IP packet transmitted from the communication terminal 100A is appropriately routed by the network device included in the communication network 200 and reaches the communication terminal 100B. When there are a plurality of IP packet destinations (that is, when there are three or more participants in the remote conference), an IP packet for each destination is generated and transmitted to each of the plurality of destinations. Alternatively, IP packets may be transmitted to the plurality of destinations by IP multicast.

  On the other hand, when the communication IF unit 133B of the communication terminal 100B receives the IP packet, the communication IF unit 133B reads the RTP packet encapsulated in the IP packet by a procedure reverse to the encapsulation, and the RTP packet is received by the RTP packet receiving unit. Delivered to 132B. Hereinafter, the RTP packet receiving process performed by the RTP packet receiving unit 132B will be described with reference to FIG.

FIG. 5 is a flowchart showing a flow of RTP packet reception processing performed by the RTP packet reception unit 132B. As shown in FIG. 5, the RTP packet receiving unit 132B first determines whether or not an RTP packet has been received at a predetermined correct timing (step SB100).
If the determination result in step SB100 is “No”, the RTP packet receiving unit 132B supplements the voice data whose data value is “−1” (step SB110) and delivers it to the state notification unit 140B (step SB100). SB130).
On the other hand, if the determination result in step SB100 is “Yes”, the RTP packet receiving unit 132B reads the audio data from the payload portion of the RTP packet received from the communication IF unit 133B (step SB120), and the audio data The process of step SB130 is executed for.

Next, the operation performed by the state notification unit 140B when voice data is delivered from the packet communication unit 130B will be described with reference to FIG.
FIG. 6 is a flowchart showing the flow of the state notification process performed by the state notification unit 140B.
As shown in FIG. 6, the state notification unit 140B first determines whether or not voice data having a data value “−1” has been delivered from the packet communication unit 130B (step SB140).
When the determination result in step SB140 is “Yes”, the state notification unit 140B causes the display unit to display “a message indicating that some kind of failure has occurred in the communication network” (step SB150). The audio data is delivered to the audio output unit 150B (step SB200). Here, the reason why the message is displayed is that the event that the data value of the voice data delivered to the state notification unit 140B is “−1” occurs because the RTP packet receiving unit 132B supplements the voice data. This is because the IP packet was not received at the correct timing due to a failure of the communication network 200.
On the other hand, if the determination result in step SB140 is “No”, state notifying unit 140B determines whether or not voice data having a data value of “0” has been delivered from packet communication unit 130B (step SB160). ).

If the determination result in step SB160 is “Yes”, state notifying unit 140B causes the display means to display a “message indicating that some failure has occurred in the transmission device” (step SB170), and the voice. Data is delivered to the audio output unit 150B (step SB200). Here, the reason why the message is displayed is that an event in which the data value of the voice data delivered to the state notification unit 140B is “0” occurs at a predetermined timing via the communication network. (That is, no failure has occurred in the communication network), but the voice data contained in the IP packet is complemented by the RTP packet transmission unit 131A on the transmission side. This is because it means that a failure has occurred in the data conversion unit 120A.
When the determination result in step SB160 is “No”, the state notification unit 140B continuously compares the audio data with the data value “1” for a certain period (for example, 30 seconds, for example). It is determined whether or not it has been received for a sufficiently long time (step SB180). If the determination result is “Yes”, a “message indicating that the transmitter side is muted” is displayed on the display means. (Step SB190), and the process of Step SB200 described above is executed. Conversely, if the determination result in step SB180 is “No”, the process in step SB200 is executed without executing the process in step SB190.

  Thereafter, in the communication terminal 100B, the audio data sequentially delivered from the state notification unit 140B to the audio output unit 150B is converted into an audio signal by the D / A converter 151, and audio corresponding to the audio signal is output from the speaker 152. Is done. Thereby, the utterance of the user A is transmitted to the user B. In the above-described operation example, the case where the user A speaks and the user B listens to the speech has been described. However, when the user B speaks and the user A listens to the speech, the communication terminal 100B uses FIGS. The processing shown in FIG. 5 and FIG. 6 is executed in the communication terminal 100A.

  As described above, according to the communication terminal 100 according to the present embodiment, whether or not the data value of the audio data to be reproduced is “0” or “−1” on the audio data receiving side. By verifying, it is possible to detect that a failure has occurred in the transmission device or the communication network, and whether or not voice data having a data value of “1” has been received for a certain period continuously. This makes it possible to detect whether or not muting is performed on the transmission device side. In addition, in this embodiment, since the state of the transmitting device is transmitted to the receiving device by converting the data value of the audio data and complementing the audio data as described above, the existing packet format (RFC1889) There is no need to make any modifications to the specified packet format.

(C: deformation)
Although the embodiment of the present invention has been described above, it is needless to say that the embodiment may be modified as described below.
(1) In the above-described embodiment, the case where RTP is used as the communication protocol of the application layer has been described, but it is needless to say that other communication protocols may be used. In short, any communication protocol may be used as long as it is a communication protocol that writes and transmits audio data for a predetermined time in a payload portion of a data block having a predetermined header portion and a payload portion. In the above-described embodiment, the case where UDP is used as the transport layer communication protocol has been described. However, TCP may be used. Similarly, it goes without saying that the communication protocol of the network layer is not limited to IP.

(2) In the above-described embodiment, the case where audio data is transmitted / received according to RTP has been described. However, when video data is transmitted / received according to RTP, an abnormality occurs on the transmission side of the video data or the communication network. It is possible for the receiving side to grasp whether or not the transmission side is muted. Specifically, in the communication terminal on the transmission side, when the data value of the video data to be transmitted is a value indicating the blackout of the video, the data value of the video data is changed to another predetermined value. Rewrite and send it.

(3) In the above-described embodiment, the case where the audio input unit 110 includes a microphone and an A / D converter has been described. However, for example, audio data such as a hard disk is stored, and the audio data is appropriately read and output. Of course, the audio input unit may be configured by a storage device that performs recording and a recording medium reading device that reads and outputs the audio data from a recording medium (for example, a Compact Disk) in which the audio data is recorded.

(4) In the above-described embodiment, a case has been described in which both the communication terminals 100A and 100B have the data conversion function and the state notification function characteristic of the present invention and perform voice data communication bidirectionally. However, when voice data communication is performed only in one direction from one to the other, the voice data transmission device includes a voice input unit 110, a transmission data conversion unit 120, an RTP packet transmission unit 131, and a communication IF. A packet communication unit consisting only of the unit 133, and the voice data receiving device includes a packet communication unit consisting of only the RTP packet receiving unit 132 and the communication IF unit 133, a status notification unit 140, An audio output unit 150 may be provided.
In the above-described embodiment, the case where the present invention is applied to the remote conference system has been described. However, the present invention may be applied to a data distribution system that performs streaming distribution of audio data and video data. Specifically, in place of the audio input unit 110, the data distribution system can be configured by combining the transmission device having a storage device in which audio data and video data are stored and the reception device. is there.

(5) In the above-described embodiment, the microphone is muted and the case where the mute is not performed and the data value of the audio data is a value indicating silence (“0”). In the above description, the data value of the audio data is converted to the same value (“1”). However, the former and the latter may be converted to different values. Specifically, the former may be converted to “1”, and the latter may be converted to “2”. In this way, it is possible to accurately distinguish the case where the microphone is muted from the case where the microphone is not muted but the other party is silent.
Further, in the above-described embodiment, the case has been described in which, when a failure occurs in the transmission data conversion unit, audio data whose data value is “0” is complemented and transmitted. However, when the cause of failure of the transmission data conversion unit can be specified, audio data having different data values for each cause of failure may be complemented and transmitted. In this way, it is possible to transmit the detailed content of the failure that has occurred in the transmission device to the reception device.

(6) In the above-described embodiment, a case has been described in which the communication terminal 100 is configured by combining hardware modules that take on unique functions and operate in cooperation according to the flowcharts illustrated in FIGS. 3 to 6. However, a program for causing a CPU (Central Processing Unit) to execute the processes shown in the flowcharts of FIGS. 3 to 6 is installed in a computer apparatus including the voice input unit 110, the voice output unit 150, and the communication IF unit 133, and the program Accordingly, it is of course possible to give the same function as that of the communication terminal 100 to the computer device by operating the CPU of the computer device.
Similarly, a program that causes the CPU to execute the processes shown in the flowcharts of FIGS. 3 and 4 is installed in a computer device that includes the voice input unit 110 and the communication IF unit 133, and the CPU of the computer device operates according to the program. Therefore, the same function as that of the transmission device according to the present invention may be given to the computer device, and the computer device including the audio output unit 150 and the communication IF unit 133 is added to the flowcharts of FIGS. A program for causing the CPU to execute the processing shown in FIG. 6 may be installed, and the CPU of the computer apparatus may be operated according to the program to give the computer apparatus the same function as the receiving apparatus according to the present invention.
When distributing the program as described above, for example, the program is written and distributed on a computer-readable recording medium such as a CD-ROM (Compact Disk-Read Only Memory) or a DVD (Digital Versatile Disk). It may also be distributed by downloading via a telecommunication line such as the Internet.

1 is a block diagram illustrating a configuration example of a remote conference system 10 according to an embodiment of the present invention. 3 is a block diagram illustrating a configuration example of the communication terminal 100. FIG. 5 is a flowchart showing a flow of data conversion processing executed by a transmission data conversion unit 120. It is a flowchart which shows the flow of the RTP packet transmission process which the RTP packet transmission part 131 performs. It is a flowchart which shows the flow of the RTP packet receiving process which the RTP packet receiving part 132 performs. It is a flowchart which shows the flow of the status alerting | reporting process which the status alerting | reporting part 140 performs.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Remote conference system, 100, 100A, 100B ... Communication terminal, 110 ... Voice input part, 120 ... Transmission data conversion part, 130 ... Packet communication part, 140 ... Status notification part, 150 ... Voice output part.

Claims (4)

When data indicating video or audio is received, it is determined whether the data value is a first value indicating no video or sound, and if it is determined that the data value is the first value, the first value Data conversion means for rewriting and outputting a predetermined value in the vicinity of the value;
A data block in which a predetermined number of data corresponding to a predetermined time interval is sequentially written to the payload portion in the output order is generated and the data block is transmitted through the communication network at the predetermined time interval. A transmission means having a transmission means,
Receiving means for receiving a data block transmitted from the transmitting device via the communication network, and reading and outputting data written in a payload portion of the data block;
Control means for performing reproduction control of video or audio according to data output from the receiving means;
With
The transmission means includes
If no new data is received from the data conversion means before the time corresponding to the predetermined time interval has elapsed since the transmission of the data block, the data whose data value is the first value is A predetermined number of data blocks written in the payload portion are generated and transmitted,
The receiving means includes
If the subsequent data block is not received before the time corresponding to the predetermined time interval elapses after receiving the data block, the first value is different from the predetermined value. Outputting the predetermined number of data having a second value as a data value;
The receiving device is:
When the predetermined number of data whose data value is the first value is continuously output from the receiving means, a first message indicating a failure of the transmitting device is notified, and the data value is When the predetermined number of data as the second value is continuously output from the receiving means, a second message indicating that a failure has occurred in the communication network is notified, and the data value When the data having the predetermined value is continuously output from the receiving means by a number corresponding to a predetermined fixed time interval, the video or audio is muted. A communication system further comprising notification means for reporting the message. The transmitter is
Determining means for determining whether or not muting is performed at a data supply source indicating the video or audio;
The data conversion means includes
When rewriting data whose data value is the first value, if the determination means determines that muting has been performed, a third value that is different from any of the first and second values. On the other hand, if it is determined by the determination means that the mute is not performed, the value is rewritten to a fourth value different from any of the first, second and third values,
The notification means includes
The third message is notified only when data whose data value is the third value is continuously output from the receiving means by a number equal to or more than a predetermined time interval. The communication system according to claim 1. When data indicating video or audio is received, it is determined whether the data value is a first value indicating no video or sound, and if it is determined that the data value is the first value, the first value Data conversion means for rewriting and outputting a predetermined value in the vicinity of the value;
A data block in which a predetermined number of data corresponding to a predetermined time interval is sequentially written to the payload portion in the output order is generated and the data block is transmitted through the communication network at the predetermined time interval. And transmitting means for transmitting
The transmission means includes
If no new data is received from the data conversion means before the time corresponding to the predetermined time interval has elapsed since the transmission of the data block, the data whose data value is the first value is A transmission apparatus that generates and transmits data blocks written in the payload portion by the predetermined number. A transmission device that receives data indicating video or audio and transmits a data block in which the data is written in a payload portion via a communication network at a predetermined time interval, wherein the data value of the data indicates no video or sound If it is the first value, it is rewritten to a predetermined value different from the first value and written to the payload portion, while the time corresponding to the predetermined time interval has elapsed since the data block was transmitted If new data is not received by the time, a data block in which the number of data whose data value is the first value is written in the payload portion corresponding to the predetermined time interval is generated and transmitted. In a receiving apparatus that communicates with a transmitting apparatus via the communication network,
The data block transmitted from the transmitting device is received via the communication network, the data written in the payload portion of the data block is read out and output, and the predetermined time interval after receiving the data block If a subsequent data block is not received before the time corresponding to elapses, data having a second value different from both the first value and the predetermined value as the data value Receiving means for outputting a predetermined number;
Control means for performing video or audio reproduction control according to data output from the receiving means;
When the predetermined number of data whose data value is the first value is continuously output from the receiving means, a first message indicating a failure of the transmitting device is notified, and the data value is When the predetermined number of data as the second value is continuously output from the receiving means, a second message indicating that a failure has occurred in the communication network is notified, and the data value When the data having the predetermined value is continuously output from the receiving means by a number corresponding to a predetermined fixed time interval, the video or audio is muted. And a notifying means for notifying the message.

Images