JP2005159652A - Video-sound communication system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video-sound communication system which optimally deals with the breakdown and the recovery of a communication line at a receive side apparatus. <P>SOLUTION: In the video-sound communication system, a transmit side apparatus codes video information and sound information, gives the frame numbers mutually corresponding to both information one after another to form specific frames, and transmits both framed information with packets synchronously with the frame cycle, while a receiving side apparatus separates the received packets into video frames and sound frames synchronously with the frame cycle, thereby reproducing the video information and the sound information. The receiving side apparatus comprises a means for deciding that the communication line is broken down, if no video frame is received for a time equal to a specified number of frame cycles after sending a request to transmit the video frame having a frame number as a reference. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a video / audio communication system for transmitting a packet in which video information and audio information are framed via a communication line.

  Conventionally, when packet communication is performed with audio information and video information framed, if a communication failure occurs in the middle of communication, packet loss occurs or audio information and video information reach the receiving side. The timing to perform may shift. In such a case, since the video and the audio do not match, there is a problem that the content becomes difficult to understand.

Therefore, in the prior art, when a packet loss or the like occurs, a packet retransmission request is sent from the receiving device to the transmitting device, and the packet is retransmitted from the transmitting device accordingly. (For example, refer to Patent Document 1).
JP 2003-189301 A

  However, if a predetermined packet does not arrive at the receiving device, it is difficult to determine whether it is caused by a missing packet or a broken communication line in the first place. A packet retransmission request is issued repeatedly to the device. In this way, the receiving device performs a useless operation such as repeating a packet retransmission request when the communication line is disconnected.

  At the time when the line is restored, if the receiving device issues a retransmission request for packets having a reference frame number to the transmitting device all at once, the communication line enters a traffic state. Furthermore, after the line is restored, both video frames and audio frames will be received. In this case, if audio frames are received and played unconditionally, there will be a shift in playback timing with the video frames. This makes it difficult to understand the content of the information.

  The present invention has been made to solve the above-described problems, and provides a video / audio communication system capable of taking an optimum response in a receiving-side device with respect to disconnection of a communication line and restoration of the line. The purpose is to do.

  In order to achieve the above object, in the present invention, the transmission side device encodes video information and audio information, respectively, and sequentially adds frame numbers corresponding to both information to each other. A specific frame is formed and both pieces of the framed information are transmitted in synchronization with the frame period. On the other hand, the receiving device converts the received packet into a video frame and an audio frame in synchronization with the frame period. A video / audio communication system that reproduces video information and audio information separately separately employs the following configuration.

  That is, in the video / audio communication system according to the first aspect of the present invention, the receiving device issues a video frame for a predetermined number of frame periods after issuing a transmission request for a video frame having a reference frame number. If the communication line is not received, it is characterized by comprising a line break determining means for determining that the communication line is disconnected.

  In the video / audio communication system according to the second aspect of the invention, the receiving-side device does not request transmission of a video frame having a reference frame number immediately after the line is restored, and the reference-side frame is transmitted from the transmitting-side device. Waiting until a video frame having a number is received, playback is started when the video frame is received, and a video playback timing adjusting means for discarding the video frame that arrives in the meantime is provided.

  In the video / audio communication system according to the third aspect of the present invention, the receiving-side device receives the same video frame having a reference frame number after line restoration and matches the timing at which playback of the video frame is started. It is characterized by comprising audio reproduction timing adjusting means for reproducing an audio frame having a frame number and discarding an audio frame that arrives in the meantime.

  In the video / audio communication system according to the fourth aspect of the present invention, when the receiving device communicates only the audio frame, the reproduction timing of the audio frame after the line restoration is transmitted from the transmitting device regardless of the frame period. It is characterized by comprising a voice reproduction timing adjusting means for matching the received voice frame with the received timing.

  In the video / audio communication system according to the fifth aspect of the present invention, the receiving-side equipment is the line disconnection determining means according to the first aspect, the video reproduction timing adjusting means according to the second aspect, and the audio according to the third and fourth aspects. All of the reproduction timing adjustment means are provided.

  According to the first aspect of the present invention, when the receiving device does not receive a video frame for a predetermined number of frame periods after issuing a transmission request for a video frame having a reference frame number. Since it is determined that the communication line is disconnected, it is possible to distinguish whether the packet is simply missing or the communication line is disconnected. For this reason, there is no useless operation such as issuing a packet retransmission request to the transmitting device over and over even though the communication line is disconnected at the receiving device. The burden of power consumption is reduced and wasteful power consumption can be reduced.

  According to the second aspect of the present invention, the receiving side device does not request the transmission of the video frame having the reference frame number immediately after the line restoration, but the transmitting side device sets the reference frame number. Waiting until the received video frame is received, and reproduction is started when the video frame is received, so that it is possible to effectively avoid the generation of traffic on the communication line after the line is restored.

  According to the third aspect of the present invention, the receiving device has the same frame number in accordance with the timing of receiving the video frame having the reference frame number after the line restoration and starting the reproduction of the video frame. Since audio frames are reproduced and audio frames that arrive between them are discarded without being reproduced, synchronized reproduction can be performed smoothly by adjusting the difference in reproduction timing between the video frames and the audio frames.

  According to the fourth aspect of the present invention, when only voice frames are communicated, the reproduction timing of the voice frame after the line restoration is sent from the transmission side device regardless of the frame period in the reception side device. Since the audio frame is received at the same timing, the call can be recovered promptly.

  According to the invention of claim 5, since the receiving device has all the effects of claims 1 to 4, it is possible to take an optimum response to line disconnection and line restoration. It becomes possible.

  Here, a case where the video / audio communication system of the present invention is applied to an apartment home automation system (hereinafter referred to as a condominium HA system) will be described as an example.

  FIG. 1 is a block diagram showing the configuration of the condominium HA system.

  The condominium HA system of this embodiment includes a dwelling unit terminal A (only one of them is shown here) installed in each dwelling unit and a lobby intercom B with a camera installed in the lobby of the apartment. A and B are connected to each other via a communication line L.

  The dwelling unit A includes a control unit A1, an interface A2, a video processing unit A3, a display unit A4, a video input unit A5, an audio processing unit A6, an audio input / output unit A7, and an operation unit A8. A packet obtained by framing audio information and video information input from the unit A7 and the video input unit A5 to the lobby intercom B, and receiving a packet of audio information and video information transmitted from the lobby intercom B; It has a function of reproducing these packets.

  The lobby intercom B includes a control unit B1, an interface B2, a video processing unit B3, a display unit B4, a video input unit B5, an audio processing unit B6, an audio input / output unit B7, and an operation unit B8. A packet obtained by framing the audio information and the video information input from the unit B7 and the video input unit B5 to the dwelling unit terminal A, and receiving the voice information and the video information packet transmitted from the dwelling unit terminal A; It has a function of reproducing these packets.

  Each of the control units A1 and B1 corresponds to a line disconnection determination unit, a video reproduction timing adjustment unit, and an audio reproduction timing adjustment unit in claims.

  Next, in the configuration described above, for example, when the lobby interphone B is used as a transmitting device and the dwelling unit terminal A is used as a receiving device, video information and audio information are transmitted from the lobby intercom B to the dwelling unit A. explain. Contrary to the above case, the operation when transmitting video information and audio information from the dwelling unit A to the lobby intercom B is basically the same.

  In the lobby intercom B, video information is captured by the video input unit B5, the video information is encoded by the video processing unit B3, and then framed into a predetermined format by the control unit B1. Also, the audio information input from the audio input / output unit B7 is encoded by the audio processing unit B6, and then framed into a predetermined format by the control unit B1. At that time, the control unit B1 assigns the same frame number to the video information and the audio information generated at the same time. The video information and the audio information thus framed in a predetermined format are both packetized and transmitted from the interface to the dwelling unit A via the communication line L.

  On the other hand, when the dwelling unit A receives both the video information and audio information packets sent from the lobby intercom B via the communication line L, the control unit A1 unpacks the information. The unpacketized video information is decoded by the video processing unit A3 and output to the display unit A4 for display. Also, the unpacketized audio information is decoded by the audio processing unit A6 and output and reproduced to the audio input / output unit A7.

  Now, when no packet loss or transmission error due to external noise or signal attenuation occurs, and when communication is normally performed without disconnection of the communication line L, the control unit A1 of the dwelling terminal A As shown in FIG. 2, the video information and audio information are unpacketized in synchronization with the frame period with reference to the frame numbers attached to the video frame and audio frame of each packet, so that they have the same frame number. Video information and audio information are reproduced at the same timing.

  Next, when the communication line L is disconnected, the processing operation in the control unit A1 of the dwelling unit A serving as the receiving device will be described with reference to FIG. 3 and FIG.

  If the communication line L is disconnected for some reason during communication of video and audio packets, the packet is not received by the dwelling unit A that is the receiving device. When the unpacketized video frame cannot be obtained, the control unit A1 issues a re-transmission request for the video frame having the reference frame number to the lobby intercom B serving as the transmission side device. The communication algorithm has a reference frame number transmitted from the transmission side device within a period of 5 to 6 frames after a transmission request for a video frame having the reference frame number is issued from the reception side device. In this embodiment, the control unit A1 starts the transmission of the video frame having the reference frame number to the lobby interphone B, and then the frame period is received. When no video frame is received for a predetermined number of times (for example, 10 frames), it is determined that the communication line L is disconnected.

  In this way, the control unit A1 of the dwelling unit terminal A does not receive a video frame for a predetermined number of frame periods from the time when a transmission request for a video frame having a reference frame number is issued. Since the disconnection is determined, it is possible to clearly distinguish whether the packet is simply missing or the line is disconnected. For this reason, even when the line is disconnected, the control unit A1 eliminates useless operations such as repeatedly sending a packet retransmission request to the lobby interphone B, thereby reducing the burden on communication. And wasteful power consumption can be reduced.

Next, the operation of the dwelling unit A that is the receiving side device when the communication line L is restored will be described with reference to FIG. 3 and FIGS.
(A) When the communication line L is restored, a packet of video information reaches the dwelling unit A. However, when the video frame obtained by unpacketizing the received packet is only a differential frame, video cannot be reproduced. . Therefore, the control unit A1 requests transmission of a video frame having a reference frame number (in this example, “20”) after the line is restored. In this case, if the control unit A1 requests the lobby interphone B to transmit a video frame having a reference frame number immediately after the line L is restored, traffic on the line L may occur and communication may become impossible.

  Therefore, the control unit A1 does not immediately request the transmission of the video frame having the reference frame number even if the line L is restored, but the reference frame from the lobby intercom B as shown in FIG. Wait until a video frame having a number arrives, and discard the difference frame that arrives in the meantime. Then, the control unit A1 starts reproduction when receiving a video frame having a reference frame number.

Thus, instead of requesting transmission of a video frame having a reference frame number immediately after line restoration, it waits until a video frame having a reference frame number is received from the transmitting device, and the video frame Since the reproduction is started when the message is received, it is possible to effectively avoid the occurrence of traffic on the communication line after the line is restored.
(B) When the communication line L is restored, a packet of audio information reaches the dwelling unit A. However, when an audio frame obtained by unpacketizing the received packet is reproduced as it is, it can be matched with the video frame. Disappear. Therefore, the control unit A1 receives a video frame having a reference frame number (in this example, “20”) after line restoration as shown in (a) above, and until the timing for starting playback of the video frame is reached. Discards audio frames and plays them silently. Then, the audio information is reproduced from the audio frame having the same frame number (“20” in this example) in accordance with the timing of reproducing the video information from the video frame having the reference frame number (“20” in this example). To do.

In this way, after receiving the video frame having the reference frame number after the line restoration, the audio frame having the same frame number is reproduced in accordance with the timing of starting the reproduction of the video frame, Since it is discarded without being reproduced, synchronized reproduction can be performed smoothly by adjusting the difference in reproduction timing between the video frame and the audio frame.
(C) When the communication line L is restored, the voice information packet reaches the dwelling unit A. At that time, when only the voice information packet is communicated after the restoration, the communication algorithm reproduces the voice information packet. The timing to be set is set every frame period (33 ms). However, when the playback timing is adjusted to one frame period in this way, the state in which a call can be made after the line is restored is delayed.

  Therefore, as shown in FIG. 7, the control unit A1 determines that the audio frame sent from the lobby intercom B is independent of the frame period, as long as the audio frame can be reproduced at a certain timing after the line is restored. Audio information is played back at the timing of receiving. Thereafter, the timing is maintained.

  As described above, when only the audio frame is communicated, in the receiving device A, the reproduction timing of the audio frame is matched with the reception timing of the audio frame transmitted from the transmitting device B regardless of the frame period. , Call recovery can be performed promptly.

  In the above embodiment, the case where the video / audio communication system is applied to the condominium HA system has been described. However, the present invention is not limited to this, and both video information and audio information are transmitted via a communication network. The present invention can be widely applied to on-demand communication systems such as a videophone system and a video conference system that need to be transmitted in real time.

It is a block diagram which shows the structure of the apartment HA system in embodiment of this invention. It is a timing chart which shows the communication procedure of the video information and audio | voice information when packet communication is performed normally. It is a timing chart which shows the communication procedure of the video information and audio | voice information when a communication line is disconnected and a line is restored after that. It is explanatory drawing of the process sequence in the receiving side apparatus when a communication line is disconnected. It is explanatory drawing of the reproduction | regeneration processing of the video information in the receiving side apparatus when a communication line is restored. It is explanatory drawing of the process which makes the reproduction | regeneration timing of audio | voice information correspond with the reproduction | regeneration timing of video information in the receiving side apparatus when a communication line is restored. It is explanatory drawing of the reproduction | regeneration processing in the receiving side apparatus in the case of communicating only audio | voice information when a communication line is restored.

Explanation of symbols

A dwelling unit terminal B lobby intercom with camera L communication line A1, B1 control unit (line disconnection judging means, video reproduction timing adjustment means, audio reproduction timing adjustment means)

Claims (5)

The transmitting device encodes video information and audio information, respectively, and sequentially adds frame numbers corresponding to both pieces of information to form a specific frame. While transmitting packets in synchronization with the period, the receiving device separates the received packets into video frames and audio frames in synchronization with the frame period to reproduce video information and audio information. In the system,
The receiving device determines that the communication line is disconnected if it does not receive a video frame for a predetermined number of times of the frame period after issuing a transmission request for a video frame having a reference frame number. A video / audio communication system comprising line disconnection determination means. The transmitting device encodes video information and audio information, respectively, and sequentially adds frame numbers corresponding to both pieces of information to form a specific frame. While transmitting packets in synchronization with the period, the receiving device separates the received packets into video frames and audio frames in synchronization with the frame period to reproduce video information and audio information. In the system,
The receiving-side device does not request transmission of a video frame having a reference frame number immediately after line restoration, and waits until receiving a video frame having a reference frame number from the transmitting-side device. A video / audio communication system comprising video playback timing adjusting means for starting playback when receiving a frame and discarding a video frame that arrives in the meantime. The transmitting device encodes video information and audio information, respectively, and sequentially adds frame numbers corresponding to both pieces of information to form a specific frame. While transmitting packets in synchronization with the period, the receiving device separates the received packets into video frames and audio frames in synchronization with the frame period to reproduce video information and audio information. In the system,
The receiving-side device receives a video frame having a reference frame number after line restoration, plays back an audio frame having the same frame number at the timing of starting playback of the video frame, and arrives in the meantime An audio / video communication system comprising audio reproduction timing adjustment means for discarding audio frames. The transmitting device encodes video information and audio information, respectively, and sequentially adds frame numbers corresponding to both pieces of information to form a specific frame. While transmitting packets in synchronization with the period, the receiving device separates the received packets into video frames and audio frames in synchronization with the frame period to reproduce video information and audio information. In the system,
When the receiving-side device communicates only audio frames, the audio that matches the audio frame playback timing after line restoration with the timing at which the audio frame sent from the transmitting-side device is received regardless of the frame period. A video / audio communication system comprising reproduction timing adjusting means. The transmitting device encodes video information and audio information, respectively, and sequentially adds frame numbers corresponding to both pieces of information to form a specific frame. While transmitting packets in synchronization with the period, the receiving device separates the received packets into video frames and audio frames in synchronization with the frame period to reproduce video information and audio information. In the system,
The receiving-side device includes all of the line disconnection judging means according to claim 1, the video reproduction timing adjusting means according to claim 2, and the audio reproduction timing adjusting means according to claims 3 and 4. Video / audio communication system.

Images