JP2005159660A - Video audio communication system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video audio communication system ensuring real time performance by reducing a time lag in reproducing video frames on a receiver side apparatus, even if a time required for coding processing of video information exceeds one frame period on transmitter side apparatus. <P>SOLUTION: If a time required for coding processing of video information exceeds one frame period, a control unit B1(A1) of the transmitter side apparatus B(A) continues the processing without interruption until the coding processing of video information is completed, and discards a portion of video information exceeding one frame period out of the video information captured during the time period required until the coding processing is completed. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a video / audio communication system that encodes video information and audio information to form a specific frame and performs packet communication.

In a conventional video / audio communication system, the transmitting device encodes video information and audio information, and sequentially adds frame numbers corresponding to both pieces of information to form a specific frame. While both pieces of framed information are packet-synchronized with the frame period, the receiving device refers to the packet received with reference to the frame number separately into a video frame and an audio frame in synchronism with the frame period. There has been proposed a technique in which video information and audio information are reproduced after separation (see, for example, Patent Document 1).
JP 2003-189301 A

  Conventionally, in such a video / audio communication system, when video information is encoded in a transmission-side device, the video information may be processed with a delay of one frame period or more. In such a case, the packet has not arrived at the timing for reproducing the predetermined video information on the receiving side, and the video information may not be reproduced. Also, if a delay occurs in the encoding process of such video information, when video information is played back on the receiving device, the video information and audio will be inconsistent due to inconsistency with the playback timing of the audio information. There was a problem that the contents could not be understood.

  The present invention has been made to solve the above-described problems. Even when the time required for the encoding process of the video information in the transmitting device exceeds one frame period, the video frame is reproduced in the receiving device. It is an object of the present invention to provide a video / audio communication system that can reduce the time lag and ensure real-time performance and can be consistent with audio information.

  In order to achieve the above object, according to the present invention, the transmitting device encodes video information and audio information, and sequentially adds frame numbers corresponding to both pieces of information to identify them. The frame information is formed in a frame and the packetized information is transmitted in synchronism with the frame period. On the other hand, the receiving side device refers to the frame number and receives the packet in synchronism with the frame period. A video / audio communication system that reproduces video information and audio information after separately separating frames and audio frames employs the following configuration.

  That is, in the video / audio communication system according to the first aspect of the present invention, when the time required for the encoding process of the video information exceeds one frame period, the transmission-side device performs the encoding process of the video information. The processing is continued as it is until completion, and the video information captured during the period required until the encoding processing is completed is discarded. It is characterized by providing a means.

  In the video / audio communication system according to the second aspect of the present invention, in the configuration of the video / audio communication system according to the first aspect, in the transmission side device, the encoding process of the video information is performed by one frame period by the transmission side time lag correcting means. In the case where it is continued as described above, it is provided with frame delay period information adding means for adding the duration as frame delay period information to the header of the video frame, while the receiving device is added to the header of the video frame. A receiving-side time lag correcting unit that repeats the reproduction process of the video information for the frame delay period based on the frame delay period information is provided.

  In the video / audio communication system according to the third aspect of the present invention, in the configuration of the video / audio communication system according to the first or second aspect, the transmission-side device is configured to encode the video information by the transmission-side time lag correcting means. After completing the above, the frame number when the captured video information is subsequently framed is the serial number of the video information that has been previously encoded, and when the audio information is framed, the transmission side The frame number assigned when the audio frames included in the period required until the completion of the video information encoding process by the time lag correcting means are combined into one frame and the captured audio information is subsequently framed is the above-mentioned frame number. It is characterized by comprising serial number adding means for serially assigning frame numbers attached to the voice frames grouped together.

  According to the first aspect of the present invention, when the time required for the encoding process of the video information exceeds one frame period, the process is continued as it is until the encoding process of the video information is completed. Of the video information captured during the period required to complete the encoding process, video information exceeding one frame period is discarded, so that it is reproduced after the video information is transmitted. The time lag of the processing time up to can be reduced. For this reason, it becomes possible to transmit the video information with an emphasis on the real-time property of the video information.

  According to the second aspect of the present invention, when it takes a long time to encode the video information in the transmission side device and delays by one frame period or more, the reception side device takes only the frame delay period. Since the video information reproduction process is repeated, the time lag when the video information is reproduced is reduced. For this reason, it becomes possible to transmit the video information with an emphasis on the real-time property of the video information.

  According to the third aspect of the present invention, since the frame numbers of the received video frame and audio frame can be matched in the receiving device, the consistency between the video information and the audio information can be achieved.

  Here, a case where the video / audio communication system of the present invention is applied to an apartment home automation system (hereinafter referred to as a condominium HA system) will be described as an example.

  FIG. 1 is a block diagram showing the configuration of the condominium HA system.

  The condominium HA system of this embodiment includes a dwelling unit terminal A (only one of them is shown here) installed in each dwelling unit and a lobby intercom B with a camera installed in the lobby of the apartment. A and B are connected to each other via a communication line L.

  The dwelling unit A includes a control unit A1, an interface A2, a video processing unit A3, a display unit A4, a video input unit A5, an audio processing unit A6, an audio input / output unit A7, and an operation unit A8. A packet obtained by framing audio information and video information input from the unit A7 and the video input unit A5 to the lobby intercom B, and receiving a packet of audio information and video information transmitted from the lobby intercom B; It has a function of reproducing these packets.

  The lobby intercom B includes a control unit B1, an interface B2, a video processing unit B3, a display unit B4, a video input unit B5, an audio processing unit B6, an audio input / output unit B7, and an operation unit B8. A packet obtained by framing the audio information and the video information input from the unit B7 and the video input unit B5 to the dwelling unit terminal A, and receiving the voice information and the video information packet transmitted from the dwelling unit terminal A; It has a function of reproducing these packets.

  Each of the control units A1 and B1 corresponds to a transmission side time lag correction unit, a frame delay period information addition unit, a reception side time lag correction unit, and a serial number addition unit in claims.

  Next, in the configuration described above, for example, when the lobby interphone B is used as a transmitting device and the dwelling unit terminal A is used as a receiving device, video information and audio information are transmitted from the lobby intercom B to the dwelling unit A. explain. Contrary to the above case, the operation when transmitting video information and audio information from the dwelling unit A to the lobby intercom B is basically the same.

  In the lobby intercom B, video information is captured by the video input unit B5, the video information is encoded by the video processing unit B3, and then framed into a predetermined format by the control unit B1. Also, the audio information input from the audio input / output unit B7 is encoded by the audio processing unit B6, and then framed into a predetermined format by the control unit B1. At that time, the control unit B1 assigns the same frame number to the video information and the audio information generated at the same time. The video information and audio information thus framed in a predetermined format are both packetized by the control unit B1 and transmitted from the interface to the dwelling unit A via the communication line L.

  On the other hand, when the dwelling unit A receives both the video information and audio information packets sent from the lobby intercom B via the communication line L, the control unit A1 unpacks the information. The unpacketized video information is decoded by the video processing unit A3 and output to the display unit A4 for display. The unpacketized audio information is decoded by the audio processing unit A6 and output and reproduced by the audio input / output unit A7.

  Here, in the lobby intercom B, the video information is reliably encoded every frame period and formed into a predetermined frame format. Further, the communication line L has a packet loss due to external noise, signal attenuation, etc. When packet communication is normally performed without any transmission error or the like, the control unit A1 of the dwelling unit terminal A, as shown in FIG. 2, each frame attached to the video frame and audio frame of each packet. Since the video information and audio information are unpacketized in synchronization with the frame period with reference to the number, the video information and audio information having the same frame number are reproduced at the same timing, and the consistency of both information is maintained. ing.

  On the other hand, as shown in FIG. 3, in the lobby intercom B, for example, when the video information of the frame number “16” captured by the video input unit B5 is encoded, the time required for this encoding process is completed in one frame cycle. If it takes up to 2 frame periods, the encoding process of the video information of the frame number “16” is continued as it is until the encoding is completed without ending in one frame period.

  Meanwhile, the video information of the frame number “17” is captured in the video input unit B5, but the video information of the frame number “17” is discarded without being encoded. Also, since the encoding process required a period of two frame periods, frame delay period information indicating that there is a delay of two frame periods in the encoding process is added to the header of the video frame with frame number “16”. To do.

  Further, as described above, the video information of the frame number “17” is discarded, but the video information is continuously captured in the video input unit B5. In this case, the frame number when the captured video information is formed in the frame is originally “18”, but the control unit B1 changes this to the frame number “17”, and the encoding process is completed first. The serial number of the video frame with frame number “16” is used. In the following, frame numbers are assigned so as to be serial numbers for video information to be subsequently captured.

  Further, when the audio information captured by the audio input / output unit B7 is encoded and formed into a frame, the encoding of the video information of the frame number “16” takes a period of two frames, The audio frames with the frame numbers “16” and “17” included in the period required until the encoding process of the video information with the number “16” is completed are transmitted together as a frame number “16”. Since audio frames have a much smaller amount of data than video frames, they can be transmitted together for about 4 frames.

  Subsequently, the frame number given when the audio information captured in the audio input / output unit B7 is framed is originally “18”, but the control unit B1 changes this to the frame number “17” It is assumed that the frame number “16” is assigned to the audio frames grouped together. In the following, frame numbers are assigned so as to be serial numbers for the audio information that is subsequently captured.

  When receiving the packet, the control unit A1 of the dwelling unit A separates the received packet into a video frame and an audio frame in synchronization with the frame period. Then, for example, based on the frame delay period information added to the header of the received video frame with the frame number “16”, it is understood that there is a delay of two frame periods, and the video frame with the frame number “16” Is decoded, the output is held for the frame delay period. As a result, the video corresponding to the frame number “16” is reproduced over a period of two frame periods. Thereby, the time lag when video information is reproduced is reduced.

  As for the audio frame, since the audio information for two frame periods is transmitted together, the audio corresponding to the frame number “16” over the period of two frame periods is received and decoded. Is played. For this reason, in the dwelling unit terminal A, since the reproduction timings of the video frame and the audio frame can be matched, it is possible to achieve matching when reproducing the video information and the audio information.

  In the above embodiment, the case where the video / audio communication system is applied to the condominium HA system has been described. However, the present invention is not limited to this, and both video information and audio information are transmitted via a communication network. The present invention can be widely applied to video and audio on-demand communication systems such as videophone systems and video conference systems that need to be transmitted in real time.

It is a block diagram which shows the structure of the apartment HA system in embodiment of this invention. It is a timing chart which shows the communication procedure of the video information and audio | voice information when packet communication is performed normally. It is a timing chart which shows the processing content of video information and audio | voice information when the encoding process of video information delays 1 frame period or more in the apparatus of a transmission side.

Explanation of symbols

A dwelling unit terminal B lobby intercom with camera L communication line A1, B1 control unit (transmission side time lag correction means, frame delay period information addition means, reception side time lag correction means, serial number addition means)

Claims (3)

The device on the transmitting side encodes video information and audio information, respectively, and sequentially adds a frame number corresponding to both information to form a specific frame, and both the framed information is converted into a frame period. A video / audio communication system for reproducing video information and audio information after a device on the receiving side separates the received packet into a video frame and an audio frame separately in synchronization with the frame period In
When the time required for the encoding process of the video information exceeds one frame period, the transmission-side device continues the process as it is until the encoding process of the video information is completed, and the encoding process A video / audio communication system, comprising: a transmission side time lag correcting means for discarding video information that exceeds one frame period of video information captured during a period required for the completion of. The video and audio communication system according to claim 1,
When the transmission side time lag correction means continues the video information encoding process for one frame period or more, the transmission side device adds a frame delay as frame delay period information to the header of the video frame. While receiving the period information addition means, the receiving apparatus repeats the reproduction process of the video information for the frame delay period based on the frame delay period information added to the header of the video frame. A video / audio communication system comprising: means. The video / audio communication system according to claim 1 or 2,
After the completion of the encoding process of the video information by the transmission side time lag correcting means, the transmission side device uses the frame number when the captured video information is subsequently framed as the video for which the encoding process has been completed previously. When the information is serialized and the audio information is framed, the audio frames included in the period required until the completion of the encoding process of the video information by the transmission side time lag correcting means are combined into one, and subsequently A video / audio comprising frame number adding means for setting a frame number assigned to the audio information to be framed as a serial number of frame numbers attached to the audio frames combined into one. Communications system.

Images