JP4252017B2 - Coded stream relay apparatus, method and program thereof - Google Patents

Description

  The present invention relates to an apparatus for relaying stream data such as encoded audio and video (hereinafter referred to as an encoded stream), a method thereof, and a program.

When transmitting and receiving an encoded stream between terminal apparatuses connected to networks with different protocols, an encoded stream relay apparatus for absorbing the protocol difference is required at a relay point between the networks. In a conventional encoded stream relay device, when relaying an encoded stream, the following two methods, namely, method 1: a received encoded stream is once decoded, re-encoded and transmitted (see Patent Document 1).
Method 2: Relay the received encoded stream without decoding (see Patent Document 2)
One of them was used.

  In the case of the above-described method 1, since the encoded stream is once decoded and re-encoded, there are problems that delay increases, sound quality and image quality deteriorate, and processing load is heavy.

  On the other hand, in the case of scheme 2, the above-described problem is unlikely to occur. However, when encoded information that is important information regarding the encoded stream is lost due to a transmission error or packet loss, the encoded stream is The problem that decoding becomes impossible occurs.

  For example, in order to decode an MPEG (Moving Picture Experts Group) -4 video stream, DCI (Decoder Configuration Information) that is encoding information related to the MPEG-4 stream is required.

  The DCI includes information such as Profile @ Level, image size, encoding tool used, etc. necessary for decoding the MPEG-4 stream, and DCI (or information corresponding to DCI) must be absent. MPEG-4 streams cannot be decoded. When decoding an MPEG-4 stream, the decoder is usually initialized using DCI, and then decoding of the MPEG-4 stream is started.

Therefore, when an MPEG-4 stream is transmitted by the method 2 described above, one of the following methods is used: (1) DCI is periodically inserted into the MPEG-4 stream.
(2) Transmit DCI separately from the MPEG-4 stream.
Therefore, it is necessary to transmit DCI to the communication partner.

  FIG. 1 is a diagram for explaining the method (1). DCI is set between a coded stream sent from a transmitting terminal to a receiving terminal, actually, a large number of packets divided and stored. Packets stored are periodically inserted and transmitted.

FIG. 2 is a diagram for explaining the method (2). Here, an example of an IP videophone using SIP (Session Inititation Protocol) is shown. In this example, DCI is transmitted to the communication partner at the time of call connection before starting the transmission of the MPEG-4 stream, that is, from the terminal device on the calling side, for example, A adds DCI to the SIP calling message INVITE to the called side. The terminal device B on the receiving side adds DCI to the SIP incoming response message 200OK and transmits it to the terminal device A on the calling side.
JP 2003-046977 A (name: relay server) JP 2003-152820 A (Name: Signaling Relay System and Signaling Relay Method)

  In general, in a system such as a video distribution system or an IP telephone, encoded information is often exchanged before actually starting transmission of an encoded stream as in the method (2) described above. For example, in a distribution system using RTSP (Real-time Streaming Protocol) and the IP telephone system using SIP described above, encoding information (for example, encoding parameters) is previously stored using SDP (Session Description Protocol). Exchange between end-to-end terminals.

  However, just because the encoded information is exchanged in advance, the encoded stream received by the terminal device on the receiving side cannot always be normally decoded.

  This is because some terminal apparatuses do not initialize a decoder using encoded information exchanged in advance, but initialize a decoder using encoded information included in a received encoded stream. there is a possibility. In such a terminal device, when encoded information is not included in the received encoded stream, the received encoded stream cannot be decoded.

  On the other hand, there is a possibility that some terminal devices do not insert information about the encoded stream into the encoded stream or insert it only once because the encoded information is exchanged in advance. .

  Therefore, when the decoder is initialized using the encoded information included in the encoded stream received by the terminal device on the receiving side, the transmitting side does not insert the information on the encoded stream into the encoded stream described above. In the case of a terminal device, the reception side cannot decode the encoded stream transmitted from the transmission side.

  In addition, even when the transmission side is a terminal device that inserts information about the encoded stream only once in the encoded stream described above, if the encoded information in the encoded stream is lost due to packet loss or the like, The encoded stream transmitted by the transmission side cannot be decoded.

  That is, taking the MPEG-4 video stream as an example, as shown in FIG. 3, if the DCI is only at the head of the MPEG-4 stream, the decoder cannot be initialized if the head DCI packet is lost on the network. In addition, the MPEG-4 stream cannot be decoded.

  In order to avoid the above problem, as in the method (1) described above, encoded information is periodically inserted into the encoded stream so that the decoder can be initialized at an arbitrary timing. It is desirable to keep it.

  Therefore, according to the present invention, the encoded information is inserted into the encoded stream, so that the received terminal device can receive the encoded code regardless of how the encoded information is transmitted to the communication partner in the transmitting terminal device. It is an object of the present invention to enable normal decoding of a stream.

  In addition, when it is known that encoded information is exchanged in advance using SDP and the decoder is initialized using the encoded information, the encoded information in the encoded stream only increases network traffic. It becomes useless information.

  Accordingly, an object of the present invention is to reduce network traffic by deleting redundant encoded information in an encoded stream.

  In order to achieve the above object, as shown in FIG. 4, the present invention provides encoded information transmitted from a transmission terminal prior to transmission of an encoded stream in an encoded stream relay device that relays the encoded stream. Or the encoded information inserted in the encoded stream is stored, and the stored encoded information is inserted into the encoded stream at a predetermined timing, preferably at a predetermined period, or The encoded information is replaced with the stored encoded information.

  In addition, the present invention is characterized in that redundant coding information, for example, coding information included exceeding a predetermined frequency is deleted from a coded stream to be relayed.

  In the present invention, the encoded stream relay device inserts encoded information, which is information necessary for decoding the encoded stream, into the encoded stream. Therefore, even if the terminal device on the transmission side does not insert the encoded information in the encoded stream or the encoded information inserted only once is lost during transmission, The terminal apparatus can initialize the decoder using the encoded information inserted into the encoded stream by the encoded stream relay apparatus, and can normally decode the received encoded stream.

  Further, according to the present invention, since the encoded stream relay device deletes useless encoded information from the encoded stream, network traffic can be reduced.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 5 shows the entire system including the coded stream relay device of the present invention. The coded stream relay device of the present invention, for example, 1 is arranged at a relay point of networks a and b having different protocols, and each network a , B are relayed between the encoded streams between the terminal devices A and B respectively connected to b. Although only a pair of terminal devices are shown in the drawing, it goes without saying that encoded streams between a plurality of pairs of terminal devices can be relayed simultaneously.

  FIG. 6 shows an example of an embodiment of an encoded stream relay device of the present invention, and here shows an example of an encoded stream relay device that relays an MPEG-4 video stream as an example of an encoded stream. 11 is a capability exchange unit, 12 is a stream receiving unit, 13 is a stream analyzing unit, 14 is an encoded information storage unit, 15 is an encoded information edit determining unit, 16 is an encoded information editing unit, and 17 is a stream transmitting unit. .

  Hereinafter, the configuration of each unit will be described together with its operation.

  The capability exchanging unit 11 performs the capability exchange processing, that is, H. Like a TV phone device using a protocol such as H.323 or SIP, it is acquired in a capability exchange process for exchanging video stream encoded information before starting transmission / reception of the video stream, and this is obtained in the encoded information storage unit 14. hand over. In the case of an MPEG-4 stream, the encoded information means DCI.

  The stream receiving unit 12 receives the MPEG-4 stream transmitted from the terminal device on the transmission side after the capability exchange.

  The stream analysis unit 13 analyzes the MPEG-4 stream received by the stream reception unit 12 and acquires this if the MPEG-4 stream includes DCI. In addition, the stream analysis unit 13 passes the acquired DCI to the encoded information storage unit 14 and also passes the MPEG-4 stream received by the stream reception unit 12 to the encoded information edit determination unit 15.

  The encoded information storage unit 14 stores one or both of the DCI acquired by the capability exchange unit 11 and the DCI acquired by the stream analysis unit 13. That is, the DCI acquired by the capability exchange unit 11 is stored, and when the DCI is acquired by the stream analysis unit 13, the stored DCI may be deleted and replaced with the stored DCI. The saved DCI may be left as it is, and the DCI acquired by the stream analysis unit 13 may not be saved, or both may be saved.

  The encoding information editing determination unit 15 analyzes the position where DCI should be inserted, replaced, or deleted in the received MPEG-4 stream passed from the stream analysis unit 13, and further performs DCI analysis on the received MPEG-4 stream. It is determined whether or not processing of insertion or replacement or deletion is to be performed (or neither is performed), and the determination result is inserted into the received MPEG-4 stream and DCI in the received MPEG-4 stream Alternatively, it is passed to the encoded information editing unit 16 together with the position to be replaced or deleted.

  Here, the position where DCI is to be inserted (when DCI is inserted, the position where DCI may be inserted) is generally immediately before the frame. Therefore, when inserting DCI, it is sufficient to detect the head of a frame from the MPEG-4 stream and insert it immediately before.

By the way, if DCI is inserted immediately before every frame, a result of deteriorating network efficiency is caused. Therefore, it is better to minimize the number of DCI insertions. In order to solve this problem, for example,
Insert DCI only immediately before the intra frame (I-VOP),
-Insert DCI at regular intervals (eg 3 seconds),
Perform the following process.

  In the encoded information edit determination unit 15, whether to insert DCI into the received MPEG-4 stream, replace DCI of the received MPEG-4 stream, or delete DCI from the received MPEG-4 stream, The following judgment is made.

  Based on the analysis result of the position where DCI is to be inserted or replaced or deleted, for example, when DCI is not inserted immediately before an intra frame (I-VOP) or for a long time of a certain interval (for example, 3 seconds) or more In the received MPEG-4 stream, the DCI (several DCIs stored in the encoded information storage unit 14 are stored at the position where the DCI detected by the above method is to be inserted. Insert one of them).

  Further, based on the analysis result of the position where DCI is inserted or replaced or deleted, for example, when DCI is inserted in addition to immediately before the intra frame (I-VOP), or when it is less than a certain interval (for example, 3 seconds) When it is determined that DCI is included in a received MPEG-4 stream exceeding a predetermined frequency, such as when DCI is inserted at short intervals, these DCIs are redundant DCI. Determine and delete these redundant DCIs from the received MPEG-4 stream until a predetermined frequency.

  Further, if the collation result between the DCI newly acquired by the stream analysis unit 13 and the DCI stored in the encoded information storage unit 14 does not match, the DCI inserted in the received MPEG-4 stream is erroneously detected. And the DCI in the received MPEG-4 stream is replaced with the DCI stored in the encoded information storage unit 14 (one of which is stored when a plurality of DCIs are stored) (encoding information) If it is determined that the DCI stored in the storage unit 14 is an error, the DCI is overwritten and stored with the newly acquired DCI.)

  The encoded information editing unit 16 determines whether to perform DCI insertion, replacement, or deletion processing on the received MPEG-4 stream passed from the encoded information editing determination unit 15 and the received result. Based on the position where the DCI in the MPEG-4 stream is to be inserted, replaced or deleted, the DCI is inserted into the received MPEG-4 stream, which is also passed from the encoding information editing determination unit 15, or the DCI is The process of replacing or deleting the DCI is performed, and the MPEG-4 stream after these processes is passed to the stream transmission unit 17.

  The stream transmission unit 17 packetizes and transmits the MPEG-4 stream after the encoded information editing process passed from the encoded information editing unit 16 in a format suitable for the receiving terminal device (network). For example, when transmitting to an IP network, an MPEG-4 stream is converted into an RTP (Real-time Transport Protocol) packet and transmitted.

With these processes, it is possible to periodically insert DCI into the MPEG-4 stream to be relayed. as a result,
Since the MPEG-4 stream includes DCI, the decoder can be initialized.
Even if the first DCI disappears, the decoder can be initialized using the periodically inserted DCI.
As a result, the terminal device on the receiving side can always normally decode the received MPEG-4 stream.

  The encoded stream relay apparatus 1 can also be realized by installing a program for realizing the functions shown in the configuration diagram of FIG. 6 via a recording medium or a communication line in a known computer.

  In the coded stream relay device, both the capacity exchange unit 11 and the stream analysis unit 13 are not necessarily required, and only one of them may be provided. In addition, with respect to insertion or replacement of DCI, encoding is performed without performing determination of a period in which DCI is not inserted or comparison between newly acquired DCI and stored DCI in encoding information editing determination unit 15. The information editing unit 16 may uniformly insert DCI at regular intervals or replace it with stored DCI. At this time, if the encoded information editing unit 16 analyzes the position where the DCI in the received MPEG-4 stream should be inserted, replaced, or deleted, the encoded information edit determining unit 15 may be omitted. Is possible.

  In the embodiments described so far, encoded stream transmission between terminal apparatuses connected to networks having different protocols has been described. However, the present invention is applicable to terminal apparatuses connected to networks having the same protocol. The present invention can also be applied to encoded stream transmission between terminal apparatuses having different capability exchange processing methods.

Explanatory drawing of the method of inserting encoding information into an encoding stream periodically. Explanatory drawing of the method of transmitting encoding information separately from an encoding stream. Explanatory drawing when the encoding information in the encoding stream is lost Explanatory drawing which shows the outline | summary of this invention Configuration diagram of the entire system including the encoded stream relay device of the present invention The block diagram which shows an example of embodiment of the encoding stream relay apparatus of this invention

Explanation of symbols

  1: encoded stream relay device, 11: capability exchange unit, 12: stream receiving unit, 13: stream analysis unit, 14: encoded information storage unit, 15: encoded information editing determination unit, 16: encoded information editing unit , 17: Stream transmission unit.

Claims (13)

An encoded stream relay device that relays an encoded stream when transmitting the encoded stream between terminal devices connected to a network,
Capability exchange means for acquiring encoded information, which is information about the encoded stream, by capability exchange processing;
Encoded information storage means for storing the encoded information acquired by the capability exchange means;
Stream receiving means for receiving the encoded stream transmitted from the terminal device on the transmitting side;
Encoded information editing determining means for determining whether to insert or replace encoded information for the encoded stream received by the stream receiving means;
Based on the determination result of the encoded information editing determining means, the encoded information stored in the encoded information storing means is inserted into the encoded stream received by the stream receiving means, or received by the stream receiving means Encoded information editing means for replacing the encoded information contained in the encoded stream with the encoded information stored in the encoded information storage means;
A coded stream relay device comprising: a stream transmission unit that transmits a coded stream in which coded information is inserted or replaced by the coded information editing unit to a terminal device on the receiving side. An encoded stream relay device that relays an encoded stream when transmitting the encoded stream between terminal devices connected to a network,
Stream receiving means for receiving the encoded stream transmitted from the terminal device on the transmitting side;
A stream analyzing unit that analyzes the encoded stream received by the stream receiving unit and obtains encoded information that is information about the encoded stream;
Encoding information storage means for storing the encoding information acquired by the stream analysis means;
Encoded information editing determining means for determining whether to insert or replace encoded information for the encoded stream received by the stream receiving means;
Based on the determination result of the encoded information editing determining means, the encoded information stored in the encoded information storing means is inserted into the encoded stream received by the stream receiving means, or received by the stream receiving means Encoded information editing means for replacing the encoded information contained in the encoded stream with the encoded information stored in the encoded information storage means;
A coded stream relay device comprising: a stream transmission unit that transmits a coded stream in which coded information is inserted or replaced by the coded information editing unit to a terminal device on the receiving side. The coded information editing determining means, based on said presence or absence of encoding information of the encoded stream received by the stream receiving unit, according to claim 1, wherein the determining whether to insert the coding information Or the encoded stream relay device according to 2; The encoded information edit determining means is based on a comparison result between the encoded information in the encoded stream received by the stream receiving means and the encoded information stored in the encoded information storing means. encoded stream relay device according to claim 1 or 2, wherein the determining whether to perform the replacement. The encoded information editing determining means further determines whether or not the encoded information is included in the encoded stream received by the stream receiving means in excess of a predetermined frequency;
The encoded information editing means further deletes encoded information included exceeding a predetermined frequency from the encoded stream received by the stream receiving means based on the determination result of the encoded information editing determining means. The encoded stream relay device according to any one of claims 1 to 4 . The encoded stream relay device according to any one of claims 1 to 5 , wherein the encoded stream is an MPEG-4 video stream, and the encoded information is Decoder Configuration Information. An encoded stream relay method in an encoded stream relay device that relays an encoded stream when transmitting the encoded stream between terminal devices connected to a network,
The encoded stream relay device
A capability exchange step of acquiring encoded information, which is information about the encoded stream, by capability exchange processing;
An encoded information storage step for storing the encoded information acquired in the capability exchange step;
A stream reception step of receiving an encoded stream transmitted from a terminal device on the transmission side;
An encoded information editing determination step for determining whether to insert or replace encoded information for the encoded stream received in the stream receiving step;
Based on the determination result of the encoding information editing determination step, the encoding information stored in the encoding information storage step is inserted into the encoded stream received in the stream reception step, or the encoding received in the stream reception step An encoded information editing step for replacing the encoded information included in the stream with the encoded information stored in the encoded information storing step;
And a stream transmission step of transmitting the encoded stream in which the encoded information is inserted or replaced in the encoded information editing step to a terminal device on the receiving side. An encoded stream relay method in an encoded stream relay device that relays an encoded stream when transmitting the encoded stream between terminal devices connected to a network,
The encoded stream relay device
A stream reception step of receiving an encoded stream transmitted from a terminal device on the transmission side;
A stream analysis step of analyzing the encoded stream received in the stream receiving step and acquiring encoded information that is information relating to the encoded stream;
An encoding information storage step for storing the encoding information acquired in the stream analysis step;
An encoded information editing determination step for determining whether to insert or replace encoded information for the encoded stream received in the stream receiving step;
Based on the determination result of the encoding information editing determination step, the encoding information stored in the encoding information storage step is inserted into the encoded stream received in the stream reception step, or the encoding received in the stream reception step An encoded information editing step for replacing the encoded information included in the stream with the encoded information stored in the encoded information storing step;
And a stream transmission step of transmitting the encoded stream in which the encoded information is inserted or replaced in the encoded information editing step to a terminal device on the receiving side. The coded information editing judgment step is based on the presence or absence of encoding information of the stream reception step encoded stream received in claim 7, characterized in that determines whether or not to insert the coding information Or the encoded stream relay method according to 8. The encoded information editing determination step replaces the encoded information based on the comparison result between the encoded information in the encoded stream received in the stream receiving step and the encoded information stored in the encoded information storing step. The encoded stream relay method according to claim 7 or 8 , wherein it is determined whether or not to perform. The encoded information edit determining step further determines whether the encoded information received in the stream receiving step includes encoded information exceeding a predetermined frequency,
The encoded information editing step further deletes encoded information included exceeding a predetermined frequency from the encoded stream received in the stream receiving step based on a determination result of the encoded information editing determining step. The encoded stream relay method according to any one of claims 7 to 10 . The encoded stream relay method according to claim 7 , wherein the encoded stream is an MPEG-4 video stream, and the encoded information is Decoder Configuration Information. A program that causes a computer that functions as an encoded stream relay apparatus to execute each step of the encoded stream relay method according to any one of claims 7 to 12 .

Images