JPH08298508A - Multiplex data mutual communication method, multiplex data mutual communication system, terminal on gateway and asynchronous network - Google Patents

Abstract

PURPOSE: To realize the mutual communication of multiplex data between a terminal on an asynchronous network and a terminal on a synchronous network with simple device constitution. CONSTITUTION: At the time of the mutual communication of multiplx data between a terminal 400 on the asynchronous network 200 dealing with structural data which is made into structure by the plural frames of fixed size and a terminal 600 on the asynchronous network 300 dealing with an elementary stream which is formed by making data into a packet and multiplexing it and to which time information is added, the size of the packet in the elementary stream is set based on the unit of the frame, and time stamps showing the period of decoding reproduction are added to the respective packets in the elementary stream. Thus, the synchronous state of multiplex data on structural data can be displayed by the time stamps and data which are packet-multiplexed can easily be synchronized at the time of decoding reproduction.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multiplexed data intercommunication method, a multiplexed data intercommunication system, a gateway and a terminal on an asynchronous network. More specifically, the invention is structured by a plurality of frames of a certain size. A terminal on a synchronous network that handles data, a terminal on an asynchronous network that handles an elementary stream formed by packetizing and multiplexing data and having time information added,
A method and apparatus for enabling mutual communication of multiplexed data between devices.

[0002]

2. Description of the Related Art A conventional telecommunications network is shown in FIG.
As shown in FIG. 6, there are a plurality of networks, and each network provides a separate service. FIG.
6 (a) is a typical connection form of a telephone network, FIG. 16 (b) is a typical connection form of a data network, FIG. 16 (c) is a typical connection form of a facsimile network, and FIG. 16 (d) is a leased line network. The respective typical connection forms are shown. In the figure, 1 and 2 are telephones, 5, 6 and 16 are personal computers,
11 and 12 are facsimile machines, 15 is a host computer, 3, 4, 7, 8, 13, 14, 17 and 1.
8 is a user network interface (UNI: User Network)
Interface), 9, 10, 19 and 20 are network terminating devices (DSU: Digital Service Unit).

In recent years, as a means for integrating the networks shown in FIGS. 16 (a) to 16 (d), an integrated services digital network (ISDN) is integrated.
twork), and various services such as telephone communication, data communication, facsimile communication, and video communication are provided on the same network in ISDN. Currently, ISDN that actually provides services is a narrowband ISDN (N-IS
DN: Narrowband aspects of ISDN), which is based on 64 kbps of the telephone network.
kbps (B channel), 384 kbps (H0 channel), 1536 kbps (H11 channel), 1920
Telephone communication at a transmission rate of kbps (H12 channel),
It provides various services such as data communication, facsimile communication, and video communication.

A synchronous transfer mode (STM: Synchronous Transfer Mode) shown in FIG. 17 is used for data transmission of various services in the above-mentioned N-ISDN. In this STM, a voice data transfer time (125) is used.
.mu.m) as a basic time, the data is bundled and sent to the transmission line. Further, in STM, since there is no header in the time slot of each channel, it is impossible to distinguish whether each transfer bit is significant information or unnecessary information. For this reason, it is necessary to constantly send not only meaningful information but also unnecessary information in a constant cycle, and as shown in FIG. 17, it is necessary to put an unnecessary transfer bit in the n-th bit of each transfer bit frame. is there.

When transmitting digitized image data and audio data on a network that employs the above-described STM (hereinafter referred to as a synchronous network), according to a clock that is synchronized with a synchronous network clock set in the synchronous network. Data is transmitted. Further, in the voice encoding used for the digital telephone on the N-ISDN, the sampling clock of voice data is generated from the above-mentioned synchronous network clock.

More recently, as shown in FIG. 18, an asynchronous transfer mode (ATM) is used.
The data transmission method called r Mode) is ITU-T (In
ternational Telecommunication Union-Telecommunic
ation Standardization Sector) for wideband ISDN (B
-ISDN: Broadband-aspects of ISDN) are being standardized. In this ATM, digital information is transmitted as 53-byte fixed-length information called cells, and more specifically, the digital information is divided into fixed lengths of 48 bytes, and control information called 5-byte headers is divided in each section. It is added and transmitted as fixed length information of 53 bytes in total.

Unlike the STM, this ATM has a variable data transmission rate and is capable of high-speed communication of 155 Mbps or higher. According to the characteristic that the ATM data transmission speed is variable, when transmitting digitally expressed image data or audio data, the data can be transmitted according to the source clock of the data transmission source.

[0008] Multimedia information has also been transmitted via the above-mentioned digital network, and video data representing an image, audio data representing an audio, and the like are multiplexed to form a videophone, a videoconference device, or the like. Has been realized. For example, a schematic configuration of a terminal (H.320 terminal) of a videophone / videoconference system device for N-ISDN is shown in FIG. The video data and audio data are the same as those in H.264 of the same figure. 221 is multiplexed or demultiplexed by the multimedia multiplexing / demultiplexing unit adopting H.221. 22
In the case of 1, the data is framed and handled. In addition, H. Two
21, the basic data of the multiplexed data is G.21. Since it becomes 711 audio data, each multiplexed data matches the octet timing (8 kHz reference signal), and the most significant bit (MSB:
Most Significant Bit to least significant bit (LSB: Le
ast Significant Bit).

For example, the structure of a frame in a 64 kbps channel is shown in FIG. As described above, H.264. Since the data in 221 is handled in octet units, in the case of a 64 kbps channel, one frame is composed of 80 octet data as shown in FIG. Each bit in the octet is called a sub-channel, and particularly the eighth sub-channel is called a service channel (SC). This service channel is a frame synchronization signal (FAS: Fram).
It includes an e Alignment Signal) and a bit rate allocation signal (BAS).

On the other hand, the frame structure of a channel of 128 kbps or more is shown in FIG. This 128kbps
In the case of the above channels, the data of each frame is 8
Every 8 bits are assigned to each of a plurality of time slots in accordance with the octet timing of kHz (125 μs). Therefore, each time slot is 64k
Corresponding to a bps channel, the first time slot is
It has the same structure as 64 kbps shown in FIG. 20, but the remaining time slots do not include a service channel.
In the frame structure data shown in FIG. 21, 8-bit data of the first octet of each time slot is serially transmitted sequentially from the first time slot to the 6n-th time slot, and this operation is performed from the second octet to the 80-th time. It is designed to be repeated in order until the octet.

An even frame and an odd frame form one sub-multi-frame, and a plurality of sub-multi-frames are multiplexed to form one multi-frame. The arrangement of the 1st to 8th bits of the service channel in each frame in the multi-frame is shown in FIG. The FAS described above is composed of a combination of even frames and odd frames, and includes multi-frame information in the first bit of even and odd frames. A multi-frame number is assigned to the first bit of an even frame, and a multi-frame synchronization signal (MA) is assigned to the first bit of an odd frame.
S: Multiframe Alignment Signal) is inserted. Further, in the 2nd to 8th bits of the even frame and the 2nd bit of the odd frame, a frame synchronization word (FAW: Frame Alignment) for determining the frame.
Word) has been inserted.

By the frame sync word, the multiframe sync signal and the multiframe number described above, FIG.
Three-dimensional data expanded in the space and time directions can be expressed from the one-dimensional serial data as shown in FIG. In addition, H. An example of the multiplexing rule in 221 is shown in FIG.
4 and FIG. 25, FIG. 24 shows a multiplexing rule for n × B channels, and FIG. 25 shows a multiplexing rule for H0 channels. In FIG. 24, F is FAS, B is BAS, MLP is Multi Layered Protocol, and H is
SD represents high-speed data (High Speed Data), and LSD represents low-speed data (Low Speed Data). These multiplexings are controlled by BAS,
That is, BAS represents the frame structure information. The BAS determines the frame structure of the next sub-multiframe in which the BAS is multiplexed.

On the other hand, the transmission of multimedia information on ATM is carried out according to ISO / IEC MPEG (Moving Picture).
Experts Group) is promoting standardization I
It is specified in SO / IEC 11172-1 (hereinafter referred to as MPEG1 system) and ISO / IEC 13818-1 (hereinafter referred to as MPEG2 system). According to the MPEG2 system, image data (hereinafter,
As shown in FIG. 26, the PES and the audio data (hereinafter referred to as audio data) are
Packetized in a format called (Packetized Elementary Stream).

In addition, the MPEG2 system defines two types of formats, a program stream and a transport stream. The program stream is mainly prepared for storage media and is used for transmission of only one program, while the transport stream is mainly prepared for communication and broadcasting media and is used for transmission of one program. Not only can it be used for transmitting multiple programs. The program mentioned here means, for example, one TV program, and includes an audio stream, a video stream, and the like.

The program stream and the transport stream are both multiplexed in a packet structure. Further, data called SCR (system clock reference) or PCR (program clock reference) obtained by sampling STC (system clock) as a reference clock is also multiplexed and added to the packet of the stream. The SCR is used in the program stream, and the PCR is used in the transport stream and is added to each program. Specifically, the SCR and PCR are used to set the time reference STC in the MPEG system decoder including the video and audio decoders, that is, the value of the reference synchronization information to the value intended by the encoder side. .

Further, when decoding and reproducing the received data on the receiving side, the time stamp added to the program stream and the transport stream is used. The time stamp is like a tag for time management of decoding / reproduction processing added to each access unit of video and audio. When the head of the access unit, which represents the unit for decoding and reproducing video and audio, is included in the packet, a time stamp is added to the header of the packet.

On the other hand, when the head of the access unit is not included in the packet, the time stamp is not added to the header of the packet. Further, even if two or more access units are at the head of one packet, only the time stamp corresponding to the first access unit is added to the header of the packet.

There are two types of time stamps, PTS (Presentation Time Stamp) representing reproduction output time management information and DTS (Decoding Time Stamp) representing decoding time management information. The PTS is represented by a value of 33 bits measured with a clock of 90 kHz, and when the STC in the MPEG system matches the PTS, the access unit corresponding to the PTS is reproduced and output.

The DTS is set corresponding to the transmission procedure of a special encoded stream of video in MPEG, that is, the DTS is set corresponding to the difference in the decoding order and the reproduction output order. Specifically, MPE
For a video stream containing B pictures in G, I
Both PTS and DTS for changing the reproduction order are added to packets corresponding to pictures and P pictures, respectively, and only PTS is added to packets corresponding to B pictures.

FIG. 27 shows the data structure of the transport stream of the program stream and the transport stream. The transport stream is obtained by repacketizing the PES, as shown in FIG. Further, as shown in FIG. 29, there is a method being considered for adoption by the ATM Forum registered as a private non-profit organization in the United States, as shown in FIG. 29, in order to map a transport stream to an ATM cell handled on an asynchronous network. . In FIG. 29, the transport stream is mapped to AAL5. In addition, the TS packet including the PCR is used in the CPCS-PDU in order to reduce the cell sticking fluctuation.
The default mapping is TS packets of 2 or less, and others are negotiated.

The protocol related to ATM corresponds to the OSI reference model, and is from the lower layer to the physical layer to the ATM.
Layer, ATM adaptation layer (AAL: ATM Adap
tationLayer) is classified into three layers. The AAL is prepared in order to absorb a difference in communication characteristics required for various communication services such as voice communication, video communication, and data communication and to secure a cell transfer function which is a basic function of ATM.

Currently, ITU-T has five types of AAL,
That is, AAL types 1 to 5 are under study. AAL
Type 1 provides fixed speed services such as conventional voice communication services and existing leased line services.
The AAL type 2 is for providing variable speed encoded audio data, video data and the like at a variable speed.

The AAL type 3 provides connection-type data communication (frame relay, packet communication, etc.) and signal transfer, and the AAL type 4 provides connectionless-type data communication. The AAL type 5 has specifications designed for application to data communication services and signaling, and has a function of disassembling and assembling user information into cells and a payload type (P
The function of AAL is simplified as much as possible, such as identification by T), so that efficient data transfer can be realized, and it is also used for transmitting call connection control data.

Even when terminals on the network adopting the STM such as the conventional telephone network or N-ISDN and terminals on the network adopting the ATM coexist, these terminals can communicate with each other. For example, the above-mentioned AAL type 1 is prepared for mutual communication between a terminal on an asynchronous network and a terminal on a synchronous network or a terminal for a synchronous network on an asynchronous network. AAL type 1 provides functions such as disassembling and assembling user information, processing for lost cells and post-inserted cells, delay fluctuation compensation, and receiving side recovery of transmitting side clock frequency.

[0025]

SUMMARY OF THE INVENTION However, for example, H.264. In order to realize mutual communication between a terminal on the synchronous network that handles 221 data and a terminal on the asynchronous network that handles the MPEG system, the H. It is necessary to have the multiplexing function of 221 and to transmit by AAL1. Therefore, the terminal on the asynchronous network is M
In addition to the functions of the PEG system, H.264. It has the function of 221 and is absolutely necessary for controlling the network such as calling procedures.
In addition to AL5, it is necessary to have the AAL1 function, which causes a problem that the device configuration of the terminal on the asynchronous network becomes complicated.

Therefore, the present invention provides a multiplexed data mutual communication method and a multiplexed data mutual communication method capable of realizing mutual communication of multiplexed data between a terminal on an asynchronous network and a terminal on a synchronous network with a simple device configuration. It is an object to provide a communication system, a gateway, and a terminal on an asynchronous network.

[0027]

According to the first aspect of the present invention,
In order to solve the above problem, a terminal on a synchronous network that handles structured data structured by a plurality of frames of a certain size is formed by packetizing and multiplexing the data, and time information is added. In a multiplexed data mutual communication method that enables mutual communication of multiplexed data between a terminal on an asynchronous network that handles elementary streams, a packet size in the elementary stream based on the unit of the frame. And a time stamp adding step of adding a time stamp indicating the time of decoding and reproduction to each packet in the elementary stream.

In order to solve the above-mentioned problems, the invention according to claim 2 is the multiplexed data mutual communication method according to claim 1, wherein multiplexed data is transmitted from a terminal on the synchronous network to an asynchronous network terminal. At this time, a decoding / reproducing step of decoding / reproducing the elementary stream based on the time stamp is provided. Claim 3
In order to solve the above-mentioned problems, the described invention is the multiplexed data mutual communication method according to claim 1, wherein frame structure information representing a frame structure of the structured data is added to a head packet in an elementary stream. The frame structure information adding step, the time stamp of the packet to which the frame structure information is added, and the time stamp of each packet in the elementary stream following the packet to which the frame structure information is added are synchronized to each packet. A frame structure changing step of changing the structure of the corresponding frame based on the frame structure information.

In order to solve the above-mentioned problems, the invention according to claim 4 is the multiplexed data mutual communication method according to claim 1, wherein when a terminal on an asynchronous network receives data from a terminal on a synchronous network, A packetization timing reproduction step of reproducing packetization timing of each packet based on a time stamp, and the packet reproduced in the packetization timing reproduction step when a terminal on an asynchronous network transmits data to a terminal on a synchronous network And a data amount determining step of determining a data amount in the elementary stream between conversion timings based on the frame structure information.

In order to solve the above-mentioned problems, an invention according to claim 5 is the multiplexed data mutual communication method according to claim 1, wherein when a terminal on an asynchronous network receives data from a terminal on a synchronous network, When the terminal on the asynchronous network sends data to the terminal on the synchronous network, the receive clock recovery step of recovering the receive clock based on the time stamp and the transmit clock based on the receive clock recovered in the receive clock recovery step And a step of generating a transmission clock to be generated.

In order to solve the above-mentioned problems, the invention according to claim 6 is the multiplexed data mutual communication method according to claim 1, wherein the structured data is ITU-T (International).
Telecommunication Union-Telecommunication Standa
rdization Sector). 221 standard, and the elementary streams are ISO / IEC (In
ternational Organization for standardization / Inter
National Electrotechnical Commission) 11172
-1 or 13818-1 standard.

In order to solve the above-mentioned problems, the invention of claim 7 packetizes and multiplexes data with a terminal on a synchronous network that handles structured data structured by a plurality of frames of a fixed size. In a multiplexed data mutual communication system that enables mutual communication of multiplexed data between a terminal on an asynchronous network that handles elementary streams formed by A packet size setting means for setting the size of the packet in the elementary stream, and a time stamp adding means for adding a time stamp indicating the time of decoding and reproduction to each packet in the elementary stream. It is characterized by.

In order to solve the above-mentioned problems, the invention according to claim 8 is the multiplexed data mutual communication system according to claim 7, wherein multiplexed data is transmitted from a terminal on the synchronous network to an asynchronous network-like terminal. At this time, a decoding / reproducing means for decoding / reproducing the elementary stream based on the time stamp is provided. Claim 9
In order to solve the above-mentioned problems, the described invention adds the frame structure information representing the frame structure of the structured data to the head packet in the elementary stream in the multiplexed data intercommunication system according to claim 7. The frame structure information adding means synchronizes the time stamp of the packet to which the frame structure information is added with the time stamp of each packet in the elementary stream following the packet to which the frame structure information is added, and Frame structure changing means for changing the structure of the corresponding frame based on the frame structure information.

In order to solve the above-mentioned problems, the invention according to claim 10 is the multiplexed data intercommunication system according to claim 7, wherein when a terminal on an asynchronous network receives data from a terminal on a synchronous network, Packetizing timing reproducing means for reproducing the packetizing timing of each packet based on the time stamp, and the packet reproduced by the packetizing timing reproducing means when the terminal on the asynchronous network transmits data to the terminal on the synchronous network. And a data amount determining means for determining the data amount in the elementary stream between conversion timings based on the frame structure information.

In order to solve the above-mentioned problems, the invention according to claim 11 is the multiplexed data intercommunication system according to claim 7, wherein when a terminal on an asynchronous network receives data from a terminal on a synchronous network, Reception clock reproduction means for reproducing the reception clock based on the time stamp, and when the terminal on the asynchronous network transmits data to the terminal on the synchronous network,
Transmission clock generation means for generating a transmission clock based on the reception clock reproduced by the reception clock reproduction means.

In order to solve the above-mentioned problems, the invention according to claim 12 is the multiplexed data intercommunication system according to claim 7, wherein the structured data is ITU-T (Internat).
ional Telecommunication Union-Telecommunication
H. Standardization Sector). 221 standard, and the elementary stream is ISO / IEC
(International Organization for standardization / I
nternational Electrotechnical Commission) 11
It is characterized in that it is formed according to the 172-1 or 13818-1 standard.

In order to solve the above-mentioned problems, the invention of claim 13 is a gateway provided between a synchronous network and an asynchronous network, which handles structured data structured by a plurality of frames of a fixed size. Enables mutual communication of multiplexed data between terminals on the network and terminals on an asynchronous network that handles elementary streams that are formed by packetizing and multiplexing data and that have time information added A packet size setting means for setting the size of a packet in the elementary stream based on the unit of the frame, and a time stamp indicating the time of decoding and reproduction for each packet in the elementary stream. And a time stamp adding means for adding.

In order to solve the above-mentioned problems, the invention according to claim 14 adds the frame structure information representing the frame structure of the structured data to the first packet in the elementary stream in the gateway according to claim 13. Frame structure information adding means for synchronizing the time stamp of the packet to which the frame structure information is added and the time stamp of each packet in the elementary stream following the packet to which the frame structure information is added, And frame structure changing means for changing the structure of the frame corresponding to the above, based on the frame structure information.

In order to solve the above problems, the invention according to claim 15 is the gateway according to claim 13, wherein the structured data is ITU-T (International Telecomm).
unication Union-Telecommunication Standardizatio
n Sector). 221 standard, and the elementary stream is an ISO / IEC (Internationa)
l Organization for standardization / International E
It is characterized in that it is formed according to the 11172-1 or 13818-1 standard of the Electrical Technical Commission.

In order to solve the above-mentioned problems, a sixteenth aspect of the present invention is formed by forming a synchronous network for handling structured data structured by a plurality of frames of a fixed size, and packetizing and multiplexing the data. A packet that sets the size of the packet in the elementary stream based on the unit of the frame in order to enable mutual communication of multiplexed data through an asynchronous network that handles elementary streams to which time information is added. A terminal on an asynchronous network applied to a multiplexed data intercommunication system, comprising size setting means and time stamp adding means for adding a time stamp indicating the time of decoding and reproduction to each packet in the elementary stream. Then, when the multiplexed data is received from the terminal on the synchronous network, It is characterized in that it has a decoding and reproducing means for decoding and reproducing the elementary stream based on the flop. In order to solve the above-mentioned problems, an invention according to claim 17 is the terminal on the asynchronous network according to claim 16, wherein when data is received from the terminal on the synchronous network, decoding / reproduction included in the elementary stream is performed. Packetizing timing reproducing means for reproducing the packetizing timing of each packet based on the time stamp indicating the timing of the packet, and the packetizing timing reproduced by the packetizing timing reproducing means when transmitting data to the terminal on the synchronous network. And a data amount determining means for determining the amount of data in the elementary stream between them based on the frame structure information.

In order to solve the above-mentioned problems, the present invention of claim 18 is, in a terminal on an asynchronous network according to claim 16, when data is received from a terminal on a synchronous network, a reception clock based on the time stamp. And a transmission clock generation means for generating a transmission clock based on the reception clock reproduced by the reception clock reproduction means when transmitting data to the terminal on the synchronous network. It is what

In order to solve the above problems, the invention according to claim 19 is the terminal on the asynchronous network according to claim 16, wherein the structured data is ITU-T (International Te
lecommunication Union-Telecommunication Standard
H .. Sector). 221 standard,
The elementary stream is ISO / IEC (Intern
ational Organization for standardization / Internati
onal Electrotechnical Commission) 11172-1
Alternatively, it is formed according to the 13818-1 standard.

[0043]

According to the first aspect of the invention, the size of the packet in the elementary stream is set based on the unit of the frame of the structured data, and the time indicating the decoding / playback time is set for each packet in the elementary stream. A stamp is added. Therefore, the synchronization state of the multiplexed data of the structured data can be represented by a time stamp, and the synchronization between the packet multiplexed data can be easily achieved at the time of decoding and reproducing.

According to the second aspect of the invention, in the multiplexed data mutual communication method according to the first aspect, when the multiplexed data is transmitted from a terminal on a synchronous network to a terminal on an asynchronous network,
The elementary stream is decoded and reproduced based on the time stamp. Therefore, a terminal on the asynchronous network has a function of handling structured data, for example, H.264. It is not necessary to have 221 and the device configuration of the terminal can be simplified.

According to the invention of claim 3, in the method of mutual communication of multiplexed data according to claim 1, frame structure information representing a frame structure of structured data is added to a head packet in an elementary stream, The time stamp of the packet to which the structure information is added and the time stamp of each packet in the elementary stream following the packet to which the frame structure information is added are synchronized, and the structure of the frame corresponding to each packet is the frame structure. Changed based on information. Therefore, it is possible to change the frame structure at an accurate time.

According to a fourth aspect of the present invention, in the multiplexed data mutual communication method according to the first aspect, when a terminal on the asynchronous network receives data from a terminal on the synchronous network, each packet is transmitted based on a time stamp. The packetization timing is regenerated. When the terminal on the asynchronous network transmits data to the terminal on the synchronous network, the amount of data in the elementary stream between the packetization timings reproduced in the packetization timing reproduction step is based on the frame structure information. It is determined. Therefore, it is possible to accurately reproduce the frame unit of the structured data.

In the fifth aspect of the present invention, in the multiplexed data mutual communication method according to the first aspect, when the terminal on the asynchronous network receives the data from the terminal on the synchronous network, the reception clock is set based on the time stamp. When the terminal on the asynchronous network is regenerated and transmits data to the terminal on the synchronous network, the transmission clock is generated based on the reception clock regenerated in the reception clock regeneration step. Therefore, the terminal on the asynchronous network can transmit in synchronization with the clock of the terminal on the synchronous network.

According to a sixth aspect of the invention, in the multiplexed data mutual communication method according to the first aspect, the structured data is I.
H. of TU-T. 221 standard, and said elementary stream is ISO / IEC 11172-1.
Or formed according to the 13818-1 standard. Therefore, for example, a video conference can be realized between the terminals on the synchronous network and the terminals on the asynchronous network.

According to the invention described in claim 7, the size of the packet in the elementary stream is set based on the unit of the frame of the structured data, and each packet in the elementary stream has a time indicating the time of decoding and reproduction. A stamp is added. Therefore, the synchronization state of the multiplexed data of the structured data can be represented by a time stamp, and the synchronization between the packet multiplexed data can be easily achieved at the time of decoding and reproducing.

In the eighth aspect of the present invention, in the multiplexed data mutual communication system according to the seventh aspect, when the multiplexed data is transmitted from the terminal on the synchronous network to the terminal on the asynchronous network, the data is transmitted based on the time stamp. The mental stream is decoded and reproduced. Therefore, the terminal on the asynchronous network
Functions that handle structured data, such as H.264. It is not necessary to have 221 and the device configuration of the terminal can be simplified.

According to a ninth aspect of the present invention, in the multiplexed data intercommunication system according to the seventh aspect, frame structure information indicating a frame structure of structured data is added to a head packet in an elementary stream, The time stamp of the packet to which the structure information is added and the time stamp of each packet in the elementary stream following the packet to which the frame structure information is added are synchronized, and the structure of the frame corresponding to each packet is the frame structure. Changed based on information. Therefore, it is possible to change the frame structure at an accurate time.

According to a tenth aspect of the present invention, in the multiplexed data interconnection system according to the seventh aspect, when a terminal on the asynchronous network receives data from a terminal on the synchronous network, each packet of each packet is based on a time stamp. The packetization timing is regenerated. When the terminal on the asynchronous network transmits data to the terminal on the synchronous network, the amount of data in the elementary stream between the packetization timings reproduced in the packetization timing reproduction step is based on the frame structure information. It is determined. Therefore, it is possible to accurately reproduce the frame unit of the structured data.

In the eleventh aspect of the invention, in the multiplexed data interconnection system according to the seventh aspect, when a terminal on the asynchronous network receives data from a terminal on the synchronous network, the reception clock is based on the time stamp. When the terminal on the asynchronous network is regenerated and transmits data to the terminal on the synchronous network, the transmission clock is generated based on the reception clock regenerated in the reception clock regeneration step. Therefore,
The terminal on the asynchronous network can transmit in synchronization with the clock of the terminal on the synchronous network.

According to the twelfth aspect of the present invention, in the multiplexed data mutual communication system according to the seventh aspect, the structured data is H.264 of ITU-T. 221 standard, and the elementary stream is ISO / IEC 1117.
2-1 or 13818-1 standard. Therefore, for example, a video conference can be realized between the terminals on the synchronous network and the terminals on the asynchronous network.

According to the thirteenth aspect of the present invention, the size of the packet in the elementary stream is set on the basis of the frame unit of the structured data, and the time indicating the decoding / playback time is set for each packet in the elementary stream. A stamp is added. Therefore, the synchronization state of the multiplexed data of the structured data can be represented by a time stamp, and the synchronization between the packet multiplexed data can be easily achieved at the time of decoding and reproducing.

According to the fourteenth aspect of the invention, in the gateway of the thirteenth aspect, the frame structure information representing the frame structure of the structured data is added to the first packet in the elementary stream, and the frame structure information is added. The time stamp of the packet and the time stamp of each packet in the elementary stream following the packet to which the frame structure information is added are synchronized, and the structure of the frame corresponding to each packet is changed based on the frame structure information. To be done. Therefore, it is possible to change the frame structure at an accurate time.

In the invention described in claim 15, in the gateway described in claim 13, the structured data is ITU-T.
H. 221 standard, and the elementary stream is formed according to the ISO / IEC 13818-1 standard. Therefore, for example, a video conference can be realized between the terminals on the synchronous network and the terminals on the asynchronous network.

According to the sixteenth aspect of the present invention, the size of the packet in the elementary stream is set based on the frame unit of the structured data, and the time indicating the decoding / reproducing time is set for each packet in the elementary stream. In a communication system with a stamp, when multiplexed data is transmitted from a terminal on a synchronous network to a terminal on an asynchronous network, an elementary stream is decoded and reproduced based on a time stamp. Therefore, a terminal on the asynchronous network has a function of handling structured data, for example, H.264. It is not necessary to have 221 and the device configuration of the terminal can be simplified.

According to the seventeenth aspect of the invention, in the terminal on the asynchronous network according to the sixteenth aspect, when the terminal on the asynchronous network receives data from the terminal on the synchronous network, the packet of each packet is based on the time stamp. The conversion timing is reproduced. When the terminal on the asynchronous network transmits data to the terminal on the synchronous network, the amount of data in the elementary stream between the packetization timings reproduced in the packetization timing reproduction step is based on the frame structure information. It is determined. Therefore, it is possible to accurately reproduce the frame unit of the structured data.

According to the eighteenth aspect of the invention, in the terminal on the asynchronous network according to the sixteenth aspect, when the terminal on the asynchronous network receives data from the terminal on the synchronous network, the reception clock is regenerated based on the time stamp. When the terminal on the asynchronous network transmits data to the terminal on the synchronous network, the transmission clock is generated based on the reception clock regenerated in the reception clock regeneration step. Therefore, the terminal on the asynchronous network can transmit in synchronization with the clock of the terminal on the synchronous network.

According to the nineteenth aspect of the present invention, in the multiplexed data mutual communication system according to the sixteenth aspect, the structured data is H.264 of ITU-T. 221 standard,
The elementary stream is ISO / IEC 111
72-1 or 13818-1 standard. Therefore, for example, a video conference can be realized between the terminals on the synchronous network and the terminals on the asynchronous network.

[0062]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the drawings. 1 to 15 are diagrams showing an embodiment of a multiplexed data conversion system according to the present invention. First, the configuration will be described.

In FIG. 1, 100 is a gateway, and the gateway 100 is a synchronous network 200 and an asynchronous network 30.
It is arranged between 0 and has a data conversion function related to mutual communication between the two. Reference numeral 400 denotes a terminal on the synchronization, and the terminal 4
00 is connected to the synchronous network 200 through the synchronous network interface unit 500. Terminal 400 in this embodiment
ITU-T Recommendation H.264. 320 video conferencing device / videophone. On the synchronous network, it is possible to handle structured data structured by a plurality of frames of a fixed size. For example, the frame structure of one sub-multi-frame at 384 Kbps is shown in FIG. Needless to say, the structured data may be in the format of another standard or recommendation.

Reference numeral 600 denotes a terminal on the asynchronous network, and the terminal 6
00 is connected to the asynchronous network 300 through the asynchronous network interface unit 700. On an asynchronous network, it is possible to handle a packetized stream that is formed by packetizing and multiplexing data and added with time information, and the packetized stream is configured as shown in FIG. 3, for example. . The time information is added to the header of each packet. In this embodiment, the packetized stream is an ISO / IEC (International Organization for
standardization / International Electrotechnical Co
mmission) 13818-1 standard, namely MPEG2
Format. Further, the MPEG2 system has two format formats, a program stream and a transport stream, but in the present embodiment, the format format of the transport stream is followed. Of course, similar to the above, the packetized stream may be in another standard or recommended format.

As shown in FIG. 4, the gateway 100 has a synchronous network receiving interface section 101, a structured data separating section 102, a packet multiplexing section 130, and a CLA section 10.
4, asynchronous network transmission interface unit 105, PLL (Ph
ase Locked Loop) unit 106, asynchronous network reception interface unit 107, CLD unit 108, packet separation unit 19
0, a structured data multiplexing unit 110 and a synchronous network interface unit 111.

The synchronous network reception interface unit 101 uses the H.264 data from the data in which the network information and the like are multiplexed. 221 data and H.264. The clock STMCK synchronized with the 221 data is separated and extracted. Structured data separation unit 102
Separates / receives the structured data (H.221 data RHD) and the clock STMCK from the synchronous network side through the synchronous network receiving interface unit 101, and the H.264 / H. 221 structured data RHD based on FAS information and BAS information existing in a frame of 221 data, an audio flag RAF indicating the position of a received audio elementary stream,
Video flag RVF indicating the position of the reception video elementary stream, reception frame synchronization signal / transmission bit rate allocation signal flag RFBF, sub-multiframe synchronization signal RSMFSY indicating the size of the sub-multiframe.
Is generated and output, and the clock STMCK is also output. In this embodiment, the structured data is H.264. Since it is 221 data RHD, the sub-multiframe synchronization signal RS
The pulse pitch of MFSY is 20 ms.

The PLL unit 106 uses the synchronous network clock STM.
CK and line type (64 kbps, 128 kbps,
(384 kbps, etc.) is input, and a reception system clock (RSCK) that matches the transmission clock of the transmission source of the structured data is generated and output. Packet multiplexer 1
Reference numeral 30 denotes an audio buffer 131, a video buffer 132, a PES multiplexing unit 13 as shown in FIG.
3, TS multiplexing section 134, STC generation section and timing control section 136.

The audio buffer 131 receives the H.264 signal from the SACK output by the timing control section. 221
The reception audio elementary stream is separated from the data RHD and input, and the transmission audio packetized elementary stream SA is input according to the transmission audio read clock SARCK synchronized with the PES generation timing.
Output as PES.

The video buffer 132 receives the H.264 signal by the SVCK output from the timing control section. The reception video elementary stream is separated from the 221 data RHD and input, and is output as the transmission video packetized elementary stream SVPES according to the transmission video read clock SVRCK synchronized with the PES generation timing.

The PES multiplexing unit 133 transmits the packetized audio elementary stream SAPES from the audio buffer 131 and the video buffer 132.
And transmission packetized video elementary stream S
From the VPES and the timing control unit 136, the presentation time stamp SPTS, the decoding time stamp SDTS and the transmission frame synchronization signal /
A transmission bit rate allocation signal is input, and these are packetized and multiplexed to be output as a transmission packetized elementary stream SPES. In the following description, even if only the transmission / reception presentation time stamp is included in the transmission / reception presentation time stamp and the transmission / reception decoding time stamp, they are generically referred to in the following description and drawings. It is called a reception time stamp DTS.

The TS multiplexing section 134 includes the timing control section 1
Timing signal STSH and clock STCK from 36
Based on the above, the transmission elementary stream SPES is input from the PES multiplexing unit 133 and the transmission program clock reference SPCR is input from the STC generation unit 135, these are multiplexed and further packetized, and 188
Generates and outputs a transport stream consisting of byte-sized packets.

The STC generation unit 135 receives the reception system clock RSCK from the structured data separation unit 102 and generates and outputs the transmission program clock reference SPCR, the transmission system clock SSCK and the transmission system time clock SSTC. The timing control unit 136 receives the audio flag RAF, the video flag RVF, the sub-multiframe synchronization signal RSMFSY, the FAS / BAS data RF from the structured data separation unit 130.
B, transmission system clock SSCK and transmission system time clock SSTC from FASSTC generation unit 135
, And the transmission audio read SAR synchronized with the PES generation timing based on these input signals.
Transmission video read clock SVRCK and transmission audio clock S synchronized with the generation timing of CK and PES
ACK, transmission video clock SVCK, transmission decoding time stamp SDTS, transmission presentation time stamp SPTS and transmission frame synchronization signal /
The transmission bit rate allocation signal SFAS / SBAS is generated and output.

SACK is an SACK when RAF is "1".
It was generated by drawing out TMCK.
SVCK is generated by rendering STMCK when RVF is "1". This allows
H. Audio and video can be separated from the 221 data and written to the respective buffers 131 and 132.

Further, the timing control unit 136 uses the PES
Header, DTS, PTS, signal SPESH indicating the multiplexing position of the FAS / BAS packet, and SPCK in which the STCK clock exists only in the PES portion of STS, are output to the PES multiplexing unit 133, and the TS header and PCR multiplexing position are also determined. STSH and clock STCK are shown in the TS multiplexing section 1
To 34. Only the PES portion of the STS is a portion excluding the TS header including the adaptation header, and a portion excluding null packets for adjusting the transmission rate. The clock STCK is generated by dividing SSCK or by using a PLL.

Referring again to FIG. 4, the CLA unit 104 receives the transport stream ST from the packet multiplexing unit 130.
S and the transmission clock STCK are input, and the asynchronous network clock ATMCK is input to form an ATM cell, and an optical signal is transmitted through the asynchronous network transmission interface unit 105.
It is electrically converted and output to the asynchronous network 300. The main data and signals involved in the process of generating the transport transport stream STS from the above structured data are shown in FIG.

On the other hand, the gateway 100 uses the asynchronous network 30.
When the ATM cell is received from 0, the CLD unit 108
The optical-electrically converted ATM cells are input through the asynchronous network reception interface unit 107, and the asynchronous network clock ATMCK is input to decompose the ATM cells to generate and output the reception transport stream RTS and the reception transport clock RTCK. To do.

As shown in FIG. 6, the packet separation unit 190 has a TS separation unit 191, an STC reproduction unit 192, a PES separation unit 193, a timing control unit 194, an audio buffer 195 and a video buffer 196. ing. The TS separation unit 191 of the packet separation unit 190
Received transport stream RT from CLD unit 108
S and the reception transport clock RTCK are input, the reception packetized elementary stream RPES and the signal indicating the PES valid period are formed and output from the reception transport stream RTS, and the reception program clock reference RPCR is extracted and RP is output.
Output with CR load timing.

The PES separation unit 193 is the TS separation unit 191.
The received packetized elementary stream RPES is input and separated from the received audio elementary stream RAPES and the received video elementary stream RV.
PES, reception time stamp RDTS and reception frame synchronization signal / reception bit rate allocation signal RFAS /
Output RBAS.

The audio buffer 195 inputs the received audio packetized elementary stream RPES from the PES separation unit 193 in accordance with the received audio write clock RAWCK generated during the audio elementary stream period in PES, and sets the audio data transfer rate. Synchronized reception audio clock RACK
Receive audio elementary stream RAE according to
Output as S.

The video buffer 196 has a PES separation 1
The received video packetized elementary stream RVES from 93 is received video write clock RVWC generated during the video elementary stream during PES.
It is input according to K and is output as a received video elementary stream RAES according to a received video clock RVCK synchronized with the transfer rate of video data.

The STC reproducing unit 192 is the TS separating unit 191.
Receive program clock reference SPCR from R
It is input in synchronization with the PCR load timing to generate and output the reception system clock RSCK and the reception system time clock RSTC. Timing control unit 194
Is the reception system clock RS from the STC reproduction unit 192.
CK and the reception system time clock RSTC are input, and the reception time stamp RDTS and the reception frame synchronization signal / reception bit rate allocation signal RFAS / RBAS are synchronized with the timing signal RPESH indicating the separation output timing from the PES separation unit 193. Received, and a received audio write clock RAWCK that is output during the audio elementary stream during PES, and a received video write clock RV that is output during the video elementary stream during PES.
WCK, reception audio clock RACK, reception video clock RVCK, audio flag RAF, video flag RVF and sub-multiframe synchronization signal RSMF
Generate and output SY.

Referring again to FIG. 4, the structured data multiplexing unit 1
Reference numeral 10 denotes a reception audio elementary stream RAES, a reception video elementary stream RVES, and a sub-multiframe synchronization signal R from the packet separation unit 190.
Input SMFSY, and synchronize network clock STM
CK is input, the data from the packet separation unit 190 is multiplexed based on the FAS / BAS data RFB to form structured data, and the structured data is output to the synchronous network 200 via the synchronous network transmission interface. On the other hand, the structured data multiplexing unit 110 uses the H.264 standard. 221 RAF and RVF indicating the multiplexing position
And the clock STMCK to the packet separation unit 190.

The above-mentioned reception transport stream RT
The main data and signals involved in the process of generating structured data from S are shown in FIG. The asynchronous network interface unit 700 has an ATM reception interface unit 710 and a CLD unit 720, as shown in FIG. The ATM receiving interface unit 710 is used for the asynchronous network 3
The ATM cell is received from 00, and is optically-electrically converted and output together with the asynchronous network clock ATMCK. CLD unit 72
0 inputs the ATM cell and the asynchronous network clock ATMCK from the ATM receiving interface unit 710,
Disassemble M cells and receive transport stream RT
S and a reception transport clock RTCK are generated and output.

As shown in FIG. 7, the terminal 600 has a TS separating section 611, an STC reproducing section 612, and a PES separating section 61.
3, timing control unit 614, audio buffer 6
15, video buffer 616, audio decoder 6
17 and a video decoder 618. The TS separation unit 611 receives the transport stream RTS and the reception clock RT from the asynchronous network interface unit 700.
Input CK to receive transport stream RTS
Separated and received packetized elementary stream RP
The ES is generated and output, and the reception program clock reference RPCR is generated and output.

The PES separation unit 613 is the TS separation unit 611.
The received packetized elementary stream RPES is input and separated from the received audio packetized elementary stream RAPES, the received video packetized elementary stream RVPES, and the received time stamp RD.
Output timing RPE of TS and reception frame synchronization signal / reception bit rate allocation signal RFAS / RBAS
Output in synchronization with SH.

The audio buffer 615 inputs the received audio packetized elementary stream RAPES from the PES separation unit 613 according to the received audio write clock RAWCK and outputs it according to the received audio clock RACK. Video buffer 616
Inputs the received video packetized elementary stream RVPES from the PES separation unit 613 according to the received video write clock RVWCK and outputs it according to the received video clock RVCK. Each of RACK and RVCK is a signal locked to each transfer rate of audio data and video data.

The audio decoder 617 inputs the reception audio elemental system RAES from the audio buffer 615 and receives the reception audio clock RAC.
Decrypt based on K. The video decoder 618 inputs the received video elementary stream RVES from the video buffer 616 and receives the received video clock RVCK.
Video elementary streams RVES based on
To decrypt.

The STC reproducing section 612 is a TS separating section 611.
The receiving program clock reference RPCR is input from, the RPCR is loaded into the STC by the RPCL, and the receiving system time clock RSTC and the receiving system clock RSCK are generated and output. The timing control unit 614 uses the clock RTC from the CLA unit.
K, the reception system time clock RSCK from the STC reproduction unit 612, the reception time stamps RDTS and RPTS, the reception frame synchronization signal / reception bit rate allocation signal RFAS / RBAS from the PES separation unit 613, and the audio elementary stream in the PES. Receive audio write clock RAW output during the period
Generates and outputs the received video write clock RVWCK output during the video elementary stream during CK and PES, the received audio clock RACK synchronized with the transfer rate of audio data, and the received video clock RVCK synchronized with the transfer rate of video data. To do.

The main data and signals involved in the process of generating an audio elementary stream from the above audio packetized elementary stream APES are shown in FIG. As shown in FIG. 8, the terminal 600 further includes an audio encoder 651, a video encoder 652, an audio buffer 653, a video buffer 654, a timing control unit 655, a PES multiplexing unit 656, an STC generation unit 657, a selector 658 and a T.
It has an S multiplexing unit 659.

The audio encoder 651 encodes the audio data based on the transmission audio clock SACK and transmits the transmission audio elementary stream SA.
Output as ES, the video encoder 652 encodes the video data based on the transmission video clock SVCK and outputs it as the transmission video elementary stream SVES.

The audio buffer 653 transmits the transmission audio elementary stream SAES from the audio encoder 651 to the transmission audio clock SACK.
According to the transmission audio read clock SARCK output during the audio elementary stream period during PES, and outputs as a transmission audio packetized elementary stream SAPES.

The video buffer 654 is a transmission video elementary stream S from the video encoder 652.
VES is input according to the transmission video clock SVCK, and transmission video packetized elementary stream SV is transmitted according to the transmission video read clock SVRCK output during the video elementary stream period in PES.
Output as PES.

The PES multiplexing unit 656 inputs the transmission audio elementary stream SAPES and the transmission video elementary stream SVPES from the audio buffer 653 and the video buffer 654, respectively, and the timing control unit 655 transmits the transmission time stamp SDTS and the transmission time stamp SDTS. A frame synchronization signal / transmission bit rate allocation signal SFAS / SBAS is input, and these are packetized and multiplexed to output a transmission packetized elementary stream SPES.

The TS multiplexing unit 659 has a timing control unit 6
P in synchronization with the multiplexing timing signal STSH from 55.
The transmission packetized elementary stream SPES from the ES multiplexing unit 656 and the transmission program clock reference SPCR from the STC generation unit 657 are input, these are packetized into 188 bytes and multiplexed to generate and output a transmission transport stream STS. , And generates and outputs the transmission clock STCK.

The STC generator 657 receives the clock from the selector 658 and outputs the transmission system clock SSC.
Generate and output K and the transmission system time counter value. The selector 658 receives the reception system clock RSCK when the destination terminal is a terminal on the synchronous network.
Is selected and output to the STC generation unit 657. When the destination terminal of the transmission destination is a terminal on the asynchronous network, the transmission clock Fs set in the terminal 600 is selected and output to the STC generation unit 657. .

The timing control unit 655 receives the transmission system clock SSCK and the transmission system time clock SSTC from the STC generation unit 657, and transmits the transmission time stamp SDTS, the transmission frame synchronization signal / transmission bit rate allocation signal SFAS / SBAS, PES. Output audio write clock SAWCK which is output during the audio elementary stream period, a transmit video write clock SVWCK which is output during the video elementary stream period during PES, the transmit audio clock SACK and the transmit video clock SVCK.

The asynchronous network interface unit 700 described above.
Further includes CLA units 730 and A as shown in FIG.
It has a TM transmission interface unit 740. CLA
The unit 730 receives the transmission transport stream STS and the transmission clock ST from the TS multiplexing unit 659 of the terminal 600.
Asynchronous network clock ATMCK while inputting CK
To form and output an ATM cell. The ATM transmission transmission interface 740 is used by the CLA unit 730.
A TM cell is input, converted into a format specified by the network, converted into an electrical signal, and output to the asynchronous network 300.

The main data and signals involved in the process of generating an audio packetized elementary stream from the above audio elementary stream AES are:
It is shown in FIG. Here, from the terminal 400 to the terminal 600
The operation of multiplexing and transmitting audio data and video data will be described below.

Structured data addressed to the terminal 600 by the gateway 100, the H.264 data shown in FIG. When 221 data is received from the terminal 400 via the synchronous network interface unit 500 and the synchronous network 200, the received structured data passes through the synchronous network receiving interface unit 101 and the structured data separation unit 1
Sent to 02. The structured data demultiplexing unit 102 demultiplexes the structured data into an audio elementary stream and a video elementary stream, and outputs the demultiplexed data to the packet multiplexing unit 130.

The RHD output from the structured data demultiplexing unit 102 is separated by RAF, RVF, and RFBF indicating the multiplexing position of each elementary stream, and the signals are input according to the transmission audio clock SAWCK and the transmission video clock SVCK, respectively. It is stored in the audio buffer 131 and the video buffer 132. Each of the audio elementary stream and the transmission video elementary stream accumulated in each buffer has a transmission audio read clock SARCK and a transmission video read clock SVRC.
Packetized elementary stream SA according to K
It is sent to the PES multiplexer 133 as a PES and a transmission video packetized elementary stream.

In the PES multiplexer 133, the input data is packetized and multiplexed in a size corresponding to the frame unit of the structured data, and the transmission time stamp DTS is added to each packet as shown in FIG. The TS multiplexing unit 13 as a transmission packetized elementary stream
Sent to 4. In the TS multiplexer 134, the input data is further packetized into 188 bytes, and the transmission presentation clock reference PCR is added to the head audio packet of each transport stream STS as shown in FIG. The structure of audio packets of the transport stream is shown in FIG. 14, and the structure of video packets is shown in FIG. As can be seen from FIGS. 14 and 15, PCR is added to the adaptation field of the audio packet in this example, but it may be added to the adaptation field of the video packet.

The transmission transport stream STS formed by the packet multiplexing unit 130 is AT-transmitted by the CLA unit 104.
Asynchronous network transmission interface unit 10 converted into M cells
5 to the asynchronous network 300. In the terminal 600, the ATM cell is received via the asynchronous network interface unit 700, and the TS separation unit 611 separates the reception program clock reference RPCR and the reception packetized elementary stream RPES. The separated reception program clock reference RPCR, together with the load timing RPCL that indicates when the data of the multiplexed PCR becomes valid, the STC reproduction unit 61.
2, the reception system time clock STC synchronized with the clock of the terminal 400 of the other party is reproduced. On the other hand, received packetized elementary stream RPES
Together with RPESF indicating the data valid period of PES,
It is sent to the PES separation unit 613.

In the packet separation unit, as shown in FIG. 12, the received audio packetized elementary stream APES and the received video packetized elementary stream RVPES from which the headers have been removed are separated and the audio buffer 615 and the video buffer 61 are separated.
6 are accumulated respectively. The data stored in the audio buffer buffer 615 is sent to the audio decoder 617 in accordance with the timing of the reception time stamp RDTS to be decoded and reproduced, as shown in FIG. Similarly, the data accumulated in the video buffer 616 is also sent to the video decoder 618 in synchronization with the timing of the reception time stamp DTS to be decoded and reproduced. Further, the STC reproducing unit 612 and the timing control unit 614
From the reception system clock RSCK indicating the timing of packetization and the sub-multiframe synchronization signal SMF
Sync is the selector 65 of the terminal 600 shown in FIG.
8 and the timing control unit 655, respectively.

Next, the operation when the audio data and the video data are multiplexed and transmitted from the terminal 600 to the terminal 400 will be described. Transmission audio elementary stream S generated by each of audio encoder 651 and video encoder 652 shown in FIG.
AES and transmitted video elementary stream SVE
The S is stored in the audio buffer 653 and the video buffer 654. Audio buffer 653
Of the transmission audio write clock SAWCK and the transmission video writing clock SWCK.
It is sent to the PES multiplexer 656 according to each VWCK, and packetized and multiplexed. At this time, FIG.
As shown in, the PES multiplexing unit 656 adds the transmission time stamp SDTS to the header at this time.

The transmission packetized elementary stream SPES formed by the PES multiplexing unit 656 is the TS multiplexing unit 6
It is sent to 59, is further packetized, a transmission presentation clock reference SPCR is added, a transmission transport stream is generated, and it is sent to the asynchronous network interface unit 700. In the asynchronous network interface unit 700, ATM cells are formed from the transmission transport stream and transmitted to the asynchronous network 300.

Next, the CLD unit 1 of the gateway 100
At 08, the ATM cell received through the asynchronous network receiving interface unit 107 is separated and sent to the packet separating unit 190. The TS separation unit of the packet separation unit 190 extracts the reception presentation clock reference RPCR and the payload data representing the multiplexed audio data and video data and sends them to the STC reproduction unit 192 and the PES separation unit 193, respectively. The PES separation unit 193 separates the audio data, the video data, and the reception time stamp RDTS from the payload data. The PES headers of the audio data and the video data are removed, and the audio buffer 195 and the video buffer 196 are removed.
Accumulated in each. Then, the reception time stamp R
Data is read from the audio buffer 195 and the video buffer 196 according to the DTS and is sent to the structured data multiplexing unit 110 as a received audio elementary stream RAES and a received video elementary stream RVES. Also, an SMF synchronization signal is generated from the reception time stamp RDTS and is sent to the structured data multiplexing unit 110. Furthermore, the packet separation unit 1
Reference numeral 90 denotes RAES according to the audio flag RAF and the video flag RFV from the structured data multiplexing unit 110.
And RAES are output respectively. The reception frame synchronization signal / reception bit rate allocation signal RFB is also sent to the structured data multiplexing unit 110.

In the structured data multiplexer 110, as shown in FIG. 11, the audio flag RAF and the video flag RVF are output to the packet separator 190 from the sub-multiframe synchronization signals RSMFY, RFB and STMCK, while the audio data is outputted. And video data is multiplexed to form structured data, which is sent to the terminal 400 through the synchronous network transmission interface unit 111, the synchronous network 200, and the synchronous network interface unit 500.

[0108]

According to the first aspect of the invention, the size of the packet in the elementary stream is set based on the frame unit of the structured data, and the decoding / reproduction of each packet in the elementary stream is performed. A time stamp indicating the time is added. Therefore, the synchronization state of the multiplexed data of the structured data can be represented by a time stamp, and the synchronization between the packet multiplexed data can be easily achieved at the time of decoding and reproducing.

According to the invention described in claim 2, in the method of mutual communication of multiplexed data according to claim 1, when the multiplexed data is transmitted from the terminal on the synchronous network to the terminal on the asynchronous network, the time stamp is used. The elementary stream is decoded and reproduced. Therefore, the terminal on the asynchronous network
Functions that handle structured data, such as H.264. It is not necessary to have 221 and the device configuration of the terminal can be simplified.

According to the invention described in claim 3, in the multiplexed data mutual communication method according to claim 1, the frame structure information representing the frame structure of the structured data is added to the head packet in the elementary stream. , The time stamp of the packet to which the frame structure information is added is synchronized with the time stamp of each packet in the elementary stream following the packet to which the frame structure information is added, and the structure of the frame corresponding to each packet is It is changed based on the frame structure information. Therefore, it is possible to change the frame structure at an accurate time.

According to the fourth aspect of the present invention, in the multiplexed data mutual communication method according to the first aspect, when a terminal on the asynchronous network receives data from a terminal on the synchronous network, each terminal is based on a time stamp. The packetization timing of the packet is reproduced. When the terminal on the asynchronous network transmits data to the terminal on the synchronous network, the amount of data in the elementary stream between the packetization timings reproduced in the packetization timing reproduction step is based on the frame structure information. It is determined. Therefore, it is possible to accurately reproduce the frame unit of the structured data.

According to the invention of claim 5, in the multiplexed data mutual communication method of claim 1, when the terminal on the asynchronous network receives the data from the terminal on the synchronous network, the data is received based on the time stamp. When the clock is regenerated and the terminal on the asynchronous network sends data to the terminal on the synchronous network,
A transmission clock is generated based on the reception clock regenerated in the reception clock regeneration step. Therefore, the terminal on the asynchronous network can transmit in synchronization with the clock of the terminal on the synchronous network.

According to the invention of claim 6, in the multiplexed data mutual communication method of claim 1, the structured data is H.264 of ITU-T. 221 standard and the elementary stream is ISO / IEC 11172.
-1 or 13818-1 standard. Therefore, for example, a video conference can be realized between the terminals on the synchronous network and the terminals on the asynchronous network.

According to the seventh aspect of the invention, the size of the packet in the elementary stream is set based on the frame unit of the structured data, and the decoding / playback time is set for each packet in the elementary stream. The time stamp shown is added. Therefore, the synchronization state of the multiplexed data of the structured data can be represented by a time stamp, and the synchronization between the packet multiplexed data can be easily achieved at the time of decoding and reproducing.

According to the invention described in claim 8, in the multiplexed data intercommunication system according to claim 7, when multiplexed data is transmitted from a terminal on a synchronous network to a terminal on an asynchronous network, based on a time stamp. The elementary stream is decoded and reproduced. Therefore, a terminal on the asynchronous network has a function of handling structured data, for example, H.264. It is not necessary to have 221 and the device configuration of the terminal can be simplified.

According to the invention described in claim 9, in the multiplexed data mutual communication system according to claim 7, the frame structure information representing the frame structure of the structured data is added to the head packet in the elementary stream. , The time stamp of the packet to which the frame structure information is added is synchronized with the time stamp of each packet in the elementary stream following the packet to which the frame structure information is added, and the structure of the frame corresponding to each packet is It is changed based on the frame structure information. Therefore, it is possible to change the frame structure at an accurate time.

According to the invention of claim 10, claim 7 is provided.
In the described multiplexed data mutual communication system, when the terminal on the asynchronous network receives the data from the terminal on the synchronous network, the packetization timing of each packet is reproduced based on the time stamp. When the terminal on the asynchronous network transmits data to the terminal on the synchronous network, the amount of data in the elementary stream between the packetization timings reproduced in the packetization timing reproduction step is based on the frame structure information. It is determined. Therefore, it is possible to accurately reproduce the frame unit of the structured data.

According to the invention of claim 11, claim 7 is provided.
In the described multiplexed data mutual communication system, when a terminal on an asynchronous network receives data from a terminal on a synchronous network, a reception clock is reproduced based on a time stamp, and the terminal on the asynchronous network is a terminal on the synchronous network. When data is transmitted to, the transmission clock is generated based on the reception clock regenerated in the reception clock regeneration step. Therefore, the terminal on the asynchronous network can transmit in synchronization with the clock of the terminal on the synchronous network.

According to the invention of claim 12, claim 7 is provided.
In the described multiplexed data intercommunication system, the structured data is ITU-T H.264 standard. 221 standard, the elementary stream is ISO / IEC 1
Formed according to the 1172-1 or 13818-1 standards. Therefore, for example, a video conference can be realized between the terminals on the synchronous network and the terminals on the asynchronous network.

According to the thirteenth aspect of the present invention, the size of the packet in the elementary stream is set based on the frame unit of the structured data, and the decoding / playback time is set for each packet in the elementary stream. The time stamp shown is added. Therefore, the synchronization state of the multiplexed data of the structured data can be represented by a time stamp, and the synchronization between the packet multiplexed data can be easily achieved at the time of decoding and reproducing.

According to the invention of claim 14, claim 1
In the gateway described in 3, the frame structure information indicating the frame structure of the structured data is added to the first packet in the elementary stream, and the time stamp of the packet to which the frame structure information is added and the frame structure information are added. Synchronize with the time stamp of each packet in the elementary stream following the
The structure of the frame corresponding to each packet is changed based on the frame structure information. Therefore, it is possible to change the frame structure at an accurate time.

According to the invention of claim 15, claim 1
In the gateway described in 3, structured data is ITU
-T H. 221 standard, and the elementary stream is formed according to the ISO / IEC 13818-1 standard. Therefore, for example, a video conference can be realized between the terminals on the synchronous network and the terminals on the asynchronous network.

According to the sixteenth aspect of the present invention, the size of the packet in the elementary stream is set based on the unit of the frame of the structured data, and the decoding / playback time is set for each packet in the elementary stream. In the communication system with the time stamps shown, when multiplexed data is transmitted from a terminal on a synchronous network to a terminal on an asynchronous network, an elementary stream is decoded and reproduced based on the time stamp. Therefore, a terminal on the asynchronous network has a function of handling structured data, for example, H.264. It is not necessary to have 221 and the device configuration of the terminal can be simplified.

According to the invention of claim 17, claim 1
In the terminal on the asynchronous network described in No. 6, when the terminal on the asynchronous network receives the data from the terminal on the synchronous network, the packetization timing of each packet is reproduced based on the time stamp. When the terminal on the asynchronous network transmits data to the terminal on the synchronous network, the amount of data in the elementary stream between the packetization timings reproduced in the packetization timing reproduction step is based on the frame structure information. It is determined. Therefore, it is possible to accurately reproduce the frame unit of the structured data.

According to the invention of claim 18, claim 1
In the terminal on the asynchronous network described in 6, when the terminal on the asynchronous network receives data from the terminal on the synchronous network, the reception clock is reproduced based on the time stamp, and the terminal on the asynchronous network is the terminal on the synchronous network. When data is transmitted to, the transmission clock is generated based on the reception clock regenerated in the reception clock regeneration step. Therefore, the terminal on the asynchronous network can transmit in synchronization with the clock of the terminal on the synchronous network.

According to the invention of claim 19, claim 1
In the multiplex data intercommunication system according to claim 6, the structured data is ITU-T H.264. 221 standard, the elementary stream is ISO / IEC 1
It is formed according to the 3818-1 standard. Therefore, for example, a video conference can be realized between the terminals on the synchronous network and the terminals on the asynchronous network.

[Brief description of drawings]

FIG. 1 is a diagram showing an embodiment of a multiplexed data mutual communication system according to the present invention.

FIG. 2 is a diagram showing an example of a frame structure of structured data handled on the synchronous network shown in FIG.

FIG. 3 is a diagram showing an example of a packet configuration of an elementary stream handled on the asynchronous network shown in FIG.

FIG. 4 is a block diagram showing a configuration of the gateway shown in FIG.

5 is a block diagram showing a configuration of a packet multiplexing unit shown in FIG.

6 is a block diagram showing a configuration of a packet separation unit shown in FIG.

7 is a block diagram showing a configuration of a receiving unit side of a terminal on an asynchronous network interface unit and an asynchronous network shown in FIG.

8 is a block diagram showing a configuration of a transmission unit side of a terminal on an asynchronous network interface unit and an asynchronous network shown in FIG.

FIG. 9 is a timing chart for explaining the demultiplexing / multiplexing operation of the structured data shown in FIG.

10 is a timing chart illustrating a packet multiplexing operation of the gateway shown in FIG.

11 is a timing chart illustrating a packet separating operation of the gateway shown in FIG.

12 is a timing chart illustrating a packet separating operation of a terminal on the asynchronous network shown in FIG.

13 is a timing chart illustrating a packet multiplexing operation of the terminal on the asynchronous network shown in FIG.

14 is a diagram showing the structure of the audio packet shown in FIG.

15 is a diagram showing the structure of the video packet shown in FIG.

FIG. 16 is a diagram showing a connection state of various conventional telecommunication networks.

FIG. 17 is a diagram illustrating a conventional data transfer method of a synchronous network.

FIG. 18 is a diagram illustrating a conventional asynchronous network data transfer system.

FIG. 19 shows a conventional H.264. It is a block diagram which shows the structure of a 320 terminal.

FIG. 20 is a diagram illustrating a conventional B channel frame structure.

FIG. 21 is a diagram illustrating a frame structure of a conventional H0, H11 or H12 channel.

FIG. 22 is a diagram showing a frame structure of a conventional multi-frame synchronization signal.

FIG. 23 shows a conventional H.264. 22 is a diagram showing a frame configuration and a transmission order in 221. FIG.

FIG. 24 shows a conventional H.264. FIG. 221 is a diagram for explaining an n × B channel multiplexing rule in 221;

FIG. 25 shows a conventional H.264. FIG. 22 is a diagram illustrating a H0 channel multiplexing rule in H.221.

FIG. 26 is a diagram showing the structure of a packetized elementary stream in a conventional MPEG system.

[Fig. 27] Fig. 27 is a diagram illustrating the configurations of a transport stream and transport packets in a conventional MPEG2 system.

[Fig. 28] Fig. 28 is a diagram for describing the mapping situation of PES packets to transport packets in a conventional MPEG2 system.

[Fig. 29] Fig. 29 is a diagram for describing the mapping situation of TS packets to ATM cells in the conventional fixed rate video on demand service.

[Explanation of symbols]

 100 Gateway 101 Synchronous Network Reception Interface Unit 102 Structured Data Separation Unit 104 CLA Unit 105 Asynchronous Network Transmission Interface Unit 105 107 Asynchronous Network Reception Interface Unit 107 108 CLD Unit 110 Structured Data Multiplexing Unit 111 Synchronous Network Transmission Interface Unit 130 Packet Multiplexing Unit 131 Audio Buffer 132 Video Buffer 133 PES Multiplexing Unit 134 TS Multiplexing Unit 135 STC Generating Unit 136 Timing Control Unit 190 Packet Separation Unit 191 TS Separation Unit 192 STC Reproducing Unit 193 PES Separation Unit 194 Timing Control Unit 194 195 Audio Buffer 196 Video buffer 200 Synchronous network 300 Asynchronous network 400 Terminal on synchronous network 500 Synchronous network interface unit 600 Terminal on asynchronous network 61 TS separation unit 612 STC reproduction unit 613 PES separation unit 614 Timing control unit 615 Audio buffer 616 Video buffer 617 Audio decoder 618 Video decoder 651 Audio encoder 652 Video encoder 653 Audio buffer 654 Video buffer 655 Timing control unit 656 PES multiplexing Part 657 STC generation part 658 Selector 659 TS multiplexing part 700 Asynchronous network interface part 710 ATM reception interface part 720 CLD part 730 CLA part 740 ATM transmission interface part

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location H04N 7/081 H04L 13/00 305B H04Q 3/00 H04N 7/08 Z

Claims (19)

[Claims] 1. A terminal on a synchronous network for handling structured data structured by a plurality of frames of a fixed size, and an elementary formed by packetizing and multiplexing the data and having time information added thereto. In a multiplexed data mutual communication method that enables mutual communication of multiplexed data with a terminal on an asynchronous network that handles a stream, the size of a packet in the elementary stream is set based on the unit of the frame. And a time stamp adding step of adding a time stamp indicating the time of decoding and reproduction to each packet in the elementary stream. 2. The method for mutual communication of multiplexed data according to claim 1, wherein when the multiplexed data is transmitted from a terminal on the synchronous network to a terminal on an asynchronous network, an elementary stream is decoded based on the time stamp. The method of claim 1, further comprising a decoding / reproducing step of reproducing. 3. The multiplexed data intercommunication method according to claim 1, further comprising: frame structure information adding step of adding frame structure information representing a frame structure of the structured data to a head packet in an elementary stream. By synchronizing the time stamp of the packet to which the frame structure information is added and the time stamp of each packet in the elementary stream following the packet to which the frame structure information is added, the structure of the frame corresponding to each packet is described. And a frame structure changing step of changing the frame structure information based on the frame structure information. 4. The multiplexed data mutual communication method according to claim 1, wherein when a terminal on an asynchronous network receives data from a terminal on a synchronous network, the packetization timing of each packet is reproduced based on the time stamp. And a packetizing timing reproducing step to perform, when the terminal on the asynchronous network transmits data to the terminal on the synchronous network, the amount of data in the elementary stream between the packetizing timings reproduced in the packetizing timing reproducing step, And a data amount determining step for determining the data amount based on the frame structure information. 5. The multiplexed data mutual communication method according to claim 1, wherein when a terminal on an asynchronous network receives data from a terminal on a synchronous network, a received clock recovery is performed based on the time stamp. And a transmission clock generation step of generating a transmission clock based on the reception clock reproduced in the reception clock reproduction step when a terminal on the asynchronous network transmits data to the terminal on the synchronous network. A method for mutual communication of multiplexed data. 6. The method according to claim 1, wherein the structured data is ITU-T (International Teleco).
mmunication Union -Telecommunication Standardizati
on Sector). 221 standard, and the elementary stream is an ISO / IEC (Internatio)
nal Organization for standardization / International
A method of multiplexed data intercommunication, characterized in that it is formed according to the 11172-1 or 13818-1 standard of the Electrotechnical Commission. 7. A terminal on a synchronous network that handles structured data structured by a plurality of frames of a fixed size, and an elementary formed by packetizing and multiplexing the data and having time information added thereto. In a multiplexed data mutual communication system that enables mutual communication of multiplexed data between terminals on an asynchronous network that handles streams, a packet size in the elementary stream is set based on the unit of the frame. And a time stamp adding means for adding a time stamp indicating the time of decoding and reproduction to each packet in the elementary stream. 8. The multiplexed data mutual communication system according to claim 7, wherein when multiplexed data is transmitted from a terminal on the synchronous network to a terminal on an asynchronous network, an elementary stream is decoded based on the time stamp. 2. The multiplexed data mutual communication system according to claim 1, further comprising decoding / reproducing means for reproducing. 9. The multiplexed data intercommunication system according to claim 7, further comprising frame structure information adding means for adding frame structure information representing a frame structure of the structured data to a head packet in an elementary stream. The time stamp of the packet to which the frame structure information is added and the time stamp of each packet in the elementary stream following the packet to which the frame structure information is added are synchronized, and the structure of the frame corresponding to each packet is described above. And a frame structure changing means for changing the frame structure information based on the frame structure information. 10. The multiplexed data intercommunication system according to claim 7, wherein when a terminal on an asynchronous network receives data from a terminal on a synchronous network, the packetization timing of each packet is reproduced based on the time stamp. And a packetizing timing reproducing means for transmitting data to a terminal on a synchronous network when the terminal on the asynchronous network transmits the data amount in the elementary stream between the packetizing timings reproduced by the packetizing timing reproducing means, And a data amount determining means for determining the data amount based on the frame structure information. 11. The multiplexed data mutual communication system according to claim 7, wherein when a terminal on an asynchronous network receives data from a terminal on a synchronous network, a received clock recovery is performed based on the time stamp. Means and a transmission clock generation means for generating a transmission clock based on the reception clock reproduced by the reception clock reproduction means when the terminal on the asynchronous network transmits data to the terminal on the synchronous network. And a multiplexed data intercommunication system. 12. The multiplexed data interconnection system according to claim 7, wherein the structured data is ITU-T (International Teleco).
mmunication Union -Telecommunication Standardizati
on Sector). 221 standard, and the elementary stream is an ISO / IEC (Internatio)
nal Organization for standardization / International
Multiplexed data intercommunication system formed according to the Electrotechnical Commission) 11172-1 or 13818-1 standard. 13. A gateway provided between a synchronous network and an asynchronous network, a terminal on a synchronous network that handles structured data structured by a plurality of frames of a fixed size, and data packetization and multiplexing. A gateway that enables mutual communication of multiplexed data between a terminal on an asynchronous network that handles elementary streams that are formed by adding time information and that is based on the unit of the frame. A packet size setting means for setting a size of a packet in the elementary stream, and a time stamp adding means for adding a time stamp indicating a decoding reproduction time to each packet in the elementary stream. A characteristic gateway. 14. The gateway according to claim 13, wherein frame structure information adding means for adding frame structure information representing a frame structure of the structured data to a head packet in an elementary stream, and the frame structure information. The time stamp of the added packet is synchronized with the time stamp of each packet in the elementary stream following the packet to which the frame structure information is added, and the structure of the frame corresponding to each packet is based on the frame structure information. And a frame structure changing means for changing the structure. 15. The gateway according to claim 13, wherein the structured data is ITU-T (International Teleco
mmunication Union -Telecommunication Standardizati
on Sector). 221 standard, and the elementary stream is an ISO / IEC (Internatio)
nal Organization for standardization / International
A gateway characterized in that it is formed in accordance with the Electrotechnical Commission) 11172-1 or 13818-1 standards. 16. A synchronous network for handling structured data structured by a plurality of frames of a fixed size, and an elementary stream formed by packetizing and multiplexing data and having time information added thereto. Packet size setting means for setting the size of the packet in the elementary stream based on the unit of the frame to enable mutual communication of multiplexed data through an asynchronous network, and each packet in the elementary stream. A time stamp adding means for adding a time stamp indicating the time of decoding and reproduction, which is a terminal on an asynchronous network applied to a multiplexed data intercommunication system, wherein the multiplexed data is transmitted from the terminal on the synchronous network. When you receive
A terminal on an asynchronous network, comprising a decoding / reproducing means for decoding / reproducing an elementary stream based on the time stamp. 17. A terminal on an asynchronous network according to claim 16, wherein when data is received from a terminal on a synchronous network, each terminal is based on a time stamp indicating the time of decoding and reproduction included in the elementary stream. Packetizing timing reproducing means for reproducing the packetizing timing of the packet and the amount of data in the elementary stream between the packetizing timings reproduced by the packetizing timing reproducing means when transmitting data to the terminal on the synchronous network. A data amount determining means for determining the data amount based on the frame structure information. 18. A terminal on an asynchronous network according to claim 16, wherein when data is received from a terminal on a synchronous network, a reception clock reproducing means for reproducing a reception clock based on the time stamp; A terminal on an asynchronous network, comprising: a transmission clock generation means for generating a transmission clock based on the reception clock reproduced by the reception clock reproduction means when transmitting data to the terminal. 19. The terminal on the asynchronous network according to claim 16, wherein the structured data is ITU-T (International Teleco
mmunication Union -Telecommunication Standardizati
on Sector). 221 standard, and the elementary stream is an ISO / IEC (Internatio)
nal Organization for standardization / International
A terminal on an asynchronous network formed according to the Electrotechnical Commission's 11172-1 or 13818-1 standard.