JP2001189708A - Multimedia information device and its transmitting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress the transmission delay of voice even while suppressing the fall of the multiplexing efficiency of a packet including the video and voice of video telephone. SOLUTION: This multimedia information device which is connected to a line exchange network, multiplexes a header in which multiplexing code information showing a multiplexing pattern and the media information of a transmission object according to the multiplexing pattern corresponding to the multiplexing code information while 1st media information AD requested to be transmitted at a constant period and 2nd media information VD whose multiplexed packet has to be completed in a frame unit are defined at least as transmission objects and transmits the header and the media information as the packet of a format stored in a payload, is provided with temporarily holding means (FIFO) 122 and 132 temporarily hold the AD and BD and a multiplexing means 143 which fetches and multiplexes the media information from the FIFOs, also holds the transmission of the other media information and finishes packet transmission when the AD is generated and processes packet transmission of the 1st media information before a packet to be next transmitted.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a terminal that handles multimedia information, multiplexes the multimedia information, communicates via a transmission line, and separates and outputs the information received via the transmission line into multimedia information. The present invention relates to a multimedia information device and a transmission method thereof.

[0002]

2. Description of the Related Art Multimedia information communication includes audio information (audio information), video information (video information), and the like, and real-time properties are important. That is, when the video and the audio are related and it is necessary to maintain a synchronous relationship, if the synchronization is broken, the receiving side cannot easily grasp the content.

Therefore, in order to prevent such a situation, a real-time transmission technique of audio information (audio information) and video information (video information) in multimedia information communication is essential.

[0004] On the other hand, conventional protocols in data communication have focused on transmitting information correctly. Therefore, in order to handle multimedia information, a condition that information is transmitted within a prescribed time is added in order to ensure real-time performance.

[0005] The multimedia information includes audio information, video information, data information, and control information, and is composed of these and various communication infrastructures. There are various protocols of interest.

Incidentally, "H.324" standardized by ITU-T as one of the protocols to be used for multimedia information communication specifies a low bit rate multimedia telephone terminal in GSTN (general switched telephone network). are doing. That is, this is a videophone standard. FIG. 6 shows a functional configuration diagram of a main part of a system adopting “H.324”, and FIG. 5 shows a network configuration example using this system.

In FIG. 5, reference numeral 590 denotes a GSTN (general switched telephone network: public network), and reference numerals 500 denote multimedia information terminals, which are connected to each other via the GSTN 590 to enable communication. Here, the multimedia information terminal 500 is, for example, a videophone, and includes a camera 500a for capturing an image of a speaker and a monitor 500 for displaying an image.
b, microphone 500c for inputting voice, speaker 500d for outputting voice, data application 500 which is a functional element for handling data
e, a system control unit 500f for controlling various components of the multimedia information terminal 500, and a signaling protocol unit 500 for transferring various signal signals in the telephone network and performing call control.
g, a control protocol unit 540 which is a protocol for transmitting and receiving a control signal in a telephone network; a video codec unit 510 which encodes a video signal and decodes the coded video signal and returns it to the original video signal , An audio codec 52 that encodes the audio signal and decodes the encoded audio signal and returns it to the original audio signal.
0, a data protocol unit 530 which is a protocol for data transfer, coded data and a signaling protocol unit 500g, a control protocol unit 540, and a data protocol unit 53 respectively supplied from the video codec 510 and the audio codec 520.
Various kinds of information given from 0 are multiplexed and output to the modem as a multiplexed signal (digital signal), and the digital multiplexed signal received from the modem is separated and returned to video, audio, signal, control signal and data. These are distributed to the video codec unit 510, the audio codec unit 520, the signaling protocol unit 500g, the control protocol unit 540, the data protocol unit 530, and the like corresponding to the type, and the multiplexing units 580 and 580 output the signals. (Digital signal) is modulated into an analog signal for a telephone network and output to GSTN 590,
It comprises a modem 580 and the like, which demodulates an analog signal received from the TN 590 into a digital signal and supplies the digital signal to the multiplexing unit 580.

The main part of the multimedia information terminal 500 will be described in more detail. Multimedia information terminal 500
6 is as shown in FIG. 6, and the video codec unit 510 is configured to encode / decode video information (video information) according to the standard of “H.263” or “H.261”. A video codec 51
0 encodes a video stream that is an image data sequence captured and input by the camera 500a as an imaging device, outputs the video stream to the multiplexing unit 570, and decodes information input from the multiplexing unit 570. A video stream is generated and output to a monitor (monitor display) 500b.

[0009] The voice codec section 520 has "G.723".
The audio information (audio signal) input from the microphone 500c that receives audio and converts it into an audio signal is encoded and output to the multiplexing unit 570.
Also, it decodes the information input from the multiplexing unit 570 to generate audio information, and outputs the audio information to the speaker 500d.

A delay circuit 550 for delaying a signal is provided on the audio input path. By adjusting the delay time of the delay circuit 550, synchronization between video and audio can be maintained.

The data protocol section 530 supports a data application.

A control protocol unit (control protocol unit) 540 conforms to the standard of “H.245”.
It defines commands and procedures for exchanging capability information between terminals and opening logical channels. Control protocol unit 540
A data protocol 560 such as SRP or LAPM is interposed between the multiplexing unit 570 and the multiplexing unit 570.

The multiplexing section 570 conforms to the standard of “H.223”, multiplexes each information of audio information, video information, data information, and control information into a serial signal, and multiplexes each information from the serial signal. Separate.

The modem 580 interfaces the digital signal output from the multiplexing unit 570 to the analog signal of the GSTN 590.

Multiplexing section 570 includes multiplexing layer section 575 and adaptation layer sections 571, 572, 573, 5
74.

Multiplexing layer section 5 in multiplexing section 570
75 packet format 610 (MUX-PD
U) is shown in FIG. The MUX-PDU 610, which is a packet format in the multiplexing layer unit 575, includes a 1-byte header 630 and a 0-byte or more payload 640.
It is composed of Then, the HDLC flag 620 is used to indicate a packet delimiter. HDLC
To ensure that the flag 620 does not appear in the MUX-PDU 610, the MUX-PDU
Of the MUX-PDU 610, the receiving side deletes the "0" after the "1" appearing five consecutive times in the MUX-PDU 610.

The header 630 includes the multiplexed code area 63
1, header error control area 632, packet marker area 6
33. The multiplexed code area 632 has a 4-bit configuration. The multiplexed code area 632 applies any one of 16 types of decimal numbers “0” to “15” that can be represented by 4 bits. In order to inform whether to perform the transmission, an entry of a multiplexing pattern of a logical channel to which each byte of the payload 640 of the MUX-PDU 610 belongs is designated in an area used for embedding and transmitting the corresponding one.

The header error control area 631 has a 3-bit configuration, and is used for error detection of a multiplexed code area by 3-bit CRC.

The header error control area 631 has a generator polynomial x ³ + x + 1 for the product of the content of the multiplexed code and x ^3.
Includes the remainder divided by.

The packet marker area 633 of the header 630
Indicates that the last byte of MUX-PDU 610 is AL-P
This is an area for displaying “1” when it is the last byte of DU 710 or 910. The entry of the 0th multiplexing pattern is assigned to the control channel by default. The entries of the first to fifteenth multiplex patterns are exchanged between terminals according to a control protocol (based on the standard of “H.245”).

The adaptation layer includes AL1 573, 574, which is the first adaptation layer, and the second adaptation layer.
AL2 572, which is an adaptation layer unit of the first embodiment, and AL3 57 which is a third adaptation layer unit,
A total of three up to 1 is specified.

The first adaptation layer section A
L1 573 and 574 are used to transfer data protocol and control information. AL2 572, which is a second adaptation layer unit, is used to transfer information of an audio codec unit (audio codec unit) 520.

A which is a second adaptation layer unit
FIG. 8 shows the format of the packet 710 (AL2-PDL) of L2 572. As shown in FIG.
The DU 710 has a 1-byte sequence number area 720, 1
It is composed of a payload 730 of more than one byte and a CRC area 740 of one byte.

Of these, the sequence number area 720
Is an option, and when this option is used, it is used for detecting a missing AL2-PDU 710 or the like.

The payload 730 is an area for transmitting and receiving information, and the information of the audio codec section (voice codec section) 520 is transmitted and received using the payload 730. The CRC area 740 part is the sequence number area 7
20 (if used) and the remainder of the product of the payload 730 and x ⁸ divided by the generator polynomial x ⁸ + x ² + x + 1.

Next, the third adaptation layer section AL3 571 is used to transfer information of the video codec section 510. The packet 810 (AL) of the third adaptation layer unit AL3 571
3-PDU) format is shown in FIG.

AL3-PDU 810, which is the third adaptation layer packet, has a 15-bit sequence number field 820 and a 1-bit payload type field 8
30, a payload 840 of 1 byte or more, and a CRC area 850 of 2 bytes. Among these, the sequence number area 820 is optional, and is used for detecting a missing AL3-PDU 810 or the like.

The payload type area 830 is AL
3-PDU 810 is used as an information field indicating a payload type. And the payload type is "1"
In the case of, video codec 410
Is transmitted and received, and when the payload type is “0”, it indicates that a monitoring message is transmitted and received.

The CRC area 850 includes the content of the sequence number field 820 (if used) the product of the content and x ¹⁶ payload 840, the remainder of division by the generator polynomial ^{x 16 + x 12 + x 5} +1.

The payload 64 of the MUX-PDU 610 which is a packet format in the multiplexing layer unit 575
0 to AL-PDU 710 or 810 of each media
10 are multiplexed.

Since there are various types of multiplexing patterns, it is necessary to indicate which one is used, and an entry for that purpose is a multiplex entry descriptor (M
ultiplex Entry Descriptior).

For example, Multiplex Entry Descriptio
n is “{LCN1, RC4}, {{LCN2, RC1}, {LCN3, RC2}, RC UC
F}. "LCNn" is a logical channel number, which defines which table to use, and the table itself is defined by the user. "RCn" is a repeat counter, and how many bytes of data "RC UCF" is an untail closing flag, indicating that the process is repeated to the end.
“{LCN1, RC4}, {{LCN2, RC1}, {LCN3, RC2}, RC UCF}”
The first item, logical channel number 1 (LCN1), has a 4-byte structure, and logical channel number 2 (LCN2)
The content is 1 byte, and the logical channel number 3 (LCN3) indicates that the content is 2 bytes and this pattern is repeated.

Now, assume that a multiplexed packet as shown in FIG. 10 has been transmitted.

Here, among the multiplexed packets, the first multiplexed packet, MUX-PDU 960, has "{LCN1, RC4}, {{LCN2, RC1}, {LCN3, RC2}, RC UC" in its header.
F} ”and the second packet, MUX-PDU
In the header 961, a Multiplex Entry Description of “{LCN1, RC UCF}” is described in the header, but here, the logical channel number 1 “LCN1,
1 ”is voice, logical channel number 2“ LCN ”
2 ”is data, logical channel number 3“ LCN ”
3 "is assigned to each video.

Then, the voice packet AL2
It is assumed that 4 bytes of PDU 910, 3 bytes of AL1-PDU 920 which is a data packet, and 3 bytes of AL3-PDU 930 which are a packet of video are generated.

The MUX-PDU 960 has its header 95
0 to “{LCN1, RC4}, {{LCN2, RC1}, {LCN3, RC2}, RC UC
F} ”, and the 4
The 4-byte audio information packet AL2-PDU 910 is inserted into the LCN1 having a capacity of 1 byte, and the LCN having a capacity of 1 byte allocated for data is used.
2, a 3-byte data information packet AL1-PDU
920 is inserted (LCN2 in the figure).
921 corresponds to this), 2 bytes of the 3-byte video information packet AL3-PDU 930 are inserted into the 2-byte capacity LCN3 allocated for video (LCN3 931 in the figure is LCN3 931). Corresponding to this). Subsequently, one byte of the remaining two bytes of the 3-byte data information packet AL1-PDU 920 is inserted into the LCN2 having the capacity of 1 byte allocated for data (LCN2 in the figure).
922 corresponds to this), and the remaining 1 byte of the 3-byte video information packet AL3-PDU 930 is inserted into the LCN3 having a 2-byte capacity allocated for video (LCN3 932 in FIG. Corresponding)
However, the packet AL3-PDU 93 of the video information
At the time of insertion of the remaining 1 byte of 0, AL3-PDU 9
Since 30 reaches the last byte, the multiplexed packet MUX
-Close PDU 960.

At this point, transmission of audio and video has been completed, but for data, the remaining one byte of the generated 3-byte data information packet AL1-PDU 920 has not been transmitted yet. Therefore, to send this, the next multiplexed packet, MUX-PDU 961
Prepare Then, the multiplexed packet MUX-PD
The packet marker of the header 951 of U 961 is “1”
Indicates that the transmission is for the last byte of the data packet AL1-PDU 920, and
-The header 951 of the PDU 961 includes "{LCN1, RC UC
F} ", the last one byte is inserted into the LCN1 having a one-byte capacity allocated for data by describing the Multiplex Entry Description (in the figure, LCN2
923 corresponds to this). This completes the transmission of the last byte of the data packet AL1-PDU 920, so that the multiplexed packet MUX-PDU 961 is terminated, and the HDLC flag 943 is added last.

The voice codec section 520 according to the “G.723.1” standard periodically generates 20 audio signals at intervals of 30 [msec].
[byte] (5.3 Kbps) or 24 [byte] (6.3 Kbps)
bps) of the AL-SDU. FIG. 11 shows the situation when the speed of the transmission line provided by the modem 580 is considered to be 56 [Kbps].

For example, when a 5.3 [Kbps] speed is selected by a voice codec based on the "G.723.1" standard,
AL2 572 which is the second adaptation layer unit
AL2-PDU 1010 (when the sequence number option is not used) is 30 [mse
21 [byte] (audio stream + CRC) audio frames are generated at every audio frame interval of [c].

Therefore, assuming that the transmission speed is 56 [kbps], there is a period that can be allocated to transmission of other multimedia information for about 180 [bytes] between generated voice frames.

Therefore, it is possible to transmit a video stream for about 180 [bytes] during this period. When terminating the multiplexing of the video stream in video frame units, the variable length frame generated by the video codec unit 510 is 30 [msec].
180 [b] that can be transmitted within the voice frame interval of
yte], the termination of a multiplexed packet at the end of a video frame does not cause an increase in overhead in the multiplexed packet or a decrease in transmission efficiency due to insertion of a flag. Since the amount of data varies greatly depending on the image content, the video frame is of an arbitrary length, and therefore cannot always satisfy the above conditions.

For example, the Multiple Entry Desc shown in Table 1
When riptor is applied, when the video frame ends,
Once the multiplex packet has to be closed, a new multiplex packet must be formed.

As described above, when data and the like are multiplexed and transmitted in addition to the audio and video information, each can be transmitted by a predetermined capacity. Therefore, in the case of multiplexing each time an audio packet is generated, it is necessary to transmit data and the video of a new video frame by using a margin until the next audio packet is generated. Then, for that purpose, a new multiplex header is added,
It is necessary to multiplex video information and data.

When a multiplexed packet of video or data that does not include audio information is generated, the number of multiplexed headers increases compared to when multiplexing is performed using a multiplexed packet that includes audio. Efficiency decreases.

[0045]

[0046]

In a videophone that requires synchronous transmission of video and audio, the video and audio are, for example,
4. With a voice codec based on the “G.723.1” standard.
When the rate of 3 [kbps] is selected, the AL2-PDU which is a packet of the second adaptation layer AL2 572
In 6010 (when the sequence number option is not used), 21 [byte] (audio stream + CRC) audio frames are generated every 30 [msec] audio frame intervals.

Therefore, assuming that the transmission speed is 56 [kbps], there is a margin of approximately 180 [bytes] between the generated voice frames, and therefore, approximately 180 [b] every period.
yte] minute, the video stream is transmitted,
When multiplexing of video streams is terminated in video frame units, a header and a flag are newly added, which causes a problem in terms of transmission efficiency.In consideration of that point, when no audio information is generated, It is conceivable that a single packet is transmitted to the end of a video frame.

For example, a Multiplex Entr as shown in Table 1
Suppose y Descriptor is applied. Then, in this example, it is unavoidable that the transmission timing is shifted to the maximum value of the video packet in the worst case of the audio frame interval.

Therefore, in this case, the audio receiving side needs a buffer for absorbing the audio delay jitter covering the maximum shift value, and in this case, the delay of the maximum audio delay jitter is always added to the audio. It is a problem to be added.

That is, when the multiplexed packet at a certain time ends and the HDLC flag ends, and thereafter there is no audio information but only video information, the multiplexing unit sets the entry of MC = 2. A packet multiplexed with the multiplexing pattern is generated.

Since the description content in the entry of MC = 2 is “{LCN3, RC UCF}”,
The multiplexing unit selects a logical channel corresponding to the video information and transmits the video information to the last byte of the frame because the packet transmission (RC UCF) including the video information (LCN3) up to the end of the frame is performed. To control.

As a result, the video packet continues to be transmitted even when the audio frame period is reached, and in the worst case, the audio frame interval reaches the maximum value of the video packet, and the transmission timing is delayed by that amount. Will be interrupted.

The interruption of the sound is hard to hear.

In order to improve this, the voice receiving side needs a buffer for absorbing the voice delay jitter covering the maximum shift value, and in this case, the delay of the maximum voice delay jitter always occurs. In addition to being added to the voice, in the case of a receiving apparatus without a buffer for absorbing voice delay jitter, the voice is interrupted and it is very difficult to listen.

Therefore, there is a demand for the development of a technique for improving such a situation.

Therefore, a first object of the present invention is to minimize the increase in overhead in a multiplexed packet and to suppress a decrease in transmission efficiency due to insertion of a header or a flag while transmitting audio information. It is an object of the present invention to provide a multimedia information terminal device and a multimedia information transmission method capable of transmitting audio and video data without delay.

A second object of the present invention is to minimize the increase in overhead in a multiplexed packet and to suppress a decrease in transmission efficiency due to insertion of a header or a flag, while transmitting audio data. It is an object of the present invention to provide a multimedia information terminal device and a multimedia information transmission method capable of transmitting audio and video data so that a delay can be minimized.

[0058]

In order to achieve the above object, the present invention is configured as follows. That is, [1] First, the present invention relates to a multimedia information device connected to a circuit switching network, and multiplexes in a frame unit with first media information required to be transmitted at a fixed period. At least the second media information whose packet needs to be terminated is set as a transmission target, and a header storing multiplexing code information indicating a multiplexing pattern, and the transmission target media information according to the multiplexing pattern corresponding to the multiplexing code information. In a multimedia information device for multiplexing and transmitting as a packet in a format stored in a payload, a temporary holding unit for temporarily holding the first media information and the second media information, and the medium held by the temporary holding unit Extracting and multiplexing information, and suspending transmission of other media information when the first media information occurs. Multiplexing means for ending the packet transmission and giving priority to the first media information to the next packet to be transmitted so as to be provided for the packet transmission.

According to the present invention, the first media information and the second media information are temporarily stored in the temporary holding means, and the media information held by the temporary holding means is taken out, multiplexed and transmitted as a packet. When the first media information is generated, the transmission of the other media information is suspended, the packet transmission is terminated, and the packet to be transmitted next is processed with priority given to the first media information for the packet transmission.

The second media information, which has to end the packet once at the end of the frame, is transmitted as much as possible to the end of the frame in one packet, thereby reducing the transmission frequency of the header and the flag. Regarding the first media information that needs to be transmitted at a constant period while suppressing the decrease in the transmission efficiency, the transmission can be immediately started at the point of occurrence, and therefore, according to the first aspect of the present invention. For example, while minimizing the increase in overhead in multiplexed packets and suppressing the decrease in transmission efficiency due to insertion of headers and flags, audio and video data can be transmitted without delay in transmission of audio information. A multimedia information terminal device and a multimedia information transmission method can be provided.

[2] Secondly, the present invention relates to a multimedia information device connected to a circuit switching network, wherein the first media information required to be transmitted at a constant period is divided into frame units. Need to terminate the multiplexed packet in the second
And a header storing multiplexing code information indicating a multiplexing pattern, and a media information to be transmitted multiplexed according to the multiplexing pattern corresponding to the multiplexing code information and stored in a payload. A multimedia information device for transmitting as a packet, a temporary holding unit for temporarily holding the first media information and the second media information; a timer for counting an elapsed time each time the first media information is generated; The media information held by the temporary holding means is taken out and multiplexed, and when the first media information is generated and the elapsed time of the timer reaches a set value, the transmission of other media information is suspended and the packet is held. The transmission is ended, and the first media information is prioritized for the next packet to be transmitted and the packet is transmitted. Characterized by comprising a multiplexing means for processing to.

According to the present invention, the first media information and the second media information are temporarily stored in the temporary holding means, the media information held by the temporary holding means is taken out, multiplexed and packet-transmitted. When the elapsed time reaches a set value after the occurrence of the first media information, the transmission of other media information is suspended and the packet transmission is terminated. The first media information is prioritized to the packet to be transmitted to be transmitted to the packet.

The second media information, which has to end the packet once at the end of the frame, is transmitted as much as possible to the end of the frame in one packet, thereby reducing the frequency of transmission of headers and flags. Regarding the first media information that needs to be transmitted at a constant period while suppressing the decrease in the transmission efficiency, the transmission can be performed within an allowable delay range, and thus the second media information can be transmitted. According to the present invention, an increase in overhead in a multiplexed packet is minimized, and a reduction in transmission efficiency due to insertion of a header or a flag is suppressed,
It is possible to provide a multimedia information terminal device and a multimedia information transmission method capable of transmitting audio and video data so that transmission delay of audio data can be minimized.

[0064]

Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

(First Embodiment) In the first embodiment, an increase in overhead in a multiplexed packet is minimized, and a reduction in transmission efficiency due to insertion of a header or a flag is suppressed, while audio information is suppressed. A multimedia information terminal apparatus and a multimedia information transmission method that can transmit audio and video data without causing a transmission delay will be described.

Here, a multimedia telephone terminal system to which "H.324" standardized by ITU-T is applied as one of the protocols to be used for multimedia information communication will be described. “H.324” is the GST
N (standard switched telephone network) is a standard for a low bit rate multimedia telephone terminal (video telephone). FIG. 2 shows a functional configuration diagram of a main part of a system adopting "H.324", and FIG. FIG. 1 shows an example of the network configuration used.

In FIG. 1, reference numeral 156 denotes a GSTN (general switched telephone network: public network), and 100 denotes multimedia information terminals of the present invention, which are connected to each other via a GSTN 156 to enable communication.

Here, the multimedia information terminal 100 is, for example, a videophone, and a camera 10 for imaging a speaker.
0a, a monitor 100b for displaying images, a microphone 100c for inputting audio, a speaker 100d for outputting audio, a data application 100e which is a functional element for handling data, and components of the multimedia information terminal 100. A system control 100f for controlling various controls, a protocol for transferring various signal signals in the telephone network, a signaling protocol unit (call control unit) 154 for controlling calls with the telephone network, and a video signal encoding unit. A video codec unit 150 for decoding the coded video signal to return to the original video signal, encoding the audio signal, and decoding the encoded audio signal to obtain the original audio signal The audio codec unit 151 returns the data to the Data protocol unit 152 has a like control protocol unit 153 is a protocol for transfer of control signals in the telephone network.

Further, the multimedia information terminal 100
Are the video codec unit 150 and the audio codec unit 151
Coded data and signaling protocol section (call control section) 154 and control protocol section (control data protocol section) 153 provided by
And multiplexes various information provided from the data protocol unit 152 and outputs the multiplexed packets (digital signals) to the modem, and separates the digital multiplexed packets received from the modem to separate video, audio, signal, and control signals. A multiplexing unit 140 that distributes these to the type corresponding to the video codec unit 150, the audio codec unit 151, the signaling protocol unit 154, the control protocol unit 153, the data protocol unit 152, and the like. The output (digital signal) of the multiplexing unit 140 is modulated into an analog signal for a telephone network and output to the GSTN 156, and the analog signal received from the GSTN 156 is demodulated into a digital signal and supplied to the multiplexing unit 140. Having.

The main part of the multimedia information terminal 100 will be described in more detail. GSTN which is a network
Multimedia information terminal 10 connected via 156
0 is as shown in FIG. 2, and the video codec unit 150 complies with the “H.263” or “H.261” standard for encoding / decoding video information (video data). The video codec unit 150 encodes a video stream, which is an image data sequence captured and input by the camera 100a as an imaging device, which is one of video I / O devices, and a multiplexing unit 140. And decodes information input from the multiplexing unit 140 to generate a video stream, and outputs the video stream to a monitor (monitor display) 100b, which is one of video I / O devices.

The voice codec 151 has the function "G.723.
The audio information (audio signal) input from the microphone 100c that receives a voice, which is one of the audio I / O devices, and converts the audio into an audio signal. 140 to the multiplexing unit 140
The input audio information is decoded to generate an audio stream, and a speaker 100d which is one of audio I / O devices
Output to

The data protocol section 152 supports a data application, outputs a data stream to the data transmission buffer 112, fetches information from the data reception buffer 111, and transfers the data to the higher-level protocol. Has functions.

The control protocol 153 (control protocol unit) 153 conforms to the standard of “H.245”, and defines commands and procedures for exchanging capability information between terminals and establishing a logical channel. The control protocol section 153 outputs the control data stream to the control data transmission buffer 115. Further, the control data protocol section 153 has a function of fetching information from the data reception buffer 114 and passing data to a higher-level protocol. A signaling protocol unit (call control unit) 154 has a function of controlling call control with the telephone network, and controls connection / release of a network path.

The multiplexing unit 140 conforms to the standard of “H.223”, multiplexes each information of audio information, video information, data information, and control information into a serial signal, and multiplexes each information from the serial signal. Separate.

The modem 155 interfaces the digital signal output from the multiplexing unit 140 to the analog signal of the GSTN 156.

The multiplexing unit 140 includes the multiplexing layer 143 and the adaptation layer units 110, 113, 120, and 13.
0, first-in first-out receiving buffer (FIFO) 11
1, 114, 121, 131, and first-in first-out transmission buffers (FIFO) 112, 115, 122, 132
And a multiplexing table 141.

Of these, the multiplexing layer unit 143 and the adaptation layer unit 11 in the multiplexing unit 140
Reference numerals 0, 113, 120, and 130 correspond to the multiplexing layer unit 574 and the adaptation layer units 571, 572, 573, and 574 in the multiplexing unit 570 described in detail in the related art.

Like the packet format 610 (MUX-PDU) in the multiplexing layer unit 575 in the prior art, the packet format (MUX-PDU) in the multiplexing layer unit 143 in the multiplexing unit 140 is as shown in FIG. It is composed of a 1-byte header 630 and a 0-byte or more payload 640. Then, the HDLC flag 620 is used to indicate a packet delimiter. To ensure that the HDLC flag 620 does not appear in the MUX-PDU 610, the transmitting side
"0" is inserted after five consecutive "1" in PDU 610, and "0" after five consecutive "1" in MUX-PDU 610 is deleted on the receiving side. .

The header 630 includes the multiplexed code area 63
1, header error control area 632, packet marker area 6
33. The multiplexed code area 632 has a 4-bit configuration. The multiplexed code area 632 applies any one of 16 types of decimal numbers “0” to “15” that can be represented by 4 bits. In order to inform whether to perform the transmission, an entry of a multiplexing pattern of a logical channel to which each byte of the payload 640 of the MUX-PDU 610 belongs is designated in an area used for embedding and transmitting the corresponding one.

The header error control area 631 has a 3-bit configuration, and is used for error detection of a multiplexed code area by 3-bit CRC.

The header error control area 631 includes a remainder obtained by dividing the product of the content of the multiplexed code and x ³ by the generator polynomial x ³ + x + 1.

Packet marker area 633 of header 630
Indicates that the last byte of MUX-PDU 610 is AL-P
This is an area for displaying “1” when it is the last byte of DU 710 or 910. The entry of the 0th multiplexing pattern is assigned to the control channel by default. The entries of the first to fifteenth multiplex patterns are exchanged between terminals according to a control protocol (based on the standard of “H.245”).

The adaptation layer units 110, 113, 120, and 130 of the multiplexing unit 140 are also the adaptation layer units 571, 572, 573, and 57 of the prior art.
4, and 110 and 113 are first adaptation layer units (AL1), and AL1 573, 7
54, and 120 is a second adaptation layer unit (AL2), which corresponds to AL2 572, and 13
0 is the third adaptation layer unit (AL3), which corresponds to AL3 571.

First adaptation layer section AL1
Reference numerals 110 and 113 are used to transfer data protocol and control information. Also, the second adaptation layer unit AL2 120 is used to transfer information of the audio codec unit (sound codec unit) 151, and the third adaptation layer unit AL3 130 is used to transfer information of the video codec unit 150. use.

Second adaptation layer section AL2
The format of the 120-packet AL2-PDL is as shown in FIG. That is, as shown in FIG. 8, the AL-PDU 710 includes a 1-byte sequence number area 720, a payload 730 of 1 byte or more, and a 1-byte CRC area 740.

Of these, the sequence number area 720
Is an option, and when this option is used, it is used for detecting a missing AL2-PDU 710 or the like.

The payload 730 is an area for transmitting and receiving information, and the information of the audio codec section (voice codec section) 151 is transmitted and received using the payload 730. The CRC area 740 part is the sequence number area 7
20 (if used) and the remainder of the product of the payload 730 and x ⁸ divided by the generator polynomial x ⁸ + x ² + x + 1.

The third adaptation layer AL3 used to transfer the information of the video codec 150
The format of the packet AL3-PDU of 130 is the same as the conventional one, as shown in FIG.

That is, the third adaptation layer packet AL3-PDU 810 includes a 15-bit sequence number area 820, a 1-bit payload type area 830, a payload 840 of 1 byte or more, and a 2-byte CRC area 850. Be composed. Of these, the sequence number area 820 is optional,
This is used for detecting a missing AL3-PDU 810 or the like.

The payload type area 830 has the AL
3-PDU 810 is used as an information field indicating a payload type. And the payload type is "1"
In the case of, the video codec 15
0 indicates that information is transmitted and received, and when the payload type is “0”, it indicates that a monitoring message is transmitted and received.

The CRC field 850 contains the remainder of the product of the sequence number field 820 (if used) and the contents of the payload 840 and x ¹⁶ divided by the generator polynomial x ¹⁶ + x ¹² + x ⁵ +1.

The first-in first-out reception buffer (FIFO) 111, which is a component of the multiplexing unit 140, is separated by the multiplexing layer unit 143 and supplied to the AL1 110, which is the first adaptation layer unit. The output buffer 110 temporarily stores the output processed in the first-in first-out system.
14 temporarily stores the output processed by the AL1 110 by being separated by the multiplexing layer unit 143 and given to the AL1 110 which is the first adaptation layer unit. (FIFO) 121 is separated by the multiplexing layer unit 143 and is a second adaptation layer unit, AL2 12.
0, the output processed by the AL2 120 is temporarily held, and the first-in first-out receiving buffer (FIFO) 131 is provided in the multiplexing layer unit 1.
43, and is given to AL3 130, which is the third adaptation layer unit.
The output processed at 130 is temporarily stored.

A first-in first-out transmission buffer is used.
The (FIFO) 112 temporarily holds the output of the data protocol unit 152, and the first-in first-out transmission buffer (FIFO) 115 stores the control protocol 1
53 temporarily holds the output of the first-in first-out system, and a first-in first-out transmission buffer (FIFO) 122 temporarily holds the output of the audio codec unit 151. The first-in first-out transmission buffer (FIFO) 132 stores the output of the video codec unit. 150 is temporarily stored.

The above-mentioned video transmission buffer 132 and reception buffer 131 are FIFOs for storing AL3-SDUs in packet units, and have a function of dividing packets and holding the number of stored packets before processing. . The audio transmission buffer 122 and the reception buffer 121 are FIFOs that store the AL2-ADU in packet units, and also have a function of separating packets and holding the number of stored packets before processing.

AL1 110, which is the first adaptation layer unit of the multiplexing unit 140,
In accordance with the request of No. 3, the data held in the transmission buffers 112 and 115 are read and passed to the multiplexing layer 143, thereby engaging in transfer of data protocol and control information. AL2 120, which is the second adaptation layer of multiplexing section 140,
In response to the request of the third embodiment, the audio codec (audio codec) 15 is read by reading out the audio information held in the transmission buffer 122 and passing it to the multiplex layer 143.
AL3130, which is the third adaptation layer unit of the multiplexing unit 140, reads out the video information held in the transmission buffer 132 according to the request of the multiplexing layer unit 143, Part 1
By passing the information to 43, the video codec unit 150 is involved in information transfer.

The multiplexing table 141 has 112, 11
5, a table holding information for providing a multiplexing pattern including a logical channel corresponding to a medium in which data exists, when data exists in any of transmission buffers among 5, 120, and 130; When data exists in the transmission buffer, the multiplexing layer unit 143 converts the multiplexing pattern including the logical channel corresponding to the medium in which the data exists into the multiplexing table.
1 and sends a data read request to the corresponding adaptation layer unit according to the multiplexing pattern, thereby receiving data from the corresponding transmission buffer via the adaptation layer unit and multiplexing according to the multiplexing pattern. To form a packet, which can be transmitted as a multiplexed packet MUX-PDU.

In this embodiment, the increase in the overhead in the multiplexed packet is minimized, the reduction in the transmission efficiency due to the insertion of the header and the flag is suppressed, and the audio and video data are transmitted without delay in the transmission of the audio information. Can be transmitted. For this purpose, the priority is set to the highest priority for the audio information, and when the audio information is generated by the audio codec unit 151 and stored in the transmission buffer 122, A
The configuration is such that priority data is generated from L2.
With this, if there is any other transmission processing that is ongoing, it is put on hold, the packet is temporarily terminated, voice information is transmitted with a new packet, and control is resumed to restart the transmission on hold The multiplexing unit 140 is configured so as to perform the above. By this interrupt processing, it is possible to prevent transmission delay of audio information.

In this embodiment, when there is no audio information, the video information is processed so as to be transmitted in one packet until the end of one frame, thereby improving the transmission efficiency.

That is, in the case where the video and the audio are multiplexed and transmitted, if the conventional method does not allow the delay of the audio, the audio generated every predetermined period T1, T2, T3,. Each time audio is generated, a new multiplexed packet is sent, and the video is sent using the margin generated at that time, so the multiplexed packet is temporarily terminated at the end of each video frame Then, the remaining leeway period is utilized and a new multiplexed packet is used again for video transmission. When such processing is performed, the number of multiplexed headers increases, and there is a risk that the transmission efficiency is reduced. In order to solve this problem, in the present invention, the multiplexing unit 140 , A transmission buffer and a reception buffer are provided, and input / output with the audio codec 151 and the video codec 150, and the data protocol 152, the control protocol 153, and the signaling protocol (call control unit) 154 are buffered to adjust the input / output timing. Therefore, when no audio information is generated, the video information is transmitted as much as possible for one frame for each multiplexed packet. Therefore, there is an effect that an increase in the number of multiplexed headers can be suppressed to a minimum.

As a result, the overhead of the multiplexed packet can be reduced, and the appearance of the flag and the header can be reduced as compared with the conventional case, so that the reduction of the multiplexing efficiency can be prevented.

The details of the processing will be described. Here, the first adaptation layer AL1 is data, the second adaptation layer AL2 is audio, the third adaptation layer AL3 is video, the logical channel number 1 “LCN1” is audio, and the logical channel number 2 Description will be made on the assumption that “LCN2” is assigned to data and logical channel number 3 “LCN3” is assigned to video.

The video codec unit 150 is provided with the “H.26
3 "or" H. The video codec unit 150 encodes a video stream, which is an image data sequence captured and input by the camera 100a as an image capturing apparatus, and transmits the video stream to the transmission buffer of the multiplexing unit 140. 132, and decodes the information fetched from the reception buffer 131 of the multiplexing unit 140 to generate a video stream, which is output to the monitor (monitor display) 100b.

The speech codec 151 conforming to the standard of “G.723” encodes the speech of the speaker input from the microphone 100c, outputs the encoded speech to the transmission buffer 122 of the multiplexing unit 140, and It decodes the information read from the reception buffer 121 of 140 to generate audio information and outputs it to the speaker 100d.

The data protocol section 152 uses the transmission buffer 112 and the reception buffer 111 for exchanging information with the supporting data application.

The control protocol unit 153 conforming to the standard of “H.245” exchanges commands and procedures for exchanging capability information between terminals and opening a logical channel via the transmission buffer 115 and the reception buffer 114.

The multiplexing section 140 conforms to the standard of “H.223”, and multiplexes information of audio information, video information, data information, and control information into the corresponding transmission buffers 132, 122, 112, Capture via 115,
The signal is multiplexed into a serial signal and passed to the modem 155. The serial signal received from the GSTN 156 via the modem 155 is separated into audio information, video information, data information, and control information, and the corresponding reception is performed. The data is temporarily stored in the buffers 131, 121, 111, and 114.

The modem 155 interfaces the digital signal output from the multiplexing unit 140 with the analog signal of the GSTN 156, converts the analog signal received from the GSTN 156 into a digital signal, and delivers the digital signal as a serial signal to the multiplexing unit 140. .

In the present embodiment, in order to maintain transmission efficiency,
Information other than audio, especially video information, must be transmitted in a single packet up to the end of the frame of the video information, and when audio information is generated, the packet is terminated and the transmission of audio information is started. , So that there is no break in the sound.

In this embodiment, the time required for video encoding / decoding processing is reduced to the time required for audio encoding / decoding processing in communication that requires synchronous reproduction of audio and video. Therefore, the fact that there is no problem even if the audio information is buffered on the transmission system side in the multiplexing unit and the delay is allowed within an allowable range is used. A reception buffer is provided, and these are used effectively to adjust the input / output timing of the audio codec section 151, the video codec section 150, and the data protocol section 152, the control protocol section 153, and the signaling protocol section (call control section) 154. On the other hand, in an audio frame period in which no audio information is generated, one frame of video information is transmitted for each multiplexed packet. And, when the voice information is generated once to terminate the multiplexed packet of video information, so as to give priority to the timing of the audio information so as to transmit the voice information.

This will be described in detail below.

FIG. 3 is a flowchart showing the operation of multiplexing section 140. In the multiplexing unit 140, the transmission buffers 132, 122, 112, and 11 are controlled by a control unit (not shown).
5 to monitor whether transmission data is stored. If there is no transmission data, the multiplexing unit 140
Control unit instructs the multiplexing layer unit 143 to transmit a flag.
The transmission of the DLC flag is repeated (steps S210 and S210).
220 is repeated).

If data exists in the transmission buffer, the multiplexing layer unit 143 loads a multiplexing pattern including a logical channel corresponding to the medium in which the data exists from the multiplexing table 141 (step S230), and follows the multiplexing pattern. , Receives data from the corresponding transmission buffer via the corresponding adaptation layer unit, multiplexes the data, and multiplexes the MUX-PDU 6
10 transmission is started.

A case where a multiplexing table as shown in Table 1 is used as the contents of the multiplexing table 141 will be considered.

HDLC flag ("01111110")
Is checked for transmission data every time is transmitted. As a result, when video information and audio information are generated simultaneously, such as at time t1 in the audio frame period T1 in FIG. Then, simultaneous communication of video information and audio information is performed.

That is, immediately after the start of the transmission, 21 bytes of audio data are written from the audio codec unit 151 to the audio transmission buffer 122 (the 21 byte audio data is transmitted from the audio codec unit 151 to the audio transmission buffer 122). Is written in audio frame units), 21 [byte] × N
(N is an integer equal to or more than “2”). When a situation where information equal to or more is accumulated is reached, the second adaptation layer unit A
The L2 120 notifies the multiplexing layer unit 143 of the completion of the voice packet transmission preparation.

The multiplexing layer unit 143 receives the completion of the transmission preparation of the AL2 120 as the second adaptation layer unit, and determines that there is transmission data (step S22).
0). The multiplexing layer unit 143 selects an entry of the multiplexing table according to the type of the medium of the transmission preparation completion notification, and loads the entry into the header. The entry of the multiplexing table selected here is the multiplexing entry value MC = 3 in Table 1.

Then, information is multiplexed in a multiplexing pattern (Mu1tiplex Entry Descripter) determined by MC = 3. This is "｛LCN1, RC21｝ ｛LCN3, R
C, UCF} ", and the multiplexing pattern transmits the audio information for 21 [bytes] and then converts the video information to 1
Since the description is to transmit until the end of the frame, the multiplexing layer unit 143 selects a logical channel corresponding to the audio information (step S240), and converts the audio information to 21 [b].
yte], the AL2 120 that is the second adaptation layer unit is instructed to extract the audio information from the transmission buffer 122, and according to this, the AL2 120 converts the voice information to 21 [byt].
e] is taken out of the transmission buffer 122 and given to the multiplexing layer section 143. The multiplexing layer unit 143 performs multiplexing processing on the multiplexed packet and transmits the multiplexed packet (step S
270, S271, S272).

When the transmission of the audio information for 21 [bytes] is completed, the multiplexing layer unit 143 next selects the logical channel corresponding to the video information, and transfers the video information from the video transmission buffer 132 for one frame. Control is performed so that the data is extracted, multiplexed, and transmitted (see time t2 in FIG. 4).

On the other hand, if the preparation for transmitting only the voice is completed, the multiplexing pattern (Mu1tiplex Entry Descripter) determined by the multiplexing entry value MC = 1 in Table 1 is loaded (step S230). This is "@ LCN1, RC2
1｝ ”, and the multiplexing pattern is a description that the voice information is transmitted for 21 [bytes]. Therefore, the multiplexing layer unit 143 selects a logical channel corresponding to the voice information (step S240). ), Voice information 21 [b
yte], the AL2 120 that is the second adaptation layer unit is instructed to extract the audio information from the transmission buffer 122, and according to this, the AL2 120 converts the voice information to 21 [byt].
e] is taken out of the transmission buffer 122 and given to the multiplexing layer section 143. The multiplexing layer unit 143 performs multiplexing processing on the multiplexed packet and transmits the multiplexed packet (step S
270, S271, S272).

When there is no video information to be transmitted at the time when the transmission of the packet is completed, the multiplexing layer unit 143 transmits the HDLC flag by the amount corresponding to (transmission rate × audio frame (multiplexed packet length)).

When the transmission of the HDLC flag is completed, the amount of data stored in the audio transmission buffer 132 is checked. As a result, information of 21 bytes or more exists. The AL3 130, which is written into the video transmission buffer 132 and is the third adaptation layer unit, notifies the multiplexing unit 140 of the completion of the video packet transmission transmission preparation. Multiplexing layer section 143
Selects and loads a corresponding multiplexing table entry in the multiplexing table 141 according to the type of medium of the transmission preparation completion notification. At this point, since the preparation for audio transmission and video transmission is completed, the multiplexing pattern (Mu
Load 1tiple Entry Descriptor).

The description content of the multiplex pattern of the entry MC = 3 is “{LCN1, RC21}, {L
CN3, RC UCF｝ ”, voice information (LC
N1) is transmitted by 21 [Byte] (RC21), and the video information (L
CN3) is transmitted as one packet (RC UCF).
The logical channel corresponding to the audio information is selected (step S240), and control is performed so that 21 [bytes] of the audio information is extracted from the audio transmission buffer 122 and transmitted (steps S270, S271, and S272).

When the transmission of the audio information for 21 [bytes] is completed, the multiplexing layer unit 143 next selects a logical channel corresponding to the video information, and transfers the video information from the video transmission buffer 132 to one frame. Control to take out and send minutes.

When the video information frame ends in less than 180 [bytes], the multiplexing layer unit 143 immediately sets the HLD.
Control is performed so as to transmit the C flag (step S21).
0) Terminate the MUX-PDU once.

Then, the storage amount in the audio transmission buffer 132 is determined, and if there is audio information of 21 [bytes] or more, control is performed to extract and transmit 21 [bytes] (steps S270, S271, S271). S272) Then, control is performed so that the video information is multiplexed by one frame of the video multiplex area video information in the previous multiplexed frame.

In this way, the apparatus of the present invention transmits 21 [Byte] of audio information when there is audio information at every fixed audio frame period, and transmits 21 [Byte] of audio information. When the transmission of the information is completed, the logical channel corresponding to the video information is selected, and one packet of the video information is transmitted. However, when the video information exceeds 180 [Byte], the audio information is transmitted more than the video information. If the transmission priority is set high, A
When there is a request for voice packet transmission from L2, multiplexing section 140 determines priority data (step S28).
1).

In this case, since the priority is set to the higher order, the priority data is output from AL2.
3 suspends the transmission of the video information, transmits the HLDC flag (step S210), and temporarily terminates the MUX-PDU (see time t3 in FIG. 4). Then, transmission of voice information is newly started (steps S270, S271, S2
72) (see audio frame period T2 in FIG. 4).

Thus, if the priority of the audio information transmission is set higher than the priority of the video information, if the audio information is generated, the audio information is always transmitted at a predetermined audio frame period. Thus, the shift of the transmission timing of the audio information can be suppressed.

By adopting such a processing method, even if the number of non-multiplexed video frames is less than 180 [bytes], the multiplexed packet length can be reduced without newly forming a multiplexed packet of only video. Can be adaptively changed, which not only prevents an increase in multiplexing overhead, but also reduces the priority of video information of lower priority when audio information is generated. The transmission is suspended, the multiplexed packet used for the transmission is immediately terminated, the audio information is transmitted in a new multiplexed packet, and then the suspended video information is transmitted. In this case, media information such as audio information that must be transmitted at a constant cycle can suppress transmission delay. Therefore, it is possible to reliably prevent interruption of the sound.

As described above, in the first embodiment, an increase in overhead in a multiplexed packet is minimized, and a reduction in transmission efficiency due to insertion of a header or a flag is suppressed, but there is no delay in transmission of voice information. The technology that enabled transmission of audio and video data was described.

However, in some environments, transmission delays of voice information can be tolerated to some extent, and a technique suitable for such an environment will be described next as a second embodiment.

(Second Embodiment) In the second embodiment, a timer is provided so that when media information that needs to be transmitted at a constant period, such as voice, is generated, the elapsed time from the generation is counted. If the elapsed time exceeds an allowable delay time set in advance, the transmission operation of media information that does not need to be transmitted at a fixed period is suspended, and the transmission of the multiplexed packet used for the transmission is terminated. However, it is also possible to adopt a configuration in which the multiplexed packet to be transmitted next is preferentially used for transmitting media information that needs to be transmitted at a fixed period.

That is, priority is given to the generated media information, and from the point of occurrence of the high-priority media information,
By monitoring the elapsed time using a timer to determine the end of the MUX-PDU, the upper limit value of the shift of the transmission timing of high-priority media information, for example, audio information, is determined by the set time of the timer. To set.

As a result, video packets are not continuously transmitted even when the audio frame period is reached,
Even in the worst case, the audio frame interval is kept within the set value, and the delay in transmission timing is suppressed. As a result, interruption of audio is not caused.

In addition, since a buffer for absorbing the audio delay jitter covering the maximum shift value is not required on the audio receiving side, the problem that the delay of the maximum audio delay jitter is always added to the audio is also solved. Is done.

In the method of the present embodiment, even if the video reaches the end of the video frame in the audio frame period (T1), the video remains in the audio frame period (T1).
1, the header and the HDLC flag are entered, and then the audio and the last part of the video at T1 are multiplexed, where they end once again, the header and the HDLC flag are entered again, and T1
The video will be sent using the remaining time. Then, assuming that the next audio frame period (T2) is reached and audio information is generated, the audio is buffered,
In addition, since the audio information is allowed to have a jitter up to a predetermined value, it is not necessary to immediately enter the transmission mode. Therefore, even in the next audio frame period (T2), transmission of only video is continued within the allowable range. assign. That is, the video information continues to be transmitted from the middle of the audio frame period of T1 to the next audio frame period of T2, and the header and HDLC are transmitted.
No flags are entered.

As described above, the restriction that the voice is always transmitted at each break of the voice frame period is released, and the voice is postponed to the next voice frame period. As a result, the header and the HDLC flag are inserted. The number of times of transmission is reduced as compared with the conventional case, and the transmission efficiency is improved.

On the other hand, on the receiving side, for example, when considering audio and video, the audio and video are separated by the multiplexing layer unit 143 of the multiplexing unit 140 and are received via the AL2 through the audio receiving buffer 121.
Is stored in the reception buffer 121, is separated by the multiplexing layer 143 of the multiplexing unit 140, and is supplied to the video reception buffer 131 via the AL3. Since the information is stored in the storage 131, the audio codec 151 and the video codec 150 sequentially read and decode the information from the corresponding reception buffers 121 and 131.

In this embodiment, a timer is set in the time range in which the interruption of the audio does not cause a problem so that the time from the generation of the audio is monitored. By setting the time to about the time required for the decoding process, it becomes possible to reproduce the video and the audio including the buffering effect of the reception buffers 121 and 131 within a range where the interruption is not a problem.

As described above, in the second embodiment, a timer is provided,
For example, when media information that needs to be transmitted at a fixed period, such as voice, is generated, the elapsed time from the generation is counted, and when the elapsed time exceeds a preset allowable delay time, Transmission of media information that does not require periodic transmission is suspended, transmission of the multiplexed packet used for the transmission is terminated, and transmission of media information that requires transmission at a fixed period is performed for the next multiplexed packet to be transmitted. This is a configuration that is used preferentially.

That is, priority is given to the generated media information, and from the point of occurrence of the high-priority media information,
By monitoring the elapsed time using a timer to determine the end of the MUX-PDU, the upper limit value of the shift of the transmission timing of high-priority media information, for example, audio information, is determined by the set time of the timer. To set.

As a result, it is no longer said that video packets continue to be transmitted even when the audio frame period is reached.
Even in the worst case, the audio frame interval is kept within the set value, and the delay in transmission timing is suppressed. As a result, interruption of audio is not caused.

Further, since a buffer for absorbing the audio delay jitter covering the maximum shift value is not required on the audio receiving side, the problem that the delay of the maximum audio delay jitter is always added to the audio is also solved. Is done.

Note that the present invention is not limited to the above-described embodiment, but can be implemented in various modifications. For example, in the embodiment of the present invention, the H.264 standardized by ITU-T is used.
The description is based on "H.223" which is a multiplexing protocol used for terminals conforming to the H.324 standard.
ANNEX A / ANNEXB in 223 "standard
Even when the multiplexing method of / ANNEX C / ANNEX D is applied, the multiplexing method can be applied only with a difference in the HDLC flag, the stuffing process when there is no transmission data, and the like.

[0145]

As described above in detail, according to the first aspect of the present invention, an increase in overhead in a multiplexed packet is minimized, and a decrease in transmission efficiency due to header and flag insertion is suppressed. In addition, it is possible to provide a multimedia information terminal device and a multimedia information transmission method capable of transmitting audio and video data without delay in transmitting audio information.

According to the second aspect of the present invention, an increase in overhead in a multiplexed packet is minimized, and a reduction in transmission efficiency due to insertion of a header or a flag is suppressed. It is possible to provide a multimedia information terminal device and a multimedia information transmission method capable of transmitting audio and video data so that the data can be minimized.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining the present invention, and is a diagram showing an entire configuration of a specific example of the present invention.

FIG. 2 is a diagram for explaining the present invention, and is a block diagram showing a main configuration of a multimedia information terminal according to the present invention.

FIG. 3 is a diagram for explaining the present invention, and is a flowchart for explaining an operation example of a multiplexing unit in the multimedia information terminal according to the present invention;

FIG. 4 is a diagram for explaining the present invention, and is a diagram for explaining the operation of the system of the present invention.

FIG. 5 is a diagram for explaining a conventional technique.

FIG. 6 is a diagram for explaining a conventional technique.

FIG. 7 is a diagram illustrating a format of a MUX-PDU.

FIG. 8 is a diagram illustrating a format of an AL2-PDU.

FIG. 9 is a diagram illustrating the format of an AL3-PDU.

FIG. 10 is a diagram for explaining a conventional technique.

FIG. 11 is a diagram for explaining a conventional technique.

[Explanation of symbols]

100: multimedia information terminal, 100a: camera,
100b: monitor, 100c: microphone, 100
d: speaker, 110, 113, 120, 130: adaptation layer section, 111, 114, 121, 13
1 ... first-in first-out receiving buffer (FIFO), 11
2, 115, 122, 132: first-in first-out transmission buffer (FIFO), 141: multiplexing table, 154
... Signaling protocol section (call control section), 150 ... Video codec section, 151 ... Audio codec section, data 15
2 Data protocol unit, 153 Control protocol unit (call control unit), 140 Multiplex unit, 143 Multiplex layer unit, 155 GSTN (general switched telephone network: public network).

Continued on the front page. (72) Inventor Tatsuo Samada 70, Yanagicho, Kochi-ku, Kawasaki-shi, Kanagawa Prefecture Inside the Toshiba Yanagimachi Plant (72) Inventor Takashi Ohara 70, Yanagicho, Kochi-ku, Kawasaki-shi, Kanagawa Prefecture Toshiba Yanagicho Plant F Term (reference) 5C064 AA01 AC04 AC06 AC17 AD02 AD08 AD09 AD13 5K028 AA11 CC02 CC06 DD01 DD02 DD03 EE03 EE05 EE07 JJ02 KK32 MM04 MM05 MM08 RR01 SS04 SS14 SS26 5K030 GA01 HB01 HB02 HB21 J04LBJT LB11 JA04B14 9A001 BB04 CC02 CZ06 JJ19

Claims (6)

[Claims] 1. A multimedia information device connected to a circuit switching network, comprising: first media information required to be transmitted at a constant period; and second media information required to terminate a multiplexed packet in frame units. And a header storing multiplexing code information indicating a multiplexing pattern, and a media information to be transmitted multiplexed according to the multiplexing pattern corresponding to the multiplexing code information and stored in a payload. In a multimedia information device for transmitting as a packet, a temporary holding unit for temporarily holding the first media information and the second media information, and the media information held by the temporary holding unit is taken out, multiplexed, When the first media information is generated, the transmission of other media information is suspended and the packet transmission is terminated, Multimedia information device according to claim in favor of the first media information to a packet to be transmitted to and a multiplexing means for processing to Kyosu the packet transmission. 2. A multimedia information device connected to a circuit switching network, comprising: first media information required to be transmitted at a constant period; and second media information required to terminate a multiplexed packet in frame units. And a header storing multiplexing code information indicating a multiplexing pattern, and a media information to be transmitted multiplexed according to the multiplexing pattern corresponding to the multiplexing code information and stored in a payload. A multimedia information device for transmitting as a packet, a temporary holding unit for temporarily holding the first media information and the second media information; a timer for counting an elapsed time each time the first media information is generated; The media information held by the temporary holding means is taken out and multiplexed, and the first media information is When the elapsed time of the timer without reaches the set value pending transmission of other media information terminates the packet transmission,
Multiplexing means for processing the packet to be transmitted next so that the first media information is prioritized for packet transmission. 3. The multimedia information apparatus according to claim 1, wherein the first media information is audio information, and the second media information is video information. 4. A method for transmitting information of a multimedia information device connected to a circuit switching network, wherein the first media information required to be transmitted at a constant period and a multiplexed packet in a frame unit need to be terminated. A second media information having at least a transmission target, and a header storing multiplexing code information indicating a multiplexing pattern, and multiplexing the transmission target media information according to the multiplexing pattern corresponding to the multiplexing code information, to a payload. A multimedia information transmitting method for transmitting as a packet in a stored format, wherein the first media information and the second media information are temporarily held in a temporary holding unit, and the media information held by the temporary holding unit is taken out. Multiplexes and transmits the packet, and transmits the other media information when the first media information is generated. A method for transmitting multimedia information, comprising suspending packet transmission and terminating packet transmission, and prioritizing the first media information for the next packet to be transmitted for use in packet transmission. 5. An information transmission method for a multimedia information device connected to a circuit switching network, wherein the first media information required to be transmitted at a constant period and a multiplex packet need to be terminated in frame units. A second media information having at least a transmission target, and a header storing multiplexing code information indicating a multiplexing pattern, and multiplexing the transmission target media information according to the multiplexing pattern corresponding to the multiplexing code information, to a payload. A multimedia information transmitting method for transmitting as a packet in a stored format, wherein the first media information and the second media information are temporarily held in a temporary holding unit, and the media information held by the temporary holding unit is taken out. The packet is multiplexed and transmitted, and the elapsed time is counted each time the first media information is generated. After the occurrence of media information 1
When the elapsed time reaches a set value, the transmission of other media information is suspended, the packet transmission is terminated, and processing is performed so that the packet to be transmitted next is given priority to the first media information and is provided for packet transmission. Characteristic multimedia information transmission method. 6. The method of transmitting multimedia information according to claim 4, wherein said first media information is audio information, and said second media information is video information. .