JP2523995B2 - Video / audio multiplex transmission system - Google Patents

Description

TECHNICAL FIELD The present invention relates to a video / audio multiplex transmission system, and more particularly to a system for encoding and multiplexing audio information and video information for transmission.

In recent years, communication for transmitting both audio information and video information has become popular, such as a video conference and a videophone. In such a multiplex communication system, encoding / decoding of both can be performed with high efficiency. It is also important to maintain the balance between audio information and video information as well as the above.

BACKGROUND ART In a conventionally known video / audio coding system in which coded audio information is multiplexed and transmitted together with video coding information and other control information, two 64 kb / s lines are used. × B method, one 128 kb / s line is used 2
When the transmission rate is low, such as B or B method using one 64 kb / s line, the transmission ratio between audio and video signals is generally about 1: 1 (for example, the audio encoding rate is 56kb / s and video coding speed is 64kb / s), about 1: 3
(For example, if the audio coding rate is 32 kb / s and the video coding rate is 96
kb / s) or about 1: 7 (for example, the voice coding rate is 1
The video coding speed is 6 kb / s and the video coding speed is 112 kb / s).

However, although the ratio of the information density between the audio signal and the video signal originally has a difference of several hundred times, since it is transmitted at the above ratio, when the transmission ratio is 1: 1,
The transmission amount of video information per unit time is reduced, which causes deterioration of moving image reproduction quality, while the transmission ratio is 1:
In the case of 7, the voice quality will be worse this time. Particularly in the latter case, when encoding a video with little motion, unnecessary bits (fill bits) are added to match the transmission capacity to match the transmission bit rates.

As described above, in the conventional method, there is a problem that either one of the image quality and the audio quality is sacrificed.

In the video encoding unit used in such a video / audio multiplexing system, the video information is encoded by an encoding method such as interframe predictive encoding, intraframe predictive encoding, motion compensation predictive encoding, and variable length encoding. Although redundancy compression is attempted, the amount of information of the video signal is much larger than that of the audio signal, which causes a delay in the encoding / decoding processing time of the audio signal.

Therefore, the receiving side cannot secure the synchronization (lip sync) between the video and the audio, and the video and the audio do not match and the reproduction is unnatural.

For this reason, a method is also used in which the average of the delay time of the processing of the video signal with respect to the treatment of the voice is obtained in advance, and a fixed delay time determined with reference to this is generated during the voice processing of the system. However, since the actual delay time changes depending on the contents of the video information, there is a problem that unnaturalness cannot be improved sufficiently.

Also, since the amount of image information is usually much larger than that of audio information, not all image information is actually sent. That is, the part that cannot be sent is ignored, and the video information is thinned out and sent. As a result, the image reproduced on the receiving side has a rough motion. This is because the real-time property is more important than the image quality.

On the other hand, for a voice signal, a transmission capacity of 16 kps / s or 56 kps / s is secured, even though there is a silent period in the voice. For this reason, it is desired to utilize the silent period in the audio signal for the video signal.

DISCLOSURE OF THE INVENTION The present invention controls the entire system including video information processing and audio information processing based on at least one piece of video information and audio information to be transmitted, and provides optimum video quality and audio quality for the entire system. The purpose is to maintain.

First, according to one aspect of the present invention, this object is realized as follows.

A / D converter that converts audio input into digital audio, audio encoder that encodes digital audio and outputs it as encoded audio with selectable transmission amount, and audio content information, and video input A / D to convert digital to digital video
A converter, a video encoding unit that encodes digital video and outputs it as encoded video, and determines the transmission ratio of encoded audio and encoded video according to the information amount of at least one of encoded audio and encoded video, An encoding control unit that outputs as an allocation signal, and a multiplexing unit that multiplexes encoded audio and coded video based on the allocation signal and control information including the allocation signal so as to have a constant transmission frame length. A transmission unit provided,
A separation unit that receives the multiplexed signal sent from the transmission unit to the transmission line and sent through the transmission line, and separates the encoded signal into a control signal that includes an encoded video and an allocation signal, and an encoded voice. An audio decoding unit for decoding and decoding digital audio, a video decoding unit for decoding coded video and decoding digital video, and a decoding for controlling the audio decoding unit and the video decoding unit based on the allocation signal Audio and video depending on the transmission contents, which are composed of a digitalization control unit, a D / A converter that converts decoded digital audio into an audio signal, and a D / A converter that converts decoded digital video into a video signal It is a video / audio multiplex transmission system that transmits by changing the transmission ratio of.

The audio information has a smaller amount of information than the video information, but thinning and the like are not allowed. Therefore, the transmission ratio of the video and the audio is fixed, and the audio information of a constant quality is transmitted regardless of the transmission content.

In the present invention, the transmission ratio can be made variable. Of course, for this purpose, it is necessary to change the transmission ratio without interruption of the voice information. Therefore, adaptive coding is performed in which the audio information is output as a plurality of coded sounds having different coding bit rates. Then, by selecting one of the most suitable coding bit rates according to the transmission content, the audio quality will fluctuate, but the audio will be transmitted with the video and audio quality most suitable for the state at that time without interruption. Be done. The receiving side also performs reproduction according to the encoding bit rate.

In addition, a plurality of encoded bit rates are compared to select the most suitable quality, that is, the bit rate, from the audio contents.

Another adaptive encoder is SB-ADPCM, which encodes separately in the low-frequency part with high information density and the high-frequency part with low information density. Only bring.

In the case of this SB-ADPCM and multiple coding bit rates, if the selection of the number of bits is performed considering not only audio information but also video information and video buffer status, the overall system becomes more balanced. The rate of information change between frames is used as the video information.

The video coding unit performs various processes to compress the video information. Therefore, a video encoding unit that encodes and quantizes digital video, a variable length encoding unit that performs variable length encoding of encoded video information, a buffer that is temporarily stored before multiplexing, and a buffer of this buffer. It is composed of a buffer determination unit that indicates a storage state. The video encoding unit outputs a storage amount signal indicating this buffer storage state.

If the buffer is full, video signals will be decimated and it will take some time before they are sent out.

In order to match the output of video and audio so that there is no deviation regardless of the transmission contents, an A / D converter that converts audio input into digital audio, an audio encoding unit that encodes digital audio, and an audio An A / D converter that converts the input into digital video, a video coding unit that codes the digital video, and video coding that synchronizes the video playback output and audio playback output from the input / output information of the video coding unit. A delay amount calculation unit that generates delay time information, a transmission unit that includes a multiplexing unit that multiplexes encoded video information and audio information and delay time information, and a multiplexed signal from a transmission path And a variable delay control unit for delaying the audio information according to the delay time information, and an audio decoding unit for decoding the audio information from the variable delay control unit. , Decrypted digital D for converting voice into a voice signal /
A converter, a video decoding unit that decodes video information, and a D / A that converts the decoded digital video output to a video output
It is realized by a video / audio multiplex transmission system composed of a converter and a receiver equipped with.

In order to use the delay amount and transmit the audio part at a level below a certain level, that is, the silent part, to transmit the audio part as a whole, in order to transmit the audio part collectively, the video encoding part that encodes the video signal, and the code A variable length coding unit for giving a variable length code to the converted result, and a speech coding unit corresponding to a speech signal, and the output from the variable length coding unit and the speech coding. In the video / audio transmission system configured to transmit the output corresponding to each section in a multiplexed manner and performing the process of extracting the video signal and the audio signal from the one transmitted at the receiving side, Based on the output from the encoding unit, a time division encoding unit for extracting and packetizing the signal during the valid period of the voice is provided, and the time division encoding unit notifies the system control unit of the voice transmission rate. Structure The system control unit receives the voice transmission rate, and stores the buffer rate in the variable length encoding unit.
The threshold data for controlling the encoding amount of the video encoding unit is changed according to the amount of data on the memory, and the transmission is performed according to the frame format suitable for the audio transmission speed. It is realized by the video / audio multiplex transmission system.

[Brief description of drawings]

FIG. 1 shows a conventional video / audio multiplex transmission system, FIG. 2 shows a transmission format based on CCITT Recommendation Y.221, FIG. 3 shows a conventional example with a delay unit, and FIG. Is an explanatory diagram of feedback control of a buffer portion in the system shown in FIG. 3, FIG. 5 shows a transmission frame format when the transmission amount in the system shown in FIG. 3 is different, and FIG. 6 is a basic configuration of the present invention. In the figure, FIG. 7 is a transmission side block diagram of one embodiment, FIG. 8 shows a voice frame format, FIG. 9 shows a voice frame format with a different number of bits, and FIG. 10 shows multiplexing / demultiplexing. FIG. 11 is an example of BAS code allocation and a table of codes, FIG. 12 is an explanatory diagram for separating the coding bit rate from BAS information, and FIG. It is explanatory drawing which shows allocation of information, 14 to 17 show examples of transmission frames when the number of bits of audio information is changed, FIG. 18 shows examples of other transmission frame formats, and FIG. 19 is writing to the format of FIG. FIG. 20 to FIG. 23 show an example of a transmission frame when the number of bits of voice information is changed, and FIG. 24 shows a method of moving the bit rate selector to the multiplexer in the system shown in FIG. 25 shows an example of the configuration of the multiplexing / demultiplexing unit of the system shown in FIG. 24, FIG. 26 shows the change in the transmission ratio of audio and video, and FIG. 27 shows that of audio and video. FIG. 28 shows a conceptual diagram of data transmission when the transmission ratio changes, FIG. 28 shows a basic configuration diagram of another embodiment, FIG. 29 shows a block diagram of the transmission side of the system shown in FIG. Figure 30 shows the procedure for determining the bit rate from the rate of change between frames of video and the optimal bit rate of audio. FIG. 31 is a row chart diagram, FIG. 31 is a basic configuration diagram of another embodiment, FIG. 32 is an explanatory diagram of a buffer determination unit of the system shown in FIG. 31, and FIG. 33 is a bit array of FAS information. An example is shown in FIG. 34, a BAS code payout example is shown, FIG. 35 is a frame format diagram showing an example of voice allocation bits, and FIG. 36 is a decoding operation based on voice allocation bits. 37 is a diagram showing an in-channel connection protocol, FIG. 38 is a diagram showing an example of additional bits for inquiring about capability in AC information, and FIG. 39 is another diagram. 40 shows a basic configuration diagram of an embodiment of FIG. 40, FIG. 40 shows a conceptual diagram of frame information used in the system shown in FIG. 39, and FIG. 41 shows an embodiment of the delay amount calculation section of the system shown in FIG. FIG. 42 is a block diagram showing the multiplex of the system shown in FIG. 39. FIG. 43 is a block configuration diagram showing an embodiment of the separation unit, FIG. 43 is a diagram showing control information in the frame format of the system shown in FIG. 39, and FIG. 44 is a basic configuration diagram of another embodiment. 45 is an explanatory diagram of a transmission frame format of the system shown in FIG. 44, FIG. 46 is an explanatory diagram of processing in the time division encoding unit, and FIG. 47 is an explanatory diagram of feedback control.

[Best mode for carrying out the invention]

Before describing the embodiments of the present invention, a conventional video / audio multiplex transmission system will be described with reference to FIGS. 1 to 5 in order to better understand the present invention.

It should be noted that the same object is denoted by the same reference numeral throughout the drawings. FIG. 1 is a block diagram showing a configuration of a conventional video / audio multiplex transmission system, and shows only a transmitting unit and a receiving unit at both end stations. As shown in the figure, the voice input is converted into digital voice by the A / D converter 1, further encoded by the voice encoding unit 2 and input to the multiplexing unit 6. The video input is converted into a digital video by the A / D converter 3, further quantized by the video coding unit 4, and then variable length coded.
The variable length coded video information is temporarily stored in the buffer and input to the multiplexing unit 6. The video encoding unit 4 is configured to have such a function. The coded audio and the coded video input to the multiplexing unit 6 are multiplexed at a fixed ratio so as to have the form of the transmission frame format as shown in FIG. 2, and further control information such as FAS, BAS, AC is added. It is multiplexed and sent to the transmission line 10.

The multiplexed signal from the transmission line 10 is encoded voice in the demultiplexing unit 11,
It is separated into encoded video and control information. The separated encoded voice is converted into digital voice by the voice decoding unit 12, further converted into a voice signal by the D / A converter 13, and output. On the other hand, the separated encoded video is converted into a digital video by the video decoding unit 14, further converted into a video signal by the D / A converter 15, and output. The voice encoding unit 2 and the voice decoding unit 12 correspond to each other so that the encoded voice can be decoded in the original form. The same applies to the video encoding unit 4 and the video decoding unit 14.

The following is the case where a delay is predicted in advance and a fixed delay is given.

FIG. 3 shows a conventional configuration of one terminal station in the video / audio transmission system. In the figure, reference numeral 20 is a terminal station, 3 is an A / D converter for video signals, 41 is a video encoding unit, 42 is a variable length encoding unit, 1 is an A / D conversion unit for audio signals, and 2 is an audio code. A demultiplexing unit, 7 a delay control unit, 6 and 11 demultiplexing / demultiplexing units,
68 is a transmission line interface unit, 14 is a video decoding unit, and 15 is
A D / A conversion unit, 12 is a voice decoding unit, 13 is a D / A conversion unit, and 19 is a system control unit.

The video signal is subjected to variable length coding processing after being processed in the video coding section 3 and supplied to the multiplexing / demultiplexing sections 6 and 11. On the other hand, the audio signal is 4KHz in the audio encoding unit 2.
A delay control unit 7 which is coded in a band of 16 Kbps or a 7 KHz band of 56 Kbps and gives a delay corresponding to a delay for encoding a video signal is provided, and after being delayed in the delay control unit 7, the multiplexing / demultiplexing unit 6, Supplied to 11.

Then, the video signal and the audio signal are multiplexed and transmitted from the transmission path interface unit 68 to the opposite terminal station.

For the received signal from the opposite terminal station, the multiplexer / demultiplexer 6,
At 11, the video signal and the audio signal are separated. The video signal is received by the video decoding unit 14 and the D / A conversion unit 15 as a video output. Further, the audio signal is received as an audio output through the audio decoding unit 12 and the D / A conversion unit 13.

The delay control amount T in the delay control unit 7 is T = (t _V1 + t _V2 )-(t _A1 + t _A2 ). Given in.

The transmission line has a fixed capacity with a line speed of 64 Kbps from 64 Kbps to 384 Kbps. For this,
Feedback control as shown in FIG. 4 is performed to control the encoding process in the video encoding unit 4.

FIG. 4 shows a mode of feedback control. Reference numerals 41, 4, 19 in the figure correspond to those in FIG. 3, 42 is a variable length coding unit, 43 is a buffer memory, 17 is a first threshold value, 18
Represents the second threshold value. The variable length coded video data is stored in the buffer memory 43, and is read from the buffer memory 43 at a constant speed and guided to the multiplexing / demultiplexing units 6 and 11.

According to the amount of video data stored in the buffer memory 43, when the storage amount exceeds the second threshold value 18 as shown in FIG. 4 (B), for example, the system control unit 19 performs video coding. Instruct the unit 41 to stop video coding. Alternatively, the quantization of the video coding unit is roughened. Also, when it reaches the stage where the accumulated amount drops below the first threshold value 17,
The system control unit 19 gives an instruction to restart video coding. Alternatively, the quantization is detailed.

FIG. 5 shows a transmission frame format in the conventional case.

FIG. 5 (A) shows a frame format in the case of 64 Kbps. 8 Kbps is allocated to the frame information portion, 16 Kbps is allocated to audio, and 40 Kb is allocated to video.
Allocating ps minutes.

FIG. 5 (B) shows a frame format in the case of 128 Kbps, and there are a case where 16 Kbps is allocated to voice (B-1) and a case where 56 Kbps is allocated to voice (B-2). Exists. FIG. 5C shows the frame format in the case of 192 Kbps, which is 128 Kbps in the first half.
For the minute, one of the case (B-1) and the case (B-2) is used. Furthermore, FIG. 5 (D) shows 38
Frame format for 4Kbps, 384Kbps
It is used as a video signal until.

In the conventional case, as shown in FIG.
It is fixed at 16 Kbps or 56 Kbps. As described with reference to FIG. 4, for the video signal, the processing in the video encoding unit 41 is controlled by the feedback control. However, for the audio signal, no sound is produced in the sound. Despite the period, 16Kb as above
Transmission capacity of ps or 56Kbps is secured.

 Next, examples of the present invention will be described below.

FIG. 6 is a basic configuration diagram of the present invention, in which a transmitter B and a receiver C are provided.
Composed of.

FIG. 7 shows the transmitting side 1 of the video / audio multiplexing system of the present invention.
In this embodiment, the speech coding unit 2 includes eight adaptive coding units (for example, ADPCM) 2 ₁ to 2 each having a different number of coding bits for digital speech.
_8, and the evaluation unit 2A for determining an encoding bit rate for generating a coded bit rate information on the basis of the reproduction output and the digital sound of each adaptive coding unit 2 ₁ to 2 ₈ to the noise evaluation method, the encoding The selection unit 2B selects the output audio information of the corresponding adaptive encoding unit according to the bit rate information. Here, in the encoding control unit 5, the evaluation unit 2A substantially performs its function. Further, the video encoding unit 4 is an encoding unit (Q) that encodes and quantizes the digital video from the A / D converter 3.
41, a variable-length coding unit (VWLC) 42 for variable-length coding the coded video information, and a buffer 43 for temporarily storing the variable-length coded video information. . Reference numeral 9 is a clock supply unit for transmitting and receiving transmission frames.

Frame (packet) format of the audio information output from the adaptive encoder 2 ₁ to 2 ₈ is as shown in FIG. 8, a frame header a, consists of a set of audio data following the frame header a, It is composed of the number of coding bits k of the adaptive coding unit that has received the optimum noise evaluation × the number of bits of sampling rate n (n = 8 KHz).

Examples of these adaptive encoder 2 ₁ to 2 ₈ speech frames 0-7 bits respectively encoding is shown in FIG. 9, the multiplexer one of these is selected by the selection unit 2B by Sent to.

FIG. 10 shows an embodiment of the voice delay / multiplexing units 6 and 7. In this embodiment, the CPU 21, the CPU bus 22, and the ninth
A ROM 23 that stores a procedure program for sending data to a transmission line in the frame format shown in the figure, and an interface (I / F) 24 for loading video information, audio information, and coded bit rate information into the bus 22 respectively. ~ 26
And a RAM 27 having an address space for temporarily writing video information, audio information, and bit rate information from these interfaces 24-26, and an interface 28 for sending each information to a transmission line. There is.

The transmission frame format shown in FIG. 2 is bit allocation based on CCITT Recommendation Y.221.
Each multi-frame constitutes one unit with 16 transmission frames (1 multi-frame = 16 transmission frames), horizontal 8 bits x
Vertical 80 bits = 640 bits in total make up one transmission frame, and one transmission frame is F as audio information, video information and control information (AC (Application Channel) information).
AS (Frame Alignment Signal) information and BAS (Bit Alloc)
ation Signal) information. Incidentally, one multi-frame is composed of 16 transmission frames as described above, but the data is as shown in FIGS. 2 (a) and 2 (b).
Two transmission frames in the direction of, eg F _{1-1 to} F _1-16 and F
_{2-1 to} F _2-16 are constructed and transmitted as a pair. However, FAS information and BAS information are different in transmission frames F ₁ and F ₂ .

FAS information is frame information and is used to establish synchronization by a synchronization procedure of (1) Y.221 frame synchronization and (2) multi-frame synchronization. That is, it is possible to distinguish each frame by the above (1), and it is possible to identify each frame by (2). The necessity of identifying all frames is to recognize a response unit for a change in BAS (Bit Allocation Signal) information described later.

The BAS information is the coding information of the audio information amount and the video information amount (for example, each coding bit rate) set on the transmitting side. By receiving the coding bit rate information from the evaluation unit 2A, the coding bit rate is obtained. Is used to capture and transmit BAS information as shown in FIG. 11 and to separate each data on the receiving side after frame synchronization is established.
This BAS information is determined for each sub-multiframe (1 multiframe = 2 sub-multiframes), and the 12th
As shown in the figure, the next coding bit rate is detected from the immediately preceding BAS information according to the majority logic (5 frames or more out of 8 frames).

Next, the transmission frame forming operation according to this embodiment will be described.

In the multiplexing unit 6 using the above transmission frame information (FIG. 2), the CPU 21 determines the bit allocation according to the encoding bit rate stored as the BAS information,
As mentioned above, each frame has a BAS
Since it has an “8 × n” bit configuration when it is excluded by including it in the information, frame allocation can be optimally performed as shown in FIG.

That is, for example, if voice data is filled up to 1 to 7 bits (horizontal axis) in the right direction for each 8-bit unit in the vertical direction (for example, shown in FIGS. The audio data is sequentially filled in a form corresponding to the selected coding bit rate.

At this time, the embedding of audio data and video data is 8 vertically.
The video data is embedded in the rest of the buffer 43 as soon as the embedding of the audio data in each stage is completed.

In this way, one transmission frame is formed,
14 to 17 show the relationship between audio data and video data in one transmission frame in the case of 0-bit encoding, 1-bit encoding, 3-bit encoding and 7-bit encoding, respectively. ing.

In this embodiment, since the voice frame header is included in the BAS information, the voice coding rate is variable for each "1 multiframe (= 16 frames)".

In addition, by including the frame header in the BAS information in this way, data can be stored in every 8 bits.
There is an advantage that odd data as shown in FIG. 23 described later does not occur.

The format of the transmission frame when the audio frame header is not included in the BAS information is shown in FIG. 18. In this case, the header is 3-bit information of vertical 1 bit and horizontal 3 bits as shown in the figure (No. 9). (See the figure), and when writing audio / video data in the horizontal direction as shown in FIG.
As shown in FIGS. 23 to 23, one transmission frame is formed for 0-bit encoding, 1-bit encoding, 3-bit encoding, and 7-bit encoding, respectively.

The transmission frame consisting of this audio information and video information is RA
The data is stored in M27, and when the transmission frame is completed (for 640 bits), the data is sent to the transmission line 10 at a rate that matches the transmission rate.

On the other hand, the demultiplexing unit 11 on the receiving side has the same configuration as the multiplexing unit 7 shown in FIG. 10 with the arrow just in the opposite direction.

That is, as described above, the FAS information is analyzed for the input data from the transmission path to establish (1) Y.221 frame synchronization and (2) multi-frame synchronization. Then, each data is separated by the separation unit 11 based on the BAS information, and the separated signal data are encoded bit rate information through the respective interfaces to the decoding control unit 1.
6. The audio information is sent to the audio decoding unit 12, and the video information is sent to the video decoding unit 14.

Then, the audio decoding unit 12 receives the decoding bit rate information from the decoding control unit and decodes the audio information according to the bit rate.

As described above, since the transmission frame used in the present invention has a constant bit length, in order to make the coding rate variable for each transmission frame, the operation synchronized with the transmission line clock rate is performed by the clock supply shown in FIG. This is done by extracting the clock from the unit 9 on the receiving side (in the case of CMI code). However, in RS422 or INCU, the clock is supplied in a separate system.

In the above embodiments is input to the multiplexing unit 6 via the selector 2B each output of the adaptive coding unit 2 ₁ to 2 _8, but in the embodiment of Figure 24, adaptive encoder 2 multiplexer 6 also select information inputs the respective outputs of the ₂₁ to ₈ directly to the multiplexer 6
I am sending it to.

System configuration of the multiplexer 6 for this case is shown in FIG. 25, receives the respective output audio data of the adaptive coding unit 2 ₁ to 2 ₈ via the interface 25 _to 253 _8, the encoding bit rate One of these outputs is selected according to the information and stored in the RAM 27 as voice information.

In the present invention, the adaptive voice encoding unit 2 performs redundant compression by appropriately changing the bit rate according to the information amount of the input voice, and sends the encoded voice information and its bit rate information to the multiplexing unit 6.

As a result, the multiplexing unit 6 multiplexes the audio information and the bit rate information together with the video information from the video coding unit 4. At this time, since the transmission frame from the multiplexing unit 6 has a constant length, The amount of video information can be increased only by the reduction of the bit number due to the compression of the audio information.

The transmission frame thus sent to the receiving side is separated into video information, audio information, and bit rate information by the separating unit 11, and the audio decoding unit 12 decodes the audio information according to the bit rate.

In this way, the data sharing of the audio information amount and the video information amount changes, which is shown in Fig. 26 (1)-
It is shown in (3).

Further, FIG. 27 shows multiplexed transmission data, and as shown in this figure, video / audio data is transmitted in constant frame units, and the audio signal is transmitted in accordance with the amount of information. It can be understood that the number of transmitted data is different and the video data is inserted into the entire empty area, and the video quality is improved.

In the embodiment shown in FIG. 7, the optimum bit rate of the voice from the voice encoding unit 2 is input to the encoding control unit 5 and used as it is as the allocation signal. However, it is the next embodiment whose principle diagram is shown in FIG. 28 that tries to determine the transmission ratio in consideration of the contents of the video information.

 FIG. 29 is a block diagram of the transmitting side of this embodiment.

The speech coding unit 2 includes eight adaptive coding units (for example, AC
PCM) 2 _{1 to} 2 ₈ and the reproduction output of each adaptive coding unit 2 ₁ to 2 ₈ and the digital voice, determine the optimum voice coding bit rate based on the noise evaluation method and generate the voice bit rate information ACI. It is composed of an evaluation unit 2A.

The video encoding unit 4 also includes an encoding unit (Q) 41 that encodes the digital image from the A / D converter 3, and a variable length encoding unit (V) that encodes the encoded video information in a variable length ( VWLC) 42
And a buffer 43 for temporarily storing the variable length coded video information. The inter-frame change rate determination unit 8 stores two frames before and after the output of the A / D converter 3. The frame memory (FM) 81 and the preceding and following two pieces of information are fetched to obtain the rate of change between frames, and the comparison result is compared with the threshold value Th from the encoding control unit 5 to the encoding control unit 5. And a change rate comparison unit 82 for giving the change rate. The threshold value Th is adjusted by the encoding control unit 5 according to the VBI indicating the storage amount of the buffer 43. Moreover, the corresponding adaptation encoded by the video and audio allocation bit rate information MI from the coding control unit 5 to enter the selection and multiplexing unit 6 encoding the output of each adaptive coding unit 2 ₁ to 2 ₈ in parallel It includes a selection unit 6a for selecting output voice information of the unit.

Frame (packet) format of the audio information output from the adaptive encoder 2 ₁ to 2 ₈ is as shown in FIG. 8, a frame header a, consists of a set of audio data following the frame header a, It is composed of the number of coding bits k of the adaptive coding unit that has received the optimum noise evaluation × the number of bits of sampling rate n (n = 8 KHz).

Examples of these adaptive encoder 2 ₁ to 2 ₈ speech frames 0-7 bits respectively encoding is shown in Figure 9, one of these is being multiplexed selected by the selection unit 6a It

FIG. 30 shows a control algorithm of the encoding control unit 5 for generating the video / audio assigned bit rate information MI in response to the comparison result of the change rate comparison unit 82.

In FIG. 30, first, as a result of the comparison in the comparison unit 82, it is checked whether the rate of change in video information between frames is larger than a threshold Th (step S1). When the information greatly changes and exceeds the threshold Th, the video information is given 2 bits (16 kb / s) as shown in step S2 in order to prioritize the video information regardless of the video / audio assigned bit rate information MI (see FIG. 9). 14 bits (112 kb / s) for video information.
Generates and outputs the video / audio allocation bit rate information to which is allocated. The frame structure of 2 bits of audio + 14 bits of video = 16 bits is used in the multiplexing operation of the multiplexer 6.

On the other hand, as a result of the comparison in step S1, when the rate of change in the video information between frames is smaller than the threshold value Th, it is checked whether the accumulated data amount VBI in the buffer 43 exceeds the threshold value Th '(step S3). . As a result, when the accumulated data amount VBI is large enough to exceed the threshold value Th ', the process proceeds to step S2 similarly to the above to preferentially allocate the video information.

When VBI <Th ', the allocated bit rates of video and audio according to the content of the audio bit rate information ACI from the evaluation unit 2A are determined in the following steps S4 to S15.

That is, when the audio bit rate information ACI indicates 2-bit encoding in step S4, the process proceeds to step S2,
When 3-bit encoding is indicated, 13 bits (104 kb / s) are allocated to the video information, so that the audio bit rate and the video bit rate are sequentially determined and the video / audio allocation bit rate information MI is multiplexed. Output to the unit 6.

In the above embodiment, the transmission ratio is set to the optimum bit rate of voice,
Although it was determined from the rate of change between frames of video information and the amount of storage in the buffer, the determination may be made with at least one piece of information, and other information regarding video information and audio information,
For example, the rate of change in video frame or the high frequency ratio of audio may be used.

In the next embodiment, SB-ADPCM (subband ADPCM) is used as the voice encoding unit 2. SB-ADPCM is a method of encoding by dividing it into two frequency bands, and the upper bits (for example, 6 bits out of 8 bits) having a large number of bits are low in the low frequency area where the information density is high. The lower frequency band (2 bits) having a smaller number of bits is allocated to the high frequency band. Therefore, the data can be truncated from the lower order, and each allocation rate can be adaptively adopted (6, 7, 8 bits). As shown in principle in FIG. 31, on the transmitting side, an A / D converter 1 for converting into a voice input digital voice, and an SB-for encoding the digital voice by dividing it into a low frequency bit part and a high frequency bit part ADPCM coding unit 2, A / D converter 3 for converting a video input into a digital video, video coding unit 41 for coding a digital video, and variable length for variable-length coding the coded video information. Coding unit
42, a buffer 43 that temporarily stores the variable length coded video information, a buffer determination unit 44 that determines an audio allocation rate according to the amount of information accumulated in the buffer 43, and an allocation rate according to the allocation rate. The SB-ADPCM coding unit 2 is provided with a multiplexing unit 6 that allocates a part of the video information to the high-frequency bit portion and multiplexes the allocation rate information, the video information, and the audio information.

On the receiving side, a separation unit 11 that separates the multiplexed signal from the transmission path 10 into audio information, video information, and allocation information, and an audio decoding unit that SB-ADPCM-decodes the audio information.
12, a D / A converter 13 for converting the SB-ADPCM-decoded digital audio into an audio signal, a buffer 143 for temporarily storing the video information, and a variable variable-length decoding for the video information in the buffer 143. A long decoding unit 142, an inverse encoding unit 141 that inversely encodes the variable length decoded video information to generate a digital video signal, a D / A converter 15 that converts a digital video output into a video output, A decoding control unit 16 that controls the speech decoding unit 12 according to the allocation rate is provided.

FIG. 31 On the transmission side, the video input is converted into a digital signal by the A / D converter 3, filtered, and then guided to the encoding unit, where the amount of information is compressed using the information of the previous frame, The value obtained there is quantized. The quantized video information is processed by the variable length coding unit (VWLC) 42 so that a short code is assigned to data with a high frequency of occurrence and a long code is assigned to data with a low frequency of occurrence. The buffered data is temporarily stored in the buffer (BUF) 43.

The data in the buffer 43 is multiplexed by the multiplexing unit (MUX) 6 and sent out. When the transmission line capacity is small, the accumulated data amount in the buffer 43 is large, and when the transmission line capacity is large, the accumulated data is large. The quantity will be smaller.

Therefore, the amount of data stored in the buffer 43 allows the amount of data to be sent to the transmission path to be ascertained, and this information is sent to the buffer determination unit 44, and the buffer determination unit 44 uses the information about the data amount It is possible to determine the allocation (coding) rate of the.

Also, the voice input is converted into digital voice by the A / D converter 1 and sent to the SB-ADPCM encoding unit 2.

Therefore, the SB-ADPCM2 encoded output is selected according to the allocation rate determined by the buffer determination unit 44, the necessary number of audio bits are truncated, video data is allocated to the truncated portion, and the multiplexing unit (MUX 6), the audio information, the video information, and the allocation rate information are multiplexed and transmitted.

On the other hand, on the receiving side, the separation unit (DMUX) 11 separates the video information, the audio information, and the allocation rate information.

When the separated voice data is decoded by the SB-ADPCM decoding unit 12, it is possible to use the separated allocation rate information.

In this way, as shown in FIGS. 26 and 27, the audio information amount becomes variable as the amount of video information increases and decreases, and the video information of higher quality can be transmitted.

FIG. 32 shows an embodiment of the buffer judgment unit 44 shown in FIG. 31, and in this embodiment, the variable length coding unit 42
The write address W to the buffer 43 composed of RAM and the read address R to the buffer 43 from the multiplexing unit 6 are input and the following judgment is made.

When WR <threshold value Th ₁ , it is determined that the amount of video information generated is large because the data reading is not progressing with respect to the data writing to the buffer 43, and the SB-ADPCM coding unit is determined. Information about the allocation rate for encoding only 6-bit audio data allocated to the low-frequency component by discarding 2 bits allocated to the high-frequency component of the 8-bit audio encoded data from 0 (0, 1) is sent to the multiplexing unit 6.

When the threshold value Th ₂ >W−R> threshold value Th ₁ , it is determined that the amount of generated video information is medium, and the high-frequency component of the 8-bit audio encoded data from the SB-ADPCM encoding unit 2 is allocated. Information about the allocation rate (0,1) for encoding only 7-bit audio data including 6 bits allocated to the low-frequency component by discarding 1 bit of the 2 bits thus generated is multiplexed. Send to 6.

When the threshold value Th ₂ <W−R, it is determined that the amount of generated video information is small, and the 8-bit audio coding data from the SB-ADPCM coding unit 2 is assigned information for coding without cutting the data. (1,1) is sent to the multiplexing unit 6.

Figure 33 shows an example of bit arrangement of FAS information. Frame synchronization (1) is FAW (Frame Alignment Word)
(In FIG. 33, "0011011") is recognized, and multi-frame synchronization (2) is identified by the information Mi arranged in the first bit of FAS information.
・ Focusing on the information Mi of the 5th, 7th, 9th and 11th frames,
Synchronization is established by the pattern of "001011".

FIG. 34 shows an example of BAS information, which is generated on the transmission side so as to set the ratio between the audio information amount and the video information amount based on the above-mentioned allocation rate ( (Refer to FIG. 12), which is information used for separating each data on the receiving side after frame synchronization is established, and the processing change unit of data separation is one multi-frame or one
For each sub-multiframe (1 multiframe = 2 sub-
Multi-frame), and the BAS information is determined for each sub-multi-frame, and as shown in FIG.
When the change in the S information is recognized by the majority decision by the demultiplexing unit 11 (5 frames or more out of 8 frames match), the speech / voice in the next multi-frame or sub-multi-frame is detected.
It is possible to recognize the separation position of the video data. This is because the BAS information also changes at the time when the audio-video allocation changes, so that it is determined which BAS information should be used within one frame. The unit of audio / video data separation is 80 bits.

Next, the operation will be described. In the multiplexing unit 6 in the case of using the frame format as described above, based on the allocation rate information output from the encoding control unit 5 by the CPU 21, as shown in FIG. Depending on the BAS information, when 8 bits of audio data is used as it is ((a) in the same figure) and when 7 bits are used after truncating 1 bit ((b) in the same figure)
2 bits are cut off and only 6 bits of the low frequency component are used ((c) in the figure), the bit allocation is determined, and the audio information and the video information are stored in the RAM 27.

Then, when the frame format of FIG. 35 is completed, the data is sent to the transmission line 10 at a rate that matches the transmission rate.

Therefore, the transmission rate of the voice information in FIG. 35 by each bit allocation per frame is as follows.

(A) 6 x 80 (480 bits) x 100 frames (1 second) =
48 kb / s (b) 7 x 80 (560 bits) x 100 frames (1 second) =
56kb / s (c) 8 x 80 (640 bits) x 100 frames (1 second) =
64 kb / s On the other hand, the separating unit 11 on the receiving side has the same configuration as the multiplexing unit 6 shown in FIG.

That is, as described above, the FAS information is analyzed for the input data from the transmission path to establish (1) Y.221 frame synchronization and (2) multi-frame synchronization. Then, each data is separated by the separation unit 11 based on the BAS information, the separated signal data, the audio information to the SB-ADPCM decoding unit 12 via each interface, and the video information to the buffer 143 and variable length. Send to the decoding unit 142.

Here, the separated audio information is the transmission line clock (8kH
(z, 64kHz), it is necessary to output data according to the number of audio allocation bits as shown in Fig. 36 at the timing synchronized with this. Therefore, the audio information is the allocation rate information described above included in the BAS information. Based on the SB-ADPCM decoding unit 12 and output as an analog signal by the D / A converter 13, the video information is decoded from the buffer 143 by the variable length decoding unit 142, and by the video decoding unit 141. The data is decoded, converted into an analog video signal by the D / A converter 15, and output.

In the above description, the ADPCM-MQ system and the SB-ADPCM system can be used as the adaptive voice encoding unit, but in addition to this, the adaptive voice encoding unit includes the DPCM system, the APC-AB system, and the like. It is necessary to confirm the match / mismatch of the transmission / reception methods on the transmission side and the reception side.

Therefore, in order to understand the mutual capabilities of the devices and search for a common operation mode, as shown in Fig. 37, after the power is turned on by the ICP (In-Channel Connection Protocol) procedure, the frame synchronization is established by the FAS information. To do.

As shown in FIG. 38, the mutual capability, that is, the encoding method is inquired by 8 bits in the 64-bit AC information.

A common encoding method based on the inquiry result is BAS
Specify using information. (However, if there is no common system, a procedure for communication at a fixed coding rate is executed.) In this way, automatic interconnection can be performed and conflict with the conventional device can be avoided. .

Next, a description will be given of an embodiment for correcting the deviation between the audio output and the video output due to the difference in processing speed between the encoding and the decoding of the audio and the video.

As shown in principle in FIG. 39, an A / D converter 1 which is composed of a transmitting unit D and a receiving unit E and converts a voice input into a digital voice at the transmitting side, and the digital voice is encoded. An audio encoding unit 2, an A / D converter 3 for converting an image input into a digital image, an image encoding unit 4 for encoding a digital image, and an image reproduction output based on input / output information of the image encoding unit 4. A delay amount calculation unit 31 that generates video coding delay time information for synchronizing the audio reproduction output, and a multiplexing unit 6 that multiplexes the coded video information and audio information with the delay time information. ing.

Also, on the receiving side, a separation unit 11 that separates the multiplexed signal from the transmission path 10 into audio information, video information, and delay time information, and a variable delay control unit 32 that delays the audio information according to the delay time information. An audio decoding unit 12 for decoding the audio information from the variable delay control unit 32, a D / A converter 13 for converting the decoded digital audio into an audio signal, and a video decoding for decoding the video information. A conversion unit 14, a D / A converter 15 for converting the decoded digital video output into a video output,
It has.

As shown in FIG. 39, a fixed processing delay in the audio signal encoding unit 2 and the decoding unit 12 with respect to the delay time (Tv: variable amount) required for the encoding process in the image signal encoding unit 4 Since the fixed processing delay time (Ta: constant amount) obtained by subtracting the fixed processing delay time in the video signal decoding unit 14 from the time is short, the audio decoding unit 1
The delay time (Td: variable) required for the variable delay control unit 32 in order that 2 and the video decoding unit 14 generate outputs at the same time is Td = T
given by v-Ta.

Therefore, since Ta is constant in the delay information Td, it is understood that Tv may be obtained.

Therefore, in the present invention, the delay amount calculation unit 31 calculates the delay time Tv of the encoding process based on the input / output information of the video encoding unit 4, and this delay time Tv and a predetermined delay time Ta known in advance. From the above, information about the above-mentioned delay time Td is given to the mulling unit 6, and this delay time information is multiplexed with video information and audio information and transmitted (see FIG. 40).

On the other hand, on the receiving side, when the demultiplexing unit 11 separates the delay time information and gives it to the variable delay control unit 32, the variable delay control unit 32 delays the separated voice information by the delay time Td and performs voice decoding. Give to part 12.

Since the adaptive delay control is performed according to the input video in this way, the audio decoding output and the video decoding output can be output in a matched form.

FIG. 41 shows an embodiment of the delay amount calculation unit 31 shown in FIG. 39. In this embodiment, a buffer (BUF) 2a for accumulating video information from the A / D converter 1 and a buffer are provided. Selector (SEL) 2b for selecting the video information or the evaluation pattern from 2a based on the select signal, the video information amount detection unit 2c for detecting the video information amount using the memory 35, and receiving the above video information amount. Operation control unit 2d that generates an evaluation pattern and a select signal and receives the evaluation pattern from the video encoding unit 4
And

In the operation, the arithmetic control unit 2d, for example, based on the change in the output of the video information amount detection unit 2c, the temporally adjacent video information amount difference (T1-T0) becomes a certain threshold value (Th) or more. In this case, the process shifts to a sequence for calculating the processing delay time T _VC of the video encoding unit 4.

At the time of this calculation, since the select signal is given to the selector 2b from the calculation control section 2d, the selector 2b operates as a buffer.
The video data of 2a is switched to the evaluation pattern, and the evaluation pattern is output.
Sent to

Then, the processing delay time T _VC is calculated by calculating the time from the time when this evaluation pattern is sent to the time when the evaluation pattern is output from the encoding unit 4.

As the evaluation pattern, a pattern that does not exist in the video data is used, and when the evaluation pattern is input in the encoding unit 4, it is output in an unprocessed state. 00000000000000 "
To use.

Also, while sending the evaluation pattern, the video data is buffered.
It is stored in 2a and its reading is prohibited, so the video data is not discarded.

From the delay time T _VC calculated in this way, the calculation control unit 2d determines the delay time Td for further synchronization. This is obtained from Td = _TVC + _TVd - _TaC - _Tad . Where T _Vd is the video decoding delay time, T
_aC is the audio coding delay time, T _ad is the audio decoding delay time, and Ta = T _Vd -T _aC -T _ad can be considered to be constant compared to the video coding delay time T _VC , so Ta By delaying in advance, the delay time Td can be obtained and sent to the multiplexing unit 6.

FIG. 42 shows an embodiment of the multiplexing unit 6, and in this embodiment, a CPU 21, a CPU bus 22, and a procedure program for sending data to the transmission line in the frame format shown in FIG. The stored ROM 23 and the video information, the audio information, and the delay time information from the delay amount calculation unit 31 are respectively transferred to the bus 22.
Interfaces (I / F) 24 to 26 for capturing in, and video information and audio information from these interfaces 24 to 26,
And a RAM 27 having an address space for temporarily writing the delay information, and an interface 28 for sending each information to the transmission path.

The transmission frame format is as shown in FIG. 2, and as shown in FIG. 6, FAS information is frame information, including (1) Y.221 frame synchronization, and (2) multi-frame.
Used to establish synchronization by the frame synchronization synchronization procedure. That is, it is possible to distinguish each frame by the above (1), and it is possible to identify each frame by (2).
Note that the necessity of identifying all frames is based on the BAS (Bit All
This is to recognize the response unit for changes in information.

The BAS information is information in which the encoded information of the audio information amount and the video information amount (for example, the ratio of both) is preset on the transmitting side, and is used for separating each data on the receiving side after the frame synchronization is established. This BAS information is determined for each sub-multiframe (1 multiframe = 2 sub-multiframes).

In addition, the delay time information from the delay amount calculation unit 31 uses the eight frames (CODEC function unit) of DLY0 to 7 for the control information (AC information) unit shown in FIG. Information about Td is indicated, and by these 8 bits,
It is possible to send 256 types of audio delay information to the receiving side device.

Next, the operation will be described. In the multiplexing unit 6 in the case of using the frame format as described above, the bit allocation is determined by the CPU 21 according to the BAS information stored in advance, and the audio information and the video information are stored in the RAM 27. . Then, at this time, the delay time information obtained by the delay amount calculation unit 31 is stored as control information. Then, when the frame format is completed, the data is sent to the transmission line 10 at a rate that matches the transmission rate.

On the other hand, the demultiplexing unit 11 on the receiving side has the same configuration as the multiplexing unit 6 shown in FIG. 42 with the arrow just in the opposite direction.

That is, as described above, the FAS information is analyzed for the input data from the transmission path to establish (1) Y.221 frame synchronization and (2) multi-frame synchronization. Then, each data is separated by the separation unit 11 based on the BAS information, and each separated signal data is sent to the variable delay control unit 32 through the respective interfaces and the video decoding unit 14 through the video information. Send to. Further, the delay time information in the control information is similarly separated and sent to the variable delay control section 32.

The variable delay control unit 32 receives the delay time information separated by the separation unit 11, delays the voice information by the delay time Td, and sends it to the voice decoding unit 12.

As another method of determining the delay time, prepare several kinds of delay tables in advance as shown below, and select the most appropriate value according to the amount of video information generation and set it as the video coding delay time T _VC. You can also

<Table example> Intra-frame coding: T _VC = 500 msec Inter-frame coding (1) Large amount of generated information: T _VC = 250 msec (2) Large amount of generated information: T _VC = 200 msec (3) Large amount of generated information: T _VC = 150 msec Also, except for the variable delay time T _V of the video encoding process, the delay amount calculation unit 31 on the transmission side was prepared in advance as fixed delay time. Td
May be requested.

As described above, when the transmission ratio is changed depending on the contents of the video information and the audio information, this deviation becomes small, but it still remains to some extent. Therefore, by changing the transmission ratio of video and audio, and further giving a delay time corresponding to the input information for reproduction, a better balance between video and audio can be maintained.

Next, an example will be described in which the delay time of the audio of the video is used to compress the silent part of the audio and collectively transmit the time corresponding to the delay time.

FIG. 44 shows the basic configuration of this embodiment. In the figure, symbol A is a terminal station, 3 is an A / D converter for video signals, 41 is a video coding unit, 42 is a variable length coding unit, and 1 is a voice signal.
A / D converter, 2 is a voice encoder, 6 and 11 are multiplexer / demultiplexer,
9 is a transmission line interface unit, 142 is a variable length decoding unit for video signals, 141 is a video decoding unit, 15 is a D / A converter,
12 is a voice decoding unit, 13 is a D / A converter, 19 is a system control unit, 71 is a time division encoding unit, and 72 is a time division decoding unit.

The video signal is subjected to the variable length coding process after the predictive coding process in the video coding unit 41, and the multiplexing / demultiplexing units 6 and 11 are provided.
Is supplied to. On the other hand, the voice signal is coded in the 4 kHz band 16 Kbps or the 7 kHz band 56 Kbps in the voice coding unit 2, and the time division coding unit 71 extracts the signal only during the effective period from which the silent period in the voice signal is removed.
After being packetized, the packets are bundled and supplied to the multiplexing / demultiplexing units 6 and 11 for a period substantially matching the desired delay control amount T.

Then, the video signal and the audio signal are multiplexed and transmitted from the transmission path interface unit 68 to the opposite terminal station.

For the received signal from the opposite terminal station, the multiplexing / dividing unit 6,
At 11, the video signal and the audio signal are separated. Regarding the video signal, the variable length decoding unit 142 and the video decoding unit 14
1. Received as a video output from the D / A converter 15. The audio signal is received as an audio output via the time division decoding unit 72, the audio decoding unit 12, and the D / A converter 13.

The time division encoding unit 71 extracts the signals of the valid period from which the silent period is removed from the voice signal based on the output from the voice encoding unit 2 and collects the signals into packets, and the packets that have been collected into the packets are described above. A time corresponding to the delay control amount T is collectively supplied to the multiplexing / demultiplexing units 6 and 11 in a unit of 8 Kbps for a minimum of 0 to a maximum of 7 (56 Kbps). Then, the time division encoding unit 71 notifies the system control unit 19 and the demultiplexing / demultiplexing units 6 and 11 which one of 0 Kbps to 56 Kbps that corresponds to the audio transmission rate is output.

Based on this notification, the system control unit 19 controls the threshold value (second threshold value) when controlling the encoding process in the video encoding unit 41 based on the data amount in the buffer memory in the variable length encoding unit 42. To change. That is, the transmission amount of the video signal increases as the transmission amount of the audio signal decreases.

On the other hand, the demultiplexing / demultiplexing units 6 and 11 receive the transmission rate of the voice signal, and include in the frame information the information for notifying the opposite terminal station of the transmission rate of the voice signal in the multiplexed frame format. Try to describe it.

FIG. 45 (A) shows the transmission frame format in the case of the embodiment, and FIG. 45 (B) is a diagram for explaining the constituent frame information.

8 Kbps is allocated to the frame information, and the frame information includes a frame header and also has constituent frame information for notifying the opposite terminal station of the voice transmission rate. As shown in FIG. 45 (B), the constituent frame information is the information "000" indicating the audio transmission rate collectively supplied to the multiplexing / dividing units 6 and 11 within the range of 0 Kbps to 56 Kbps in the time division coding unit 41. ”Through“ 111 ”.

In the case of the present invention, the first
As shown as "Variable" in Fig. 45 (A),
Minimum 0Kbps to 56Kbps for audio signal transmission
The categories up to are assigned and transmitted. For voice signals
Given a Kbps, the remaining (56-a) Kbps in the figure
Are used for transmitting video signals.

FIG. 46 shows a processing mode in the time division encoding unit.

If a voice input as shown in the figure is given, the 44th
It is digitized by the A / D converter 1 shown in the figure and becomes an A / D output. In the figure, "1", "2" and "3" are signals during the valid period. The A / D output has the silent period deleted as shown in the figure, and is summarized as the packetized output in the figure.

The packetized output is shown at (a), (b), (c) ...> at predetermined time intervals t _S as shown in the illustrated packet information.
Is generated as follows. Such information (a),
(B), (c), etc. are collected for a time corresponding to the delay control amount T described above, and 0 to 0 are set in units of 8 Kbps as input data to the multiplexing / demultiplexing unit shown in the figure. It is supplied to the multiplexing / demultiplexing units 6 and 11 as seven pieces. The above-mentioned voice transmission rate is given depending on the number of the collected (a), (b) ...>. That is, when the number is 0, “000” shown in FIG.
"001" is generated when there are 7 pieces, ..., "1" when there are 7 pieces
11 ”is generated.

FIG. 47 shows a mode of feedback control in the case of the present invention.

The voice transmission rate (encoding rate shown in the figure) generated in the time division encoding section 41 is supplied to the system control section 19 as a second threshold value like the threshold value 2'in the present invention. The first threshold is given as a fixed value as in the conventional case.

In the system control unit 19, as shown in FIG. 47 (B), the condition for instructing the video encoding unit 41 to stop the encoding process is changed based on the audio transmission rate (encoding rate). Alternatively, change the quantization table. That is, the higher the voice transmission speed, the smaller the amount of data stored in the buffer memory 43, and the more the stop instruction is given. Alternatively, change the quantization table to a rough one. Of course, conversely, the lower the voice transmission speed, the more the data stored in the buffer memory 43 becomes larger, and the stop is instructed. Alternatively, the quantization table can be made more detailed.

“Control A” shown in the figure is a control signal for the video encoding unit 3, and “Control B” in the figure is a control signal for notifying the multiplexing / demultiplexing units 6 and 11 of the audio transmission rate.

[Potential for industrial use]

INDUSTRIAL APPLICABILITY The present invention is used in a video conference system or the like that uses a video / audio multiplex transmission system, and is more effective in a relatively low-order system where the transmission capacity cannot be said to be sufficient. It can be applied.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (31) Priority claim number Japanese Patent Application No. 1-66783 (32) Priority date Hei 1 (1989) March 18 (33) Priority claiming country Japan (JP) (31) Priority Claim number Japanese Patent Application No. 1-178454 (32) Priority date 1 (1989) July 11 (33) Country of priority claim Japan (JP) (56) References Japanese Patent Laid-Open No. 64-16040 (JP, A) Kai 63-4042 (JP, A) JP 55-136741 (JP, A) JP 60-144037 (JP, A)

Claims (41)

(57) [Claims] 1. An A / D for converting a voice input into a digital voice.
A converter (1), an audio encoding unit (2) that encodes the digital audio and outputs it as encoded audio with a selectable transmission amount, and outputs audio content information. A / D converter to convert to (3)
A video encoding unit (4) for encoding the digital video and outputting the encoded video information as encoded video information, the encoded audio and the encoded audio according to the information amount of at least one of the encoded audio and the encoded video information. Encoding control unit (5) that determines the transmission ratio of encoded video and outputs it as an allocation signal
And a multiplexing unit (6) that multiplexes the coded audio based on the allocation signal, the coded video, and control information including the allocation signal so as to have a constant transmission frame length. A video-audio multiplex transmission system having a transmitter (B), wherein the audio encoder (2) encodes the digital audio by dividing the digital audio into a low frequency bit part and a high frequency bit part.
A video / audio multiplex transmission system, characterized in that a CM encoding unit selects an allocation amount of a high frequency bit portion according to the allocation signal. 2. An A / D for converting a voice input into a digital voice.
A converter (1), an audio encoding unit (2) that encodes the digital audio and outputs it as encoded audio with a selectable transmission amount, and outputs audio content information. A / D converter to convert to (3)
A video encoding unit (4) for encoding the digital video and outputting the encoded video information as encoded video information, the encoded audio and the encoded audio according to the information amount of at least one of the encoded audio and the encoded video information. Encoding control unit (5) that determines the transmission ratio of encoded video and outputs it as an allocation signal
And a multiplexing unit (6) that multiplexes the coded audio based on the allocation signal, the coded video, and control information including the allocation signal so as to have a constant transmission frame length. A video / audio multiplex transmission system having a transmission section (B), wherein the transmission section obtains an inter-frame information change rate of the digital video and compares it with a threshold value, and a determination result is obtained by the encoding control section (5).
A video / audio multiplex transmission system, further comprising an inter-frame change rate determination unit (8) for outputting to. 3. An A / D for converting a voice input into a digital voice.
A converter (1), an audio encoding unit (2) that encodes the digital audio and outputs it as encoded audio with a selectable transmission amount, and outputs audio content information. A / D converter to convert to (3)
A video encoding unit (4) for encoding the digital video and outputting the encoded video information as encoded video information, the encoded audio and the encoded audio according to the information amount of at least one of the encoded audio and the encoded video information. Encoding control unit (5) that determines the transmission ratio of encoded video and outputs it as an allocation signal
And a multiplexing unit (6) that multiplexes the coded audio based on the allocation signal, the coded video, and control information including the allocation signal so as to have a constant transmission frame length. A video / audio multiplex transmission system having a transmitting section (B), wherein the video encoding section (4) encodes the digital video, and the encoded video information. A variable length coding unit (42) for variable length coding, a buffer (43) for temporarily storing the variable length coded video information, and a buffer (43) for storing the amount of information stored in the buffer (43). And a buffer determination section (44) for outputting a storage amount signal to the encoding control section (5). 4. An A / D for converting a voice input into a digital voice.
A converter (1), an audio encoding unit (2) that encodes the digital audio and outputs it as encoded audio with a selectable transmission amount, and outputs audio content information. A / D converter to convert to (3)
A video encoding unit (4) for encoding the digital video and outputting the encoded video information as encoded video information, the encoded audio and the encoded audio according to the information amount of at least one of the encoded audio and the encoded video information. Encoding control unit (5) that determines the transmission ratio of encoded video and outputs it as an allocation signal
And a multiplexing unit (6) that multiplexes the coded audio based on the allocation signal, the coded video, and control information including the allocation signal so as to have a constant transmission frame length. A video / audio multiplex transmission system having a transmission unit (B), wherein the transmission unit (B) delays from the input / output information of the video encoding unit (4) to synchronize the video reproduction output and the audio reproduction output. Further comprising a delay amount calculation unit (31) for generating time information,
A video / audio multiplex transmission system characterized in that the delay time information is also multiplexed by a multiplexing unit (6). 5. A separation unit (11) for receiving and separating a signal obtained by multiplexing coded audio from a transmission line (10), coded video, and a control signal including an allocation signal, and the coded audio. And a video decoding unit (14) for decoding the encoded video into a decoded digital video, a video decoding unit (14) for decoding the encoded video into a decoded digital video, and the audio decoding unit (14) based on the allocation signal. 12) and a decoding control section (16) for controlling the video decoding section (14), a D / A converter (13) for converting the decoded digital voice into an audio signal, and the decoded digital video A video / audio multiplex transmission system having a receiving section (C) comprising a D / A converter (15) for converting a video signal into a video signal, wherein the receiving section (C) is a multiplexed signal from a transmission line. Variable delay control section (32) for delaying the voice information according to the delay time information multiplexed in Further video and audio multiplex transmission system characterized by comprising. 6. An A / D for converting a voice input into a digital voice.
A converter (1), an audio encoding unit (2) that encodes the digital audio and outputs it as encoded audio with a selectable transmission amount, and outputs audio content information. A / D converter to convert to (3)
A video encoding unit (4) for encoding the digital video and outputting the encoded video information as encoded video information, the encoded audio and the encoded audio according to the information amount of at least one of the encoded audio and the encoded video information. Encoding control unit (5) that determines the transmission ratio of encoded video and outputs it as an allocation signal
And a multiplexing unit (6) that multiplexes the coded audio based on the allocation signal, the coded video, and control information including the allocation signal so as to have a constant transmission frame length. A transmission section (B), a separation section (11) for receiving and separating a signal obtained by multiplexing encoded voice and encoded video from a transmission line (10) and a control signal including an allocation signal; An audio decoding section (12) for decoding the encoded audio into a decoded digital audio, a video decoding section (14) for decoding the encoded video into a decoded digital video, and the audio decoding based on the allocation signal A decoding control section (16) for controlling the section (12) and the video decoding section (14), a D / A converter (13) for converting the decoded digital voice into an audio signal, and the decoding With a receiver (C) including a D / A converter (15) for converting a digital image into a video signal Video that is transmitted by changing the transmission ratio of audio and video according to the configured transmission contents.
A voice multiplex transmission system, wherein the voice encoding unit (2) encodes the digital voice by dividing the digital voice into a low frequency bit portion and a high frequency bit portion.
In the CM encoding unit, the allocation amount of the high frequency bit portion is selected by the allocation signal, and the speech decoding unit SB-codes the encoded speech according to the allocation signal.
-A video / multiplex transmission system characterized by ADPCM decoding. 7. An A / D for converting a voice input into a digital voice.
A converter (1), an audio encoding unit (2) that encodes the digital audio and outputs it as encoded audio with a selectable transmission amount, and outputs audio content information. A / D converter to convert to (3)
A video encoding unit (4) for encoding the digital video and outputting the encoded video information as encoded video information, the encoded audio and the encoded audio according to the information amount of at least one of the encoded audio and the encoded video information. Encoding control unit (5) that determines the transmission ratio of encoded video and outputs it as an allocation signal
And a multiplexing unit (6) that multiplexes the coded audio based on the allocation signal, the coded video, and control information including the allocation signal so as to have a constant transmission frame length. A transmission section (B), a separation section (11) for receiving and separating a signal obtained by multiplexing encoded voice and encoded video from a transmission line (10) and a control signal including an allocation signal; An audio decoding section (12) for decoding the encoded audio into a decoded digital audio, a video decoding section (14) for decoding the encoded video into a decoded digital video, and the audio decoding based on the allocation signal A decoding control section (16) for controlling the section (12) and the video decoding section (14), a D / A converter (13) for converting the decoded digital voice into an audio signal, and the decoding With a receiver (C) including a D / A converter (15) for converting a digital image into a video signal Video that is transmitted by changing the transmission ratio of audio and video according to the configured transmission contents.
An audio multiplex transmission system, wherein the transmission unit obtains an inter-frame information change rate of the digital video and compares it with a threshold value, and a determination result is obtained by the encoding control unit (5).
A video / audio multiplex transmission system, further comprising an inter-frame change rate determination unit (8) for outputting to. 8. An A / D for converting a voice input into a digital voice.
A converter (1), an audio encoding unit (2) that encodes the digital audio and outputs it as encoded audio with a selectable transmission amount, and outputs audio content information. A / D converter to convert to (3)
A video encoding unit (4) for encoding the digital video and outputting the encoded video information as encoded video information, the encoded audio and the encoded audio according to the information amount of at least one of the encoded audio and the encoded video information. Encoding control unit (5) that determines the transmission ratio of encoded video and outputs it as an allocation signal
And a multiplexing unit (6) that multiplexes the coded audio based on the allocation signal, the coded video, and control information including the allocation signal so as to have a constant transmission frame length. A transmission section (B), a separation section (11) for receiving and separating a signal obtained by multiplexing encoded voice and encoded video from a transmission line (10) and a control signal including an allocation signal; An audio decoding section (12) for decoding the encoded audio into a decoded digital audio, a video decoding section (14) for decoding the encoded video into a decoded digital video, and the audio decoding based on the allocation signal A decoding control section (16) for controlling the section (12) and the video decoding section (14), a D / A converter (13) for converting the decoded digital voice into an audio signal, and the decoding With a receiver (C) including a D / A converter (15) for converting a digital image into a video signal Video that is transmitted by changing the transmission ratio of audio and video according to the configured transmission contents.
An audio multiplex transmission system, wherein the video encoding unit (4) encodes the digital video, and a variable length code that performs variable length encoding on the encoded video information. An encoding unit (42), a buffer (43) for temporarily accumulating the variable-length-encoded video information, and an encoding control of an accumulation amount signal according to the amount of information accumulated in the buffer (43). A video / audio multiplex transmission system, comprising: a buffer determination section (44) for outputting to a section (5). 9. An A / D for converting a voice input into a digital voice.
A converter (1), an audio encoding unit (2) that encodes the digital audio and outputs it as encoded audio with a selectable transmission amount, and outputs audio content information. A / D converter to convert to (3)
A video encoding unit (4) for encoding the digital video and outputting the encoded video information as encoded video information, the encoded audio and the encoded audio according to the information amount of at least one of the encoded audio and the encoded video information. Encoding control unit (5) that determines the transmission ratio of encoded video and outputs it as an allocation signal
And a multiplexing unit (6) that multiplexes the coded audio based on the allocation signal, the coded video, and control information including the allocation signal so as to have a constant transmission frame length. A transmission section (B), a separation section (11) for receiving and separating a signal obtained by multiplexing encoded voice and encoded video from a transmission line (10) and a control signal including an allocation signal; An audio decoding section (12) for decoding the encoded audio into a decoded digital audio, a video decoding section (14) for decoding the encoded video into a decoded digital video, and the audio decoding based on the allocation signal A decoding control section (16) for controlling the section (12) and the video decoding section (14), a D / A converter (13) for converting the decoded digital voice into an audio signal, and the decoding With a receiver (C) including a D / A converter (15) for converting a digital image into a video signal Video that is transmitted by changing the transmission ratio of audio and video according to the configured transmission contents.
A voice multiplex transmission system, wherein the transmission unit (B) generates delay time information for synchronizing the video reproduction output and the audio reproduction output from the input / output information of the video encoding unit (4). (31) is further provided,
A video / audio multiplex transmission system characterized in that the delay time information is also multiplexed by a multiplexing unit (6). 10. A / A for converting voice input into digital voice
A D converter (1), an audio encoding unit (2) for encoding the digital audio, outputting the encoded audio with a selectable transmission amount, and outputting audio content information, and a digital input for the video input. A / D converter for converting to video (3)
A video encoding unit (4) for encoding the digital video and outputting the encoded video information as encoded video information, the encoded audio and the encoded audio according to the information amount of at least one of the encoded audio and the encoded video information. Encoding control unit (5) that determines the transmission ratio of encoded video and outputs it as an allocation signal
And a multiplexing unit (6) that multiplexes the coded audio based on the allocation signal, the coded video, and control information including the allocation signal so as to have a constant transmission frame length. A transmission section (B), a separation section (11) for receiving and separating a signal obtained by multiplexing encoded voice and encoded video from a transmission line (10) and a control signal including an allocation signal; An audio decoding section (12) for decoding the encoded audio into a decoded digital audio, a video decoding section (14) for decoding the encoded video into a decoded digital video, and the audio decoding based on the allocation signal A decoding control section (16) for controlling the section (12) and the video decoding section (14), a D / A converter (13) for converting the decoded digital voice into an audio signal, and the decoding With a receiver (C) including a D / A converter (15) for converting a digital image into a video signal Video that is transmitted by changing the transmission ratio of audio and video according to the configured transmission contents.
A voice multiplex transmission system, wherein the receiving section (C) further comprises a variable delay control section (32) for delaying the voice information according to delay time information multiplexed in a multiplexed signal from a transmission line. Video / audio multiplex transmission system characterized by. 11. An allocated bit in which the audio encoding unit (2) outputs an encoded voice having a different encoding bit rate, and one of the encoded voices is output as the assigned signal. Selected based on the rate, and the transmitting unit obtains the inter-frame information change rate of the digital video and compares it with a threshold value, and the determination result is the encoding control unit (5).
The video / audio multiplex transmission system according to any one of claims 1 to 4, further comprising an inter-frame change rate determination unit (8) for outputting to. 12. An allocated bit in which the speech coding unit (2) outputs a plurality of coded speeches of different coding bit rates, and one of the plurality of coded speeches is output as the allocation signal. A video coding unit (41) selected based on a rate, and the video coding unit (4) coding the digital video, and a variable length code coding a variable length of the coded video information. An encoding unit (42), a buffer (43) for temporarily accumulating the variable-length-encoded video information, and an encoding control of an accumulation amount signal according to the amount of information accumulated in the buffer (43). The video / audio multiplex transmission system according to any one of claims 1 to 4, which comprises a buffer determination unit (44) for outputting to a unit (5). 13. An allocated bit in which the audio encoding unit (2) outputs an encoded signal having a plurality of encoded bit rates and one of the encoded audio signals is output as the assigned signal. A delay amount calculation unit which is selected based on a rate and which generates delay time information for the transmission unit (B) to synchronize the video reproduction output and the audio reproduction output from the input / output information of the video encoding unit (4). (31) is further provided,
The video / audio multiplex transmission system according to any one of claims 1 to 4, wherein the delay time information is also multiplexed together by the multiplexing unit (6). 14. The video / audio multiplex transmission system according to claim 5, wherein said decoding control unit (16) performs control based on the allocated bit rate which is the separated allocation signal. 15. An assigned bit in which the speech coding unit (2) outputs a plurality of coded speeches having different coding bit rates, and one of the plurality of coded speeches is output as the assigned signal. The decoding control unit (16) controls based on the allocated bit rate that is the separated allocation signal, and the transmission unit selects the inter-frame information change rate of the digital video. The encoding control unit (5) obtains the comparison result with the threshold value and determines the determination result.
The video / audio multiplex transmission system according to any one of claims 6 to 10, further comprising an inter-frame change rate determination unit (8) for outputting to. 16. An assigned bit in which the speech coding unit (2) outputs a plurality of coded speeches having different coding bit rates, and one of the plurality of coded speeches is output as the assigned signal. The decoding control unit (16) performs control based on the allocated bit rate that is the separated allocation signal, and the video encoding unit (4) selects the digital video. A video coding unit (41) for coding, a variable length coding unit (42) for variable length coding the coded video information, and temporarily storing the variable length coded video information 11. A buffer (43) and a buffer determination unit (44) for outputting a storage amount signal to the encoding control unit (5) according to the amount of information stored in the buffer (43). The video / audio multiplex transmission system described in any one of 1. 17. An allocated bit in which the speech coding unit (2) outputs a plurality of coded speeches having different coding bit rates, and one of the plurality of coded speeches is output as the allocation signal. The decoding control unit (16) performs control based on the allocated bit rate which is the separated allocation signal, and the transmission unit (B) selects the video encoding unit (4). ) Further comprises a delay amount calculation unit (31) for generating delay time information for synchronizing the video reproduction output and the audio reproduction output from the input / output information of
The video / audio multiplex transmission system according to any one of claims 6 to 10, wherein the delay time information is also multiplexed by the multiplexing unit (6). 18. The audio encoding unit (2) outputs a signal having an optimum audio bit rate, and the transmitting unit obtains an inter-frame information change rate of the digital video and compares it with a threshold value. The encoding control unit (5)
The video / audio multiplex transmission system according to any one of claims 11 to 13, further comprising an inter-frame change rate determination unit (8) for outputting to. 19. A video encoding unit (41) for outputting a signal of an optimum audio bit rate by the audio encoding unit (2) and encoding the digital image by the video encoding unit (4). A variable length coding unit (42) for variable length coding the coded video information, a buffer (43) for temporarily storing the variable length coded video information, and the buffer (43 The video signal according to any one of claims 11 to 13, which comprises a buffer determination unit (44) for outputting a storage amount signal to the encoding control unit (5) according to the amount of information stored in Audio multiplex transmission system. 20. The audio encoding unit (2) outputs a signal having an optimum audio bit rate, and the transmission unit (B) outputs a video reproduction output from input / output information of the video encoding unit (4). A delay amount calculation unit (31) for generating delay time information for synchronizing the audio reproduction output,
The video / audio multiplex transmission system according to any one of claims 11 to 13, wherein the delay time information is also multiplexed together in the multiplexing unit (6). 21. The encoding control section (5) outputs the optimum audio bit rate as the allocation signal as it is, and the transmission section (B) outputs from the input / output information of the video encoding section (4). A delay amount calculation unit (31) for generating delay time information for synchronizing the video reproduction output and the audio reproduction output,
21. The video / audio multiplex transmission system according to any one of claims 18 to 20, wherein the delay time information is also multiplexed together by the multiplexing unit (6). 22. The audio encoding unit (2) outputs a signal having an optimum audio bit rate, and the transmitting unit obtains an inter-frame information change rate of the digital video and compares it with a threshold value. The encoding control unit (5)
The video / audio multiplex transmission system according to any one of claims 15 to 17, further comprising an inter-frame change rate determination unit (8) for outputting to. 23. A video encoding unit (41) for outputting the signal of the optimum audio bit rate by the audio encoding unit (2) and encoding the digital image by the video encoding unit (4). A variable length coding unit (42) for variable length coding the coded video information, a buffer (43) for temporarily storing the variable length coded video information, and the buffer (43 And a buffer determination unit (44) for outputting a storage amount signal to the encoding control unit (5) according to the amount of information stored in the video image according to any one of claims 15 to 17. Audio multiplex transmission system. 24. The audio encoding unit (2) outputs a signal of an optimal audio bit rate, and the transmitting unit (B) outputs a video reproduction output from the input / output information of the video encoding unit (4). A delay amount calculation unit (31) for generating delay time information for synchronizing the audio reproduction output,
The video / audio multiplex transmission system according to any one of claims 15 to 17, wherein the delay time information is also multiplexed together by the multiplexing unit (6). 25. The encoding control unit (5) outputs the optimum audio bit rate as the allocation signal as it is, and the transmission unit (B) outputs from the input / output information of the video encoding unit (4). A delay amount calculation unit (31) for generating delay time information for synchronizing the video reproduction output and the audio reproduction output,
25. The video / audio multiplex transmission system according to any one of claims 22 to 24, wherein the delay time information is also multiplexed together by the multiplexing unit (6). 26. An SB-ADPCM coding unit, wherein said speech coding unit (2) divides and encodes said digital speech into a low frequency bit portion and a high frequency bit portion, and allocates a high frequency bit portion by said allocation signal. Amount, and the transmitting unit obtains an inter-frame information change rate of the digital video and compares it with a threshold value, and the determination result is the encoding control unit (5).
The video / audio multiplex transmission system according to any one of claims 1 to 4, further comprising an inter-frame change rate determination unit (8) for outputting to. 27. An SB-ADPCM coding unit, wherein the speech coding unit (2) divides the digital speech into a low frequency bit portion and a high frequency bit portion and encodes the high frequency bit portion by the allocation signal. A video coding unit (41) for selecting the amount, and the video coding unit (4) for coding the digital video, and a variable length coding unit for variable-length coding the coded video information. (42), a buffer (43) for temporarily accumulating the variable length coded video information, and an encoding amount control section for supplying an accumulation amount signal according to the amount of information accumulated in the buffer (43). The video / audio multiplex transmission system according to any one of claims 1 to 4, wherein the video / audio multiplex transmission system comprises a buffer determination unit (44) for outputting to 5). 28. An SB-ADPCM encoding unit for encoding the digital voice by dividing the digital voice into a low-frequency bit portion and a high-frequency bit portion, wherein the high-frequency bit portion is assigned by the assignment signal. A delay amount calculation unit (31) for selecting the amount and generating delay time information for the transmission unit (B) to synchronize the video reproduction output and the audio reproduction output from the input / output information of the video encoding unit (4). ) Is further provided,
The video / audio multiplex transmission system according to any one of claims 1 to 4, wherein the delay time information is also multiplexed by the multiplexing unit (6). 29. The video according to claim 5, wherein the audio decoding unit SB-ADPCM-decodes the coded audio according to the allocation signal.
Audio multiplex transmission system. 30. A video coding unit (41) for coding the digital video by the video coding unit (4), and a variable length coding unit (41) for variable length coding the coded video information. 42), a buffer (43) for temporarily storing the variable-length coded video information, and a storage amount signal according to the amount of information stored in the buffer (43) by the encoding control unit (5). 19. The video / audio multiplex transmission system according to claim 18, which comprises a buffer determination unit (44) for outputting to the). 31. A video coding unit (41) for coding the digital video by the video coding unit (4), and a variable length coding unit (41) for variable length coding the coded video information. 42), a buffer (43) for temporarily storing the variable-length coded video information, and a storage amount signal according to the amount of information stored in the buffer (43) by the encoding control unit (5). 27. The video / audio multiplex transmission system according to claim 26, further comprising a buffer determination unit (44) for outputting to the). 32. The encoding signal generator according to claim 11 or 26, wherein the encoding control unit (5) generates an allocation signal based on a determination result output from the inter-frame change rate determination unit (8). Video / audio multiplex transmission system. 33. The video / audio multiplex transmission system according to claim 12 or 27, wherein said encoding control unit (5) generates an allocation signal based on a storage amount signal. 34. The video / audio multiplex transmission system according to claim 18, wherein the encoding control unit (5) generates an allocation signal according to the interframe change rate determination result and the optimum audio bit rate. 35. The video / audio multiplex transmission according to claim 19 or 30, wherein the encoding control unit (5) generates an allocation signal according to an accumulated amount signal and an optimum audio bit rate. system. 36. The video / audio multiplex transmission according to claim 30, wherein said encoding control unit (5) generates an allocation signal according to an interframe change rate determination result, a storage amount signal and an optimum audio bit rate. system. 37. The video / audio multiplex transmission system according to claim 31, wherein the encoding control unit (5) generates an allocation signal according to the interframe change rate determination result and the accumulated amount signal. 38. A / for converting a voice input into a digital voice
D converter (1), audio encoding unit (2) for encoding the digital audio, and A / D converter (3) for converting an image input into a digital image
A video coding unit (4) for coding the digital video, and video coding delay time information for synchronizing the video playback output and the audio playback output from the input / output information of the video coding unit (4). A video having a transmission amount (D) including a delay amount calculation unit (31) that occurs and a multiplexing unit (6) that multiplexes the coded video information and audio information and the delay time information. -Voice multiplex transmission system. 39. A separation unit (11) for separating a multiplexed signal from a transmission line (10) into audio information, video information and delay time information, and a variable for delaying the audio information according to the delay time information. A delay control section (32), a voice decoding section (12) for decoding voice information from the variable delay control section (32), and a D / C for converting the decoded digital voice into a voice signal.
An A converter (13), a video decoding unit (14) for decoding the video information, and a D / A converter (15) for converting the decoded digital video output into a video output. A video / audio multiplex transmission system having a receiving unit (E) that operates. 40. A / A for converting voice input into digital voice
D converter (1), audio encoding unit (2) for encoding the digital audio, and A / D converter (3) for converting an image input into a digital image
A video coding unit (4) for coding the digital video, and video coding delay time information for synchronizing the video playback output and the audio playback output from the input / output information of the video coding unit (4). A transmission unit (D) including a delay amount calculation unit (31) that occurs, and a multiplexing unit (6) that multiplexes the coded video information and audio information with the delay time information; A separation unit (11) for separating the multiplexed signal from the path (10) into audio information, video information and delay time information, and a variable delay control unit (32) for delaying the audio information according to the delay time information. A voice decoding unit (12) for decoding the voice information from the variable delay control unit (32), and a D / C for converting the decoded digital voice into a voice signal.
An A converter (13), a video decoding unit (14) for decoding the video information, and a D / A converter (15) for converting the decoded digital video output into a video output. A video / audio multiplex transmission system configured with a receiving unit (E). 41. A video coding unit (41) for coding a video signal corresponding to the video signal, and a variable length coding unit (42) for giving a variable length code to the coded result. At the same time, a voice coding unit (2) is provided corresponding to the voice signal, and the output from the variable length coding unit (42) and the output corresponding to the voice coding unit (2) are multiplexed. In the video / audio transmission system configured to be transmitted by the receiving side and performing a process of extracting a video signal and an audio signal with respect to the transmitted one, the output from the audio encoding unit (2). Based on this, a time division encoding unit (71) for extracting and packetizing a signal during the effective period of the voice is provided, and the time division encoding unit (71) controls the voice transmission rate by the system control unit (19). The system control unit (19) is configured to notify Receiving the voice transmission rate, the threshold data for selecting the quantizer is changed according to the amount of data on the buffer memory (43) in the variable length coding unit (42), and the video coding unit. A video / audio transmission system configured to control the amount of information generated by the encoding in (41) and adapted to perform transmission according to a frame format suitable for the above-mentioned audio transmission rate.