JPH10341217A - Multimedia multiplex communication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To regulate re-transmission processing in response to a characteristic of each medium in the case of multiplexing media data of plural kinds and transmitting the multiplexed data from a transmitter to a receiver. SOLUTION: Block generating means 3, 7, 11 divide media data into blocks, provide an error check code to them, a multiplexer means 13 of a transmitter mixes and arranges them, a transmission means 15 transmits them, and re- transmission means 4, 8, 12 apply re-transmission processing to the block data informed from a receiver. A block data stream received by a reception means of the receiver is separated into each medium by a demultiplexer means, a reproduction means uses an error detection means to inform a block whose transmission error is detected to the transmitter. Furthermore, at least the transmitter or the receiver is provided with a re-transmission setting means that sets whether or not re-transmission processing is applied to each medium and a re-transmission control means that regulates re-transmission processing for each medium based on the setting.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for dividing data for each medium into blocks, multiplexing block data of these plural kinds of media and communicating, and in particular, performing retransmission processing for each media data. The present invention relates to a multimedia multiplexing communication system and method for setting whether to perform or not, and restricting retransmission processing for each piece of media data based on the setting.

[0002]

2. Description of the Related Art As a system for transmitting and receiving a plurality of types of media data such as image data, audio data and text data, a multimedia multiplex communication system for multiplexing and communicating these data is known.
That is, in such a system, a plurality of types of media data as described above are multiplexed, and, for example, a large amount of multiplexed data is transferred to a narrowband integrated service digital network (N-network).
ISDN) and broadband integrated service digital network (B-IS)
Transmission from a transmitting apparatus to a receiving apparatus via a wired network such as DN) is performed.

[0003] In the above system, it may be desired to transmit multimedia data even in a place distant from the end of the wired network in the same manner as described above.
In such a case, the multiplexed data is transmitted, for example, via a wireless line. However, since the transmission quality of a wireless line is generally lower than that of a wired line, when such a wireless line is used as a transmission line, the probability of an error occurring in data being transmitted increases. . Note that errors occurring in data being transmitted include, for example, random errors that occur randomly in data and burst errors that occur over many continuous data, and such errors include, for example, the frequency used for transmission. And various modes depending on the characteristics of the modulation method and the environment such as obstacles in the transmission path.

As a method for dealing with the above transmission error, an error correction technique such as a BCH code, a Reed-Solomon code, a Viterbi decoding method or a combination thereof, or an error diffusion technique such as an interleave is used. May be However, especially when a transmission path with low transmission quality such as a wireless line is used, it is difficult to obtain the communication quality necessary for reproducing each media data even using such a technique. There are many. For this reason, in the above system,
Retransmission processing is performed by the transmitting device on data that has not been transmitted normally from the transmitting device to the receiving device, thereby improving the quality of multimedia data reproduced by the receiving device.

[0005]

However, in the above-described multimedia multiplex communication system and method, when performing retransmission processing of multimedia data to be multiplexed and transmitted, retransmission processing according to characteristics of each medium is performed. There was a problem that it was not performed. That is,
In the communication of multimedia data, for example, media data requiring immediateness and media data particularly requiring data reliability (transmission quality) may be simultaneously multiplexed and transmitted. However, there is a problem in that it is not possible to set whether or not to perform retransmission processing according to characteristics of each medium such as images, sounds, and texts. Similarly, for example, even when retransmission processing is performed on media data requiring immediacy, retransmission data is not obtained within the required time, and the effect of performing the retransmission processing is hardly obtained. However, since retransmission processing is performed for all multiplexed and transmitted media data, there is a problem that the transmission efficiency of the entire multimedia data is reduced.

The present invention has been made to solve such a conventional problem. Data of each medium is divided into blocks by a transmitting apparatus, and block data of these plural kinds of media are mixed and multiplexed. When the receiving side device receives the block data sequence and reproduces each media data, it sets whether or not to perform retransmission processing for each media data according to the characteristics of each media, etc. An object of the present invention is to provide a multimedia multiplex communication system capable of restricting retransmission processing for each piece of media data based on a setting.

Further, according to the present invention, when data of each medium is divided into blocks, and block data of these plural types of media are multiplexed and transmitted, each of the media data is selected according to the characteristics of each medium. It is an object of the present invention to provide a multimedia multiplex communication method capable of setting whether or not to perform retransmission processing and restricting retransmission processing for each piece of media data based on the setting.

[0008]

In order to achieve the above object, in a multimedia multiplex communication system according to the present invention, a plurality of types of media data are multiplexed and the following data is transmitted between a transmitting apparatus and a receiving apparatus. Communicate in such a way. In the transmitting device, the block generating means divides each media data into a plurality of block data, adds a check code for detecting a transmission error for each block data, and the multiplexing means generates the block data of each generated media data. Are mixed and the transmitting means transmits the block data sequence generated by the mixed arrangement. On the other hand, in the receiving device, the receiving unit receives the block data sequence transmitted from the transmitting device, the separating unit separates the received block data sequence into block data for each media data, and the error detecting unit performs the separation. A transmission error is detected based on the check code for each of the block data thus obtained, and the reproducing unit reproduces the media data from the separated block data for each media data.

In the receiving device, the retransmission request means notifies the transmitting device of the block data in which the transmission error has been detected, while in the transmitting device, the retransmitting device retransmits the block data notified from the receiving device. Perform processing.
Here, at least one of the transmitting device and the receiving device has retransmission setting means for setting whether or not to perform retransmission processing for each piece of media data, and each medium based on the setting of the retransmission setting means. Retransmission control means for restricting retransmission processing by the retransmission means is provided for each data, and retransmission processing from the transmitting device to the receiving device is restricted for each piece of media data based on the setting of the retransmission setting means.

Therefore, for example, when image data, audio data, and text data are mixed and multiplexed and communicated, retransmission processing is not performed for audio data requiring immediacy, and image data for which data quality is regarded as important. Whether or not to perform retransmission processing of each media data according to the characteristics of each media, such as retransmission processing for text and text data, and restrict the retransmission processing for each media data based on the setting can do. Also,
In this manner, the retransmission processing is regulated according to the characteristics of each medium, and, for example, the transmission efficiency of the entire multimedia data can be improved by focusing on only the media data particularly required to perform the retransmission processing. The transmission quality can be ensured by performing the retransmission processing on the data of the media in which the data quality is particularly important without deteriorating.

Further, in the multimedia multiplex communication method according to the present invention, when dividing data for each medium into blocks and multiplexing and transmitting block data of these plural types of media, the type of each media data Whether retransmission processing is performed or not is set for each block. When a transmission error occurs in block data transmitted for media data for which retransmission processing is set, retransmission processing is performed for the block data. Therefore, similarly to the above, it is possible to set whether or not to perform retransmission processing for each piece of media data in accordance with the characteristics of each medium and to restrict the retransmission processing for each piece of media data based on the setting.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment according to the present invention will be described with reference to the drawings. The multimedia multiplex communication system according to the present invention includes a transmitting device that transmits data and a receiving device that receives data. In this example, these devices are connected via a wireless transmission path. A case where image data, audio data, and text data are multiplexed for communication will be described.

FIG. 1 shows an example of a transmitting side device of a multimedia multiplex communication system according to the present invention. The transmitting side device has an image input section 1 for inputting image data.
An image encoding unit 2 for encoding image data, a cell unit 3 for dividing image data into cells, a retransmission unit 4 for controlling retransmission processing of image data, and an audio input unit 5 for inputting audio data And an audio encoding unit 6 for encoding audio data
A cell unit 7 for dividing audio data into cells; a retransmission unit 8 for controlling retransmission processing of audio data; a data input unit 9 for inputting text data; and a data encoding unit 10 for encoding text data A cell unit 11 for dividing text data into cells; a retransmitting unit 12 for controlling retransmission processing of text data; a multiplexing unit 13 for multiplexing image data, audio data, and text data; A packetizing unit 14 for packetizing the received data, a transmitting / receiving unit 15 for transmitting / receiving data, a control data decrypting unit 16 for decrypting control data received from the receiving side device, and a control for controlling the above units 1 to 16 collectively. A part 17 is provided.

The image input unit 1 is composed of, for example, a still camera or a video camera. The image input unit 1 performs analog-to-digital conversion processing, correction processing, image size conversion processing, and the like on an analog signal representing input image data, and performs image coding. Output to 2. When a digital image signal is input to the image input unit 1, there is no need to perform the above-described analog-to-digital conversion processing. Further, the image input unit 1 can be configured as a means for reading and inputting image data stored in a memory, an external storage medium, or the like. The image encoding unit 2 includes an encoder that encodes image data. The image encoding unit 2 encodes digital image data input from the image input unit 1 and outputs the digital image data to the cell conversion unit 3. here,
As a method for encoding image data, for example, a method such as JPEG is used for still image data.
For moving image data, see H.264. H.261 and MPEG are used.

The cell forming section 3 has a block generating means for dividing image data into a plurality of block data and adding a check code for detecting a transmission error for each block data. As shown in (1), the coded image data stream 21 input from the image coding unit 2 is divided into user data for each fixed data length, and a check code is added to these data to add a plurality of cells 22a, 22b, 2
2c, ... are generated. Here, each cell 22a, 22
.., b, 22c,..., as shown in FIG. 3, for example, a header 23 having control data such as a sequence number, a time stamp, and destination information having a fixed data length, and user data corresponding to the above-described block data. 24 and the cyclic check character (C
RC) and a trailer 25 in which check codes such as various error correction codes have a fixed data length. In FIG. 3, the cells 22a and 22 shown in FIG.
, 22c,... are collectively shown as cells 22.

Further, the above-mentioned header 23 contains an identifier indicating the type of the medium (ie, image data in this case). The media type and transmission order of each cell are specified. As the order number, a number such as “0” to “255” is cyclically added to each cell in the output order of the cells. The retransmission unit 4 has a retransmission buffer for storing cells for image data, outputs the cells input from the cellifying unit 3 to the multiplexing unit 13, and stores the cells in the retransmission buffers. I do. In addition, the retransmitting unit 4 takes out a cell to be retransmitted from the retransmission buffer and outputs it to the multiplexing unit 13 according to an instruction from the control unit 17 described later. The cells in the retransmission buffer are discarded.

In this example, the retransmitting section 4 performs the retransmitting process on the cell notified from the receiving side device in accordance with the retransmitting instruction from the control section 17 as described above, thereby forming a retransmitting means for the image data. You. Voice input unit 5
Is composed of, for example, a microphone, and converts the input audio data into an analog-to-digital signal and outputs it to the audio encoding unit 6. Further, the audio input unit 5 may be configured as a means for reading and inputting audio data stored in a memory or an external storage medium. The audio encoding unit 6 includes an encoder that encodes audio data, and encodes the audio data input from the audio input unit 5 and outputs the encoded audio data to the cell unit 7.

The celling section 7 has a block generating means for dividing the audio data into a plurality of block data and adding a check code for detecting a transmission error for each block data. Similarly to the celling unit 3 for image data, a plurality of cells are generated from the encoded audio data stream input from the audio encoding unit 6 and output to the retransmission unit 8. The retransmitting unit 8 has a retransmission buffer for storing cells for voice data, outputs cells input from the cellifying unit 8 to the multiplexing unit 13, and stores the cells in the retransmission buffer. I do. Also,
The retransmitting unit 8 performs cell retransmission processing and the like in accordance with an instruction from the control unit 17 described later, similarly to the retransmitting unit 4 for image data described above. In this example, the retransmitting unit 8 performs retransmission processing on the cell notified from the receiving side device in accordance with the retransmission instruction from the control unit 17, thereby forming a retransmission unit for voice data.

The data input section 9 has a means for inputting a text file stored in, for example, a memory or an external storage medium. Output to
As the data input unit 9, for example, the data input unit 9 may be constituted by input devices such as a keyboard and a mouse, and an operator may operate these input devices to input characters and the like. Data encoding unit 10
Is composed of an encoder that encodes text data. The encoder encodes text data input from the data input unit 9 and outputs the encoded text data to the cell conversion unit 11.

The cell forming section 11 has a block generating means for dividing text data into a plurality of block data and adding a check code for detecting a transmission error for each block data. Similar to the celling unit 3 for image data, a plurality of cells are generated from the coded text data stream input from the data coding unit 10 and output to the retransmission unit 12. The retransmitting unit 12 has a retransmission buffer for storing cells for text data, outputs cells input from the cellifying unit 12 to the multiplexing unit 13, and stores the cells in the retransmission buffer. I do. In addition, the retransmission unit 12 includes a control unit 1 described later, similarly to the retransmission unit 4 for image data described above.
The cell retransmission processing and the like are performed in accordance with the instruction from. In addition,
In this example, the retransmitting unit 12 performs retransmission processing on the cell notified from the receiving side device in accordance with the retransmission instruction from the control unit 17, thereby forming a retransmission unit for text data.

The multiplexing unit 13 includes the above-mentioned celling unit 3,
A multiplexing means for mixing and arranging the cells of each media data generated by 7 and 11 is provided.
As shown in, the cell of each media included in the cell stream 31 for the image data, the cell stream 32 for the audio data, and the cell stream 33 for the text data input from each of the retransmission units 4, 8, and 12, respectively. The multiplexed stream 34 is generated by the mixed arrangement, and the generated multiplexed stream is output to the packetizing unit 14. In this example, the multiplexing ratio is instructed by the control unit 17 described later.

The packetizing section 14 collectively converts cells included in the multiplexed stream input from the multiplexing section 13 into packets of, for example, a predetermined number, and outputs the generated packets to the transmitting / receiving section 15. Here, FIG. 5 shows an example of the configuration of a packet 35 generated by the packetizing unit 14. This packet 35 is composed of control information such as a preamble and a unique word for synchronization during communication. Header 36, user data 37 in which cells are grouped by a fixed number, and a trailer 38 including an error correction code and the like. In this example, data communication is performed between the transmitting device and the receiving device using this packet as a transmission unit.

The transmission / reception unit 15 has a transmission means for transmitting the block data sequence generated by the multiplexing unit 13 by the mixed arrangement, that is, the multiplexed stream.
As described above, the multiplexed stream is transmitted to the receiving side device for each fixed number of cells via the wireless transmission path, using the packet input from the packetizer 14 as a transmission unit. Further, the transmitting / receiving unit 15 transmits the ACK transmitted from the receiving side device.
There is provided a means for receiving control data including the data and NAK via a wireless transmission path, and the received control data is output to the control data decoding unit 16.

The above-mentioned ACK or NAK is generally used to notify the transmitting device of information as to whether or not the receiving device has successfully received the cell. If the device does not request the transmitting device to retransmit the cell, an ACK shall be notified,
Conversely, when requesting retransmission of a cell, NAK is notified. The transmission / reception unit 15 described above is, for example, a phase shift keying (PSK) or a quadrature phase modulation (QPS).
K), and a demodulator that demodulates data by a similar method, etc. The transmission processing and the reception processing by the transmission / reception unit 15 are performed by a control unit 17 described later. Switching is performed, for example, at a certain timing according to the instruction.

The control data decoding unit 16 decodes the control data input from the transmission / reception unit 15 and notifies the control unit 17 of the decoded contents. That is, the control data decoding unit 16
Is based on the sequence number notified as control data together with ACK or NAK, an identifier indicating the type of media, etc.
It specifies the type of media and the sequence number of the cell for which retransmission processing is requested, and notifies the control unit 17 of it. Similarly,
No retransmission processing is requested, and the retransmission units 4, 8,
The cell to be discarded is specified from the 12 retransmission buffers and notified to the control unit 17.

The control unit 17 is a CP for performing various arithmetic processing.
U, a ROM storing a control program, a RAM used as a processing area for operations performed by the CPU and a storage area for various data, a memory for storing data, and the like. The control program is developed in the RAM and executed, whereby the above-described units 1 to 16 are collectively controlled, thereby controlling a multimedia communication process such as a retransmission process in the transmitting apparatus. In this example, each of the processing units in the above cell units 3, 7, 11; retransmitting units 4, 8, 12, data input unit 9, multiplexing unit 13, packetizing unit 14, and control data decoding unit 16 The CPU provided in the control unit 17 expands a control program corresponding to each process in the RAM and executes the control program.

The memory provided in the control unit 17 stores a retransmission status table 41 shown in FIG. 6 as retransmission setting means for setting whether or not to perform retransmission processing for each piece of media data. I have. In the retransmission state table 41 shown in the figure, retransmission processing is performed for image data (indicated as “present” in the figure), and retransmission processing is not performed for audio data (“none” in the figure). ) And text data (shown as “data” in the figure) are set to be retransmitted (shown as “present” in the figure). In FIG. 6, whether or not to perform retransmission processing is indicated by "
Although the presence / absence is indicated using “present” or “absent”, it is also possible to indicate whether or not to perform the retransmission processing by using a numerical value such as “1” (retransmission processing is performed) or “0” (no retransmission processing is performed). it can.
Further, the control unit 17 includes the retransmission status table 41 described above.
, The retransmission units 4, 8,
Retransmission control means for regulating retransmission processing by the twelve retransmission means is provided.

A retransmission cell table 42 shown in FIG. 7 is stored in a memory provided in the control unit 17, and the control unit 17 performs retransmission processing on the retransmission cell table 42 from the receiving apparatus. Of the cell requested (retransmission cell sequence number) is sequentially added and stored for each medium, and based on the contents of this table 42, the retransmission units 4, 8, and 12 are controlled to Causes retransmission processing. In the above, the retransmission unit 4 is used as a buffer.
Although only the retransmission buffer provided in each of the sections 8 and 12 has been described, in each of the above-described sections 1 to 16, a buffer for temporarily storing data is provided in a component in which a waiting time occurs when processing data. ing. With the above configuration, the transmitting device multiplexes a plurality of types of media data and transmits the multiplexed media data to the receiving device. For a cell for which a retransmission request has been notified from the receiving device, retransmission processing is performed on the media of the cell. When it is set to perform, the retransmission processing is performed, and when it is set not to perform the retransmission processing for the media of the cell, the retransmission processing is restricted and the processing is not performed.

Next, FIG. 8 shows an example of a receiving side device of the multimedia multiplex communication system according to the present invention. The receiving side device includes a transmitting / receiving section 51 for transmitting and receiving data, and a packet transmitting / receiving section. A packet decomposing unit 52 for decomposing, a demultiplexing unit 53 for demultiplexing a multiplexed stream, and a retransmission determining unit 54 for detecting a cell error in image data and the like
A streaming unit 55 that generates cells into an image data stream; an image decoding unit 56 that decodes encoded image data; an image output unit 57 that outputs image data; and cell error detection for audio data. And the like, a streaming unit 59 for generating cells into an audio data stream, an audio decoding unit 60 for decoding encoded audio data, and an audio output unit 61 for outputting audio data.
A retransmission determination unit 62 for performing cell error detection and the like on text data, a streaming unit 63 for generating cells as a text data stream, a data decoding unit 64 for decoding coded text data, And a data output unit 65 that outputs the data.

The transmission / reception unit 51 has a receiving means for receiving the block data sequence transmitted from the transmission side device, that is, the multiplexed stream. In this example, as described above, the radio transmission is performed using the packet as a transmission unit. The received packet is output to the packet decomposing unit 52 via the path. The transmission / reception unit 51 includes a unit that transmits control data input from a control data generation unit 66 described later to a transmission-side device via a wireless transmission path. The transmission / reception unit 51 includes a modulator, a demodulator, and the like, similarly to the transmission / reception unit 15 provided in the transmission-side device, and performs transmission processing and reception processing in accordance with an instruction from a control unit 67 described later. Are switched at every certain timing, for example.

The packet decomposing unit 52 performs an error correction process on the packet input from the transmission / reception unit 51 using the error correction code in the trailer, and processes the packet in FIG.
The header 36, user data 37, and trailer 38 shown in FIG.
And outputs the decomposed user data (that is, the multiplexed stream) to the demultiplexing unit 53. Separation unit 5
Reference numeral 3 denotes a demultiplexing unit that demultiplexes a block data sequence, that is, a multiplexed stream into block data for each piece of media data. At the same time, the type of media of each cell is determined based on the above-mentioned identifier included in the header of each cell, and each of these cells is determined by a retransmission determination unit 54, 5
8 and 62.

The retransmission judging section 54 has an error detecting means for detecting a transmission error for each cell of the image data based on the check code. In this example, the retransmission judging section 54 includes a cell included in the cell input from the demultiplexing section 53. Based on a check code such as a CRC and an error correction code in the trailer, whether or not a transmission error has occurred in the cell is detected, and user data included in each cell is transmitted according to an instruction from a control unit 67 described later. Output to the streaming unit 55. In addition, when an error correction code is used as a check code and a transmission error generated in a cell is corrected by the error correction code, the cell can be regarded as normally received.

The retransmission determination section 54 performs retransmission processing for each cell input from the demultiplexing section 53 based on the above-described transmission error detection result and the setting of a retransmission state table 71 shown in FIG. In addition to determining whether or not to perform the retransmission processing, the control unit 67 reads the sequence number in the header included in the cell and notifies the control unit 67 of the read sequence number and the determination result as to whether or not to perform the retransmission processing. Also,
The retransmission determination unit 54 includes a reception buffer that stores the user data in the cell for the image data,
Based on the setting of the retransmission state table 71 shown in FIG. 9 described later, it is determined whether or not to store the user data in the cell input from the demultiplexing unit 53 in this reception buffer, and it is determined that the user data is to be stored. In this case, the user data in the input cell is stored in the reception buffer.

That is, for example, by temporarily storing user data in a cell in which a transmission error has occurred in the above-described reception buffer without discarding the data, the user data can be normally obtained even by retransmission processing. If not, the user data stored in the reception buffer can be output to the streamer 55 instead of the normal user data and used for the decoding process. Specifically, for example, when an image frame is divided into a plurality of blocks and encoded using the correlation between adjacent blocks in the frame, when decoding the image frame, the original block is generally used. In this case, the same number of blocks (that is, user data) is required, and in such a case, the reception buffer as described above is effective.

That is, if the number of one or several block data in the image data frame encoded using the intra-frame correlation as described above is missing, the image data frame is normally decoded. In such a case, even in the case where a transmission error has occurred, the same number of block data (that is, user data) as the original number may be required in the decoding process. For this reason, the above-mentioned reception buffer also stores the user data in the cell in which the transmission error has occurred, and the cell is normally obtained by the retransmission processing, or the reception buffer is released from the control unit 67 described later. The stored cells are held until there is an instruction to be performed. When the user data is stored in the reception buffer, the arrangement order of the user data may be different from the original arrangement order due to a retransmission process or the like. By providing a table or the like for storing the order numbers of the respective user data, it is ensured that the order of outputting the respective user data is the same as the original order number when outputting the respective user data to the streamer 55.

The streaming unit 55 has a buffer for storing data, stores the user data of the image data input from the retransmission determination unit 54 in the order of input, and stores the stored user data in a fixed order, for example. And outputs them to the image decoding unit 56 in a coded image data stream for each number (data length). Image decoding unit 56
Is composed of a decoder for decoding the encoded image data, and decodes the encoded image data stream input from the streaming unit 55 by a method corresponding to the above-described encoding method in the transmitting device. Image output unit 57
Output to

In this example, since the encoding and decoding processes of the image data are performed, the stream data 55 and the image decoding device 56 perform block data (ie, user data ) Constitutes a reproducing means for reproducing the media data. The image output unit 57 includes, for example, a display screen, and causes the image data input from the image decoding unit 56 to be displayed and output on the screen. Note that the image output unit 57 may be configured as a unit that outputs and stores the image data input from the image decoding unit 56 in an external storage medium such as a memory.

The retransmission determination unit 58 has error detection means for detecting a transmission error based on a check code for each cell of audio data, similarly to the retransmission determination unit 54 for image data. In the example, based on a check code such as a CRC or an error correction code in a trailer included in a cell input from the demultiplexing unit 53, it is detected whether or not a transmission error has occurred in the cell, and a control described later is performed. The user data included in each cell is converted into a stream
9 is output. In addition, the retransmission determination unit 58 determines whether or not to perform the retransmission processing of the cell input from the demultiplexing unit 53 by performing the same processing as the above-described retransmission determination unit 54 for the image data. The control section 67 notifies the control section 67 of the cell sequence number.

The retransmission determination section 58 is provided with a reception buffer for storing user data in the cell for the audio data, and, like the retransmission determination section 54 for the image data, the demultiplexing section 53. Then, it is determined whether or not to store the user data in the cell input from the reception buffer in the reception buffer. If it is determined that the user data is to be stored, the user data in the cell is temporarily stored in the reception buffer. Similarly to the above, in the reception buffer, the stored cell is held until the cell is normally obtained by the retransmission processing or until there is an instruction to release the reception buffer from the control unit 67 described later. deep.

The streaming section 59 has a buffer for storing data, and stores the user data for the audio data input from the retransmission determination section 58, similarly to the streaming section 55 for the image data. Then, the stored user data is grouped into, for example, an encoded audio data stream of a predetermined number (data length) and output to the audio decoding unit 60. The audio decoding unit 60 is composed of a decoder that decodes the encoded audio data, and converts the encoded audio data stream input from the streaming unit 59 into an encoding method in the above-described transmitting device. The signal is decoded by the system and output to the audio output unit 61.

In the present embodiment, since the encoding and decoding of the audio data are performed, the above-mentioned streaming unit 59 is used.
The audio decoding unit 60 constitutes a reproducing unit that reproduces media data from block data (ie, user data) of audio data. Audio output unit 6
Reference numeral 1 includes a speaker, for example, and outputs audio data input from the audio decoding unit 60. Note that the audio output unit 61 may be configured as a unit that outputs and stores the audio data input from the audio decoding unit 60 in an external storage medium such as a memory.

The retransmission determination unit 62 has error detection means for detecting a transmission error for each cell of text data based on a check code, similarly to the retransmission determination unit 54 for image data described above. In the example, based on a check code such as a CRC or an error correction code in a trailer included in a cell input from the demultiplexing unit 53, it is detected whether or not a transmission error has occurred in the cell, and a control described later is performed. The user data included in each cell is output to the streaming unit 63 according to the instruction from the unit 67. Further, the retransmission determination section 62 determines whether or not to perform the retransmission processing of the cell input from the demultiplexing section 53 by performing the same processing as the above-described retransmission determination section 54 for the image data. The control section 67 notifies the control section 67 of the cell sequence number.

Further, the retransmission determination section 62 is provided with a reception buffer for storing the user data in the cell for the text data. Like the retransmission determination section 54 for the image data, the separation section 53 is provided. Then, it is determined whether or not to store the user data in the cell input from the reception buffer in the reception buffer. If it is determined that the user data is to be stored, the user data in the input cell is temporarily stored in the reception buffer. Note that, similarly to the above, in the reception buffer, whether the stored cell is normally obtained by the retransmission process,
Alternatively, the reception buffer is held until an instruction to release the reception buffer is issued from the control unit 67 described later.

The streaming section 63 has a buffer for storing data, and stores the user data of the text data input from the retransmission determination section 62, similarly to the streaming section 55 of the image data described above. Then, the stored user data is combined into, for example, a coded text data stream of a predetermined number (data length) and output to the data decoding unit 64. The data decryption unit 64
It comprises a decoder for decoding the coded text data, and decodes the coded text data stream input from the streaming unit 63 by a method corresponding to the above-described coding method in the transmission side device to output data. Output to the unit 65.

In this embodiment, since the encoding and decoding of the text data are performed, the stream data 63 and the data decoding unit 64 described above use the block data (ie, the user data) for the text data. A playback means for playing back the media data from is constructed. The data output unit 65 is connected to, for example, a hard disk device that stores a text file, and stores the reproduced text data input from the data decoding unit 64 as a text file in the hard disk device. In addition, as the data output unit 65, for example, the data output unit 65
May be configured on a display screen, and the reproduced text data input from the data decoding unit 64 may be displayed and output on the screen.

The control data generating means 66 has means for transmitting the control data including ACK, NAK and the like to the transmitting device by the transmitting / receiving means 51 in accordance with an instruction from the control section 67 described later. Note that the control data includes, together with the ACK and NAK described above, an identifier indicating the type of media of the cell to be notified, and an order number of the cell. The control unit 67 includes a CPU that performs various types of arithmetic processing, a ROM that stores a control program,
The RAM is used as a processing area for operations performed by the U and a storage area for various data, a memory for storing data, and the like, and the CPU expands and executes the control program stored in the ROM in the RAM. Controls the above-described units 51 to 66, thereby controlling multimedia communication processing such as retransmission request processing in the receiving apparatus. In this example, each processing means in the packet decomposing unit 52, the demultiplexing unit 53, the retransmission determining units 54, 58, 62, the streaming units 55, 59, 63, the data output unit 65, and the control data generating unit 66 Is the control unit 6
The CPU provided in the CPU 7 loads a control program corresponding to each process into the RAM and executes the control program.

The memory provided in the control section 67 stores a retransmission status table 71 shown in FIG. 9 as retransmission setting means for setting whether or not to perform retransmission processing for each piece of media data. I have. In the retransmission status table 71 shown in the figure, similar to the retransmission status table 41 in the transmitting side device shown in FIG. 6, retransmission processing is performed on image data (indicated as "present" in the figure), Retransmission processing is not performed for voice data (indicated as “absent” in the figure), and retransmission processing is performed for text data (indicated as “data” in the figure) (“present” in the figure). Is set).

The retransmission status table 71 includes the retransmission determination sections 54, 58, 6 for each piece of media data.
It is set whether or not the user data in the cell is to be stored in the reception buffer provided in the reception buffer 2. In the retransmission state table 71 shown in FIG. 9, the image data is stored in the reception buffer (see FIG. 9). The voice data is not stored in the reception buffer (indicated as “do not enter” in the figure), and the text data is not stored in the reception buffer (indicated as “insert” in the figure). No ") is set. Further, as described above, the control unit 67 releases the user data for which normal data could not be obtained even by, for example, the retransmission processing from the reception buffer and converts the user data into the streaming units 55, 59, and 63.
An instruction is sent to the retransmission determination units 54, 58, and 62 to output them to

Here, the retransmission determination units 54 and 58 described above
At 62, the transmission error of the cell input from the demultiplexing unit 53 is detected, and the sequence number of the cell in which the transmission error is detected is notified to the control unit 67, whereby the control data generation unit 66 By transmitting a retransmission request such as the type of media and the sequence number of the cell to the transmitting apparatus, a retransmission request unit for each media data is configured. In this example, the retransmission request notification process by the retransmission request unit is controlled based on the setting of the retransmission status table 71 shown in FIG. 9 described above, and the media data set to perform the retransmission process is retransmitted. The retransmission control unit is configured by notifying the request and not performing the retransmission request notification but restricting the retransmission processing by the retransmission unit of the transmitting apparatus without performing the retransmission request notification.

In the above, only the reception buffers provided in the retransmission determination units 54, 58, 62 and the buffers provided in the streaming units 55, 59, 63 have been described. In the above, a component that causes a waiting time when processing data is provided with a buffer for temporarily storing data. With the above configuration, the receiving device performs retransmission processing when a packet including the multiplexed stream transmitted from the transmitting device is received and reproduced when a transmission error occurs in the received cell. Notify the transmitting device of a retransmission request for the cell for media data for which retransmission is set, and restrict the retransmission request notification for media data for which retransmission processing is not Do not do.

Next, a procedure of a retransmission process performed by the multimedia multiplex communication system including the above-described transmitting side apparatus and receiving side apparatus will be described with reference to the drawings. FIG. 10 shows a procedure of a retransmission process performed by the transmission side device of the multimedia multiplex communication system. In the transmitting device, when a NAK, that is, a retransmission request is notified from the receiving device for the cell transmitted to the receiving device, the retransmission request is notified to the control unit 17 via the control data decoding unit 16, and thereby, The control unit 17 instructs the retransmission units 4, 8, and 12 to start the retransmission process and starts the retransmission process (step S1).

When the retransmission processing is started, first, the control unit 1
In step 7, the retransmission status table 41 shown in FIG. 6 is referred to (step S2), and it is determined whether or not retransmission processing is to be performed on a cell of the media for which the retransmission request has been notified (step S3). For example, when a retransmission request is notified for a cell of image data and the retransmission state table 41 is set to perform retransmission processing on the image data, the retransmission cell table 42 shown in FIG. ,
If the sequence number of the cell to be retransmitted is stored in the retransmission cell table 42, the stored cell number is read out according to the written order, that is, the order in which the retransmission request is notified from the receiving side device (step S4).

The retransmission processing table 42 shown in FIG.
Stores that the cells of sequence numbers "3" and "35" should be retransmitted for the image data, and the sequence numbers "1" and "1" for the text data (shown as "data" in FIG. 7). 15 "," 23 "," 35 "," 5 "
It is stored that the cells 1 "and" 67 "are to be retransmitted. For example, for the image data, the sequence numbers" 3 "and" 35 "are set as the sequence numbers of the cells to be retransmitted by the above processing. Is read.

Next, if there is a cell to be retransmitted (step S5), the retransmitting units 4, 8,
12, retransmission processing is performed. That is, for example, when resending the cell with the sequence number "3" for the image data, the cell with the sequence number "3" is searched from the retransmission buffer provided in the retransmission unit 4 (step S6). The retrieved cell is retrieved (step S
7), and outputs the result to the multiplexing unit 13 (step S8). Here, it is determined whether or not there is a vacancy that can be further included in the packet generated by the packetizer 14 (step S9). It is determined whether there is another cell to be retransmitted (step S10). For example, when it is determined that the cell having the sequence number “35” is to be retransmitted for the image data, similar to the above, The cell is extracted from the retransmission buffer and output to the multiplexing unit 13 (steps S6 to S9). At this time, the retransmission cell table 42 described above may be referred to again.

Here, the sequence number of the retransmitted cell is deleted from the retransmission cell table 42 described above.
When there is no more room to include a cell in the packet currently generated by the packetizing unit 14 (step S9), the packet is retransmitted by transmitting the packet. Then, the cell retransmission process ends (step S15). And
With the next packet generated by the packetizing unit 14, retransmission processing for other cells to be retransmitted is performed in the same manner as described above. Note that retransmission processing is performed on text data in the same manner as in the case of image data described above.

When there are no more cells to be retransmitted and there is a vacancy to include cells in the packet currently generated by the packetizing unit 14 (step S10), the retransmitting units 4 and 12 Then, the cell which is normally transmitted from the cellifying units 3 and 11 (that is, a cell which is not a retransmission) is extracted (step S11) and output to the multiplexing unit 13 (step S12). Next, similarly to the above, it is determined whether or not the packet generated by the packetizer 14 has a vacancy for including the cell (step S13). And outputs it to the multiplexing unit 13 (step S1).
1 to S12), and if there is no free space, the generated packet is transmitted to the receiving-side device, so that the retransmission cell included in the packet is retransmitted (step S14).

In the above-described processing, for example, for audio data for which retransmission processing is not performed in the retransmission state table 41, the retransmission processing is restricted even when a NAK is notified from the receiving apparatus. Without being performed (step S3), only normal cells are transmitted as described above (steps S11 to S14). Also, even if the medium is to be retransmitted, the retransmission cell table 42
If the sequence number of the cell to be retransmitted is not written in (step S5), no retransmission is performed.

Therefore, for example, the retransmission processing is not performed for audio data that requires immediacy, and the retransmission processing is performed for image data or text data that emphasizes data quality. Whether or not to perform the retransmission processing of each media data can be set according to the above, and the retransmission processing for each piece of media data can be regulated based on the setting. In addition, the retransmission processing is regulated in accordance with the characteristics of each medium as described above, and, for example, the transmission efficiency of the entire multimedia is improved by emphasizing retransmission processing only for media data particularly required to perform the retransmission processing. The transmission quality can be ensured by performing the retransmission processing on the data of the media in which the data quality is particularly important without reducing the transmission quality.

For example, instead of performing retransmission processing on all multiplexed and transmitted media data, retransmission processing is performed only on media data of which data quality is particularly important. It is also possible to improve the transmission efficiency of the entire data. In this example, the processing procedure is such that the cell related to retransmission is transmitted to the receiving apparatus with priority over the normal cell (non-retransmission cell), thereby ensuring the immediacy of the retransmission processing. It is also possible to set the number of retransmission cells that can be included in the packet, and perform the cell retransmission processing according to this setting.

Next, FIG. 11 shows a procedure of a retransmission request notification process performed by the receiving side device of the multimedia multiplex communication system. Upon receiving the cell transmitted from the transmitting device, the receiving device starts the retransmission determination process (step S21). First, the retransmission determination units 54 and 5
8 and 62, the retransmission status table 71 shown in FIG.
(Step S22), the setting of whether or not to perform retransmission processing for each piece of media data is read, and the setting of whether or not to store user data in a cell for each piece of media data in the reception buffer. Read.

At the same time, retransmission determination sections 54, 58, 6
In step 2, the received cell is extracted from the demultiplexing unit 53 and error correction processing is performed (step S23), and the sequence number is extracted from the header in the cell (step S2).
4). Further, it is determined whether or not there is an error in the user data in the cell (step S25).
It is determined whether or not to retransmit the cell based on the setting of the retransmission state table 71 read as described above (step S26). Here, for example, when it is determined that retransmission processing is to be performed on a cell having an error in image data or text data, NAK is set as a response to notify the transmitting apparatus (step S27).

If there is no error in the user data in the cell (step S25), ACK is set as a response for notifying the transmitting side device (step S32). However, if it is set that the retransmission processing is not restricted and performed for the medium (step S26), ACK is set as a response to notify the transmitting apparatus (step S33).

Next, the retransmission determination units 54, 58, 62
It is determined whether or not the user data in the cell is stored in the reception buffer based on the retransmission state table 71 read as described above (step S28). For example, it is determined that the user data for the image data is stored in the reception buffer. In this case, the user data is stored in the reception buffer (step S29). For example, when it is determined that the user data is not stored in the reception buffer for voice data or text data, the normally obtained user data is stored. The data is output to the streamers 59 and 63 according to the sequence numbers (step S34). As mentioned above,
The user data stored in the reception buffer is such that the user data is normally obtained by the retransmission processing, or
It is held until the control unit 67 gives an instruction to release the reception buffer.

At the same time, the ACK or NAK response set by the retransmission determination sections 54, 58, and 62 and the type and sequence number of the cell medium are notified to the control section 67 as described above. Is generated by the control data generation unit 66 and transmitted to the transmitting device (step S30). This allows
The retransmission request notification process for one or several cells received by the receiving device ends (step S3).
1) The same processing as described above is performed on the subsequently received cells.

Therefore, as in the case of the above-mentioned transmitting apparatus, whether or not to perform the retransmission processing of each media data is set according to the characteristics of each medium, and based on the setting, each media data is retransmitted. Multimedia data communication can be performed by restricting retransmission processing. As described above, in the above-described multimedia multiplexing communication system, multimedia data communication can be performed by restricting retransmission processing for each piece of media data according to the characteristics of each piece of media. Similarly, according to the above-described multimedia multiplexing communication method, when data of each medium is divided into blocks and block data of these plural types of media are multiplexed and transmitted, the characteristics and the like of each medium may vary. By setting whether or not to perform retransmission processing for each piece of media data in accordance with the setting and regulating the retransmission processing for each piece of media data based on the setting, for example, it is possible to improve the transmission efficiency of the entire multimedia data. it can.

In this example, as described above, it is possible to set whether or not the user data in the cell received by the receiving device is stored in the reception buffer for each medium. Even if there is an error in the user data of the image data, even if there is an error, by temporarily storing the user data in the reception buffer, it is ensured that the decoding process of the image data is normally performed. By performing the following processing without storing such unnecessary audio data and text data in the reception buffer, the size of the reception buffer can be reduced. That is, it is possible to set whether or not to store the user data in the cell received by the receiving side device in the reception buffer according to the characteristics of the encoding and decoding processes performed for each media data. .

Here, whether or not retransmission processing is performed is set for each piece of media data. In addition to this setting, retransmission is performed for the same cell for each piece of media data for which retransmission processing is set. It is also possible to set an upper limit number of times that the processing can be performed. For example, such setting is realized by storing the retransmission state table 81 shown in FIG. 12 in the control unit 17 provided in the transmission-side device. In the retransmission status table 81 shown in FIG. 6, in addition to the contents of the retransmission status table 41 shown in FIG.
The upper limit number of times that retransmission processing can be performed for the same cell is set for each piece of media data for which retransmission processing is set.

In other words, in the retransmission state table 81 shown in FIG. 12, for example, for image data for which the quality of data is regarded as important and the immediacy is required for reproduction, the same cell may be used up to once. It is set to perform retransmission processing. For example, text data in which data quality is particularly important but immediacy is not relatively required is set to retransmit the same cell up to three times. ing. In this example, it is set that retransmission processing is not performed for audio data. Also,
The control unit 17 provided in the transmission side device stores, for example, a retransmission cell history table 82 shown in FIG. 13 in order to manage the number of times of retransmission processing of cells for each piece of media data. In the retransmission cell history table 82, the sequence numbers of the cells that have undergone retransmission processing including normal cell transmission (transmission other than retransmission) are stored in accordance with the transmission order. That is, for example, for image data, “10”, “1”
It is stored that the cells having the sequence numbers of 5 "," 30 "," 10 ", and" 45 "have been transmitted.

As a result, for example, the cell with the sequence number “10” for the image data has been transmitted twice (ie, once retransmitted) including normal transmission,
Similarly, it is determined that the cell with the sequence number “35” of the text data has been transmitted three times including the normal transmission (that is, retransmitted twice). In this example,
"0" as the sequence number for the cell of each media
Since the numbers of "~ 255" are used cyclically, the same numbers used in adjacent cycles are stored in the retransmission cell history table 82 so as not to be duplicated in the retransmission cell history table 82. The sequence number is deleted after one cycle has elapsed and before the same number is used as the sequence number.

The transmitting side apparatus transmits these retransmission status tables 8
1 and the retransmission cell history table 82, the procedure for restricting the maximum number of retransmissions of the same cell for each piece of media data is the same as that shown in FIG. 10 except that the number of retransmissions is restricted. Is the same as the processing procedure shown in FIG.
At the time of referring to No. 1 (step S2 in FIG. 10), it also refers to the upper limit number of times that the retransmission processing set for each piece of media data can be performed. Before performing the process of searching the retransmission buffer for a cell to be retransmitted (before step S6 in FIG. 10), the retransmission cell history table 8
2, the number of retransmissions of the cell is detected, and if the number of times of retransmission does not reach the set upper limit, the cell is retransmitted (step S in FIG. 10).
6 to S10 and S15).

When the number of times of retransmission processing has reached the set upper limit number, retransmission processing is stopped for this cell, and it is determined whether there is another cell to be retransmitted. A determination is made (step S10 in FIG. 10), and if there is another cell to be retransmitted, the retransmission processing is performed for this cell. The other processing procedures are the same as the procedures shown in FIG. In this way, the upper limit number of times that the retransmission process can be performed for the same cell for each piece of media data can be set in accordance with the characteristics of each type of each piece of media.

The retransmission status tables 41, 7
The setting contents of 1, 81 may be set in advance by an operator before performing multimedia communication, for example.
Further, for example, based on the rate of occurrence of transmission errors occurring in cells of each media data during communication and the like, the setting contents may be changed as needed by the control units 17 and 67 provided in the transmitting device or the receiving device. You can also. That is,
For example, the receiving apparatus changes the setting contents of the retransmission state table 71 stored in the control section 67 based on the frequency of occurrence of cell transmission errors for each piece of media data detected by the retransmission determination sections 54, 58, and 62. Also, in the transmitting device, the retransmission state table stored in the control unit 17 based on the frequency of receiving the retransmission request notification of the cell for each media data from the receiving device decrypted by the control data decrypting unit 16 The setting contents of 41 and 81 are changed.

Further, for example, information for notifying a change in the setting of the retransmission status table 71 is included in the control data transmitted from the receiving apparatus to the transmitting apparatus, and conversely,
Retransmission status tables 41, 8 from transmitting device to receiving device
By sending information notifying the change of the setting of 1,
Retransmission status table 4 between the transmitting device and the receiving device
The setting contents of 1, 71 and 81 can be the same. As a specific method of performing such a change in the setting content, for example, the threshold value of the error occurrence probability allowable for the cell for each media data is set in the transmitting device or the receiving device in advance, and the detected error probability is detected. When the error occurrence probability is equal to or less than the threshold, the setting is changed so that the retransmission process is not performed, and the retransmission process is regulated. On the other hand, when the detected error occurrence probability exceeds this threshold, the retransmission process is performed. The data quality can be ensured by changing the settings.

Similarly, for example, by setting a threshold value of an error occurrence probability for each cell of each piece of media data for which retransmission processing is set to be performed, the detected error occurrence probability is equal to or smaller than the threshold value. If there is, the setting of the upper limit number of times that the retransmission process can be performed is changed to three times, and if the detected error occurrence probability exceeds this threshold, the upper limit number of times that the retransmission process can be performed. Can be changed to five times to further improve data quality. In this way, settings such as whether or not to perform retransmission processing, and how many times to perform retransmission processing on the same cell at the time of performing retransmission processing are set for each media data during data communication for each medium. By making a change based on the error occurrence probability or the like, even if the quality of a transmission path for a certain medium fluctuates and deteriorates during data communication, for example, a setting is made to perform the retransmission processing of the media data. Or by increasing the upper limit of the number of times that the same cell can be retransmitted when the media data is retransmitted, the quality of the data for each media data according to the characteristics of each media. And multimedia communication can be performed.

Here, in the above embodiment, the retransmission state table shown in FIG. 6, FIG. 12, or FIG. 9 is provided in both the transmission side apparatus and the reception side apparatus. Although the process of restricting the retransmission process by setting whether or not to perform the process was performed, means for performing the setting and the restriction process may be provided in at least one of the transmitting device and the receiving device. I just need. That is, for example, the transmission-side apparatus is provided with a unit for performing the above-described setting and restriction processing, and the transmission-side apparatus is configured to perform the retransmission processing on the ACK or NAK notified from the reception-side apparatus. By performing the retransmission processing on the cell only when the NAK for is received, the same effect as shown in the above embodiment can be obtained.

Further, for example, the receiving apparatus is provided with means for performing the above-mentioned setting and regulating processing only, and the receiving apparatus is configured to perform the retransmission processing on the cell received from the transmitting apparatus. By notifying the transmitting side apparatus of a retransmission request for the cell only when a transmission error has occurred, the same effect as shown in the above embodiment can be achieved. Further, in the above-described embodiment, the encoding process is performed on each media data in the transmitting device, and the decoding process is performed on the receiving device. However, the encoding and decoding processes are not necessarily performed.

Further, in the above embodiment, by using a packet composed of several cells as a transmission unit, the number of transmissions is reduced as compared with a case where a preamble or a unique word is added to each cell and transmitted. Although the configuration is such that data can be transmitted with the transmission amount, a packet is not necessarily used as a transmission unit, and for example, data for each cell may be used as a transmission unit as described above. In short, any configuration may be used as long as it is possible to set whether or not to perform retransmission processing for each piece of media data according to the characteristics of each medium and to restrict the retransmission processing.

Further, in the above-described embodiment, an example has been described in which multimedia communication is performed between the transmission-side device for transmitting data and the reception-side device for receiving data. Using a plurality of communication devices having both a function configuration for performing a transmission process provided in the transmission-side device and a function configuration for performing a reception process provided in the reception-side device, as described above between the plurality of communication devices, Multimedia communication can also be performed. Further, in the above-described embodiment, a wireless transmission path is used as a transmission path for performing multimedia communication. However, the present invention is applied not only to a wireless transmission path but also to a wired transmission path.

In the above embodiment, the processor executes the control program in the hardware resources including the processor and the memory, thereby controlling the process of setting and regulating the retransmission process for each media data. Although the configuration is described, in the present invention, each functional unit for executing the processing may be configured as an independent hardware circuit. Further, the present invention can be understood as a computer-readable storage medium such as a floppy disk or a CD-ROM storing the above-mentioned control program, and the control program is input from the storage medium to the computer and executed by the processor. Thereby, the processing according to the present invention can be performed.

[0080]

As described above, according to the multimedia multiplexing communication system of the present invention, the data of each medium is divided into blocks by the transmitting side device, and the block data of these plural kinds of media are mixed. When multiplexing and transmitting, and receiving the block data sequence by the receiving device and reproducing each media data, it is set whether or not to perform the retransmission processing for each media data according to the characteristics of each media and the like. Since the retransmission processing is regulated for each media data, for example, it is possible to efficiently perform multimedia data communication while ensuring immediacy and transmission quality required for each media data.

Further, according to the multimedia multiplex communication method of the present invention, when dividing data for each medium into blocks and multiplexing and transmitting block data of these plural types of media, the same as above Whether or not to perform retransmission processing for each piece of media data is set in accordance with the characteristics and the like of each medium, and the retransmission processing for each piece of media data is regulated based on the setting. , Multimedia data communication can be performed efficiently.

[Brief description of the drawings]

FIG. 1 is a configuration example of a transmission side device of a multimedia multiplex communication system according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a process of dividing an encoded data stream into cells.

FIG. 3 is a diagram illustrating a configuration of a cell.

FIG. 4 is a diagram illustrating a process of multiplexing cells.

FIG. 5 is a diagram illustrating a configuration of a packet.

FIG. 6 is an example of a retransmission status table in a transmitting device.

FIG. 7 is an example of a retransmission cell table in a transmitting device.

FIG. 8 is a configuration example of a receiving side device of a multimedia multiplex communication system according to an embodiment of the present invention.

FIG. 9 is an example of a retransmission status table in a receiving device.

FIG. 10 is a diagram for explaining a procedure of a retransmission process performed by the transmitting device.

FIG. 11 is a diagram illustrating a procedure of a retransmission request notification process performed by a receiving device.

FIG. 12 is an example of a retransmission status table in a transmitting device.

FIG. 13 is an example of a retransmission cell history table in a transmitting device.

[Explanation of symbols]

1 image input unit, 2 image encoding unit, 3 cell conversion unit, 4 retransmission unit, 5 audio input unit, 6
Voice coding unit, 7 ··· cell conversion unit, 8 ··· retransmission unit, 9
..The data input section, the data encoding section, and the data encoding section
-Celling section, 12-Retransmission section, 13-Multiplexing section,
14 ・ ・ Packetizing part 、 15 ・ ・ Transceiving part 、 16 ・ ・
Control data decoding section, 17 control section, 41 retransmission status table, 42 retransmission cell table, 51 transmission / reception section, 52 packet decomposition section, 53 demultiplexing section, 54 retransmission determination , 55 ... Streaming part,
56 image decoding unit 57 image output unit 58
..Retransmission determination section, 59..Streaming section, 60 ..
Voice decoding section, 61 voice output section, 62 retransmission determination section, 63 streaming section, 64 data decoding section, 65 data output section, 71 retransmission state table, 81 Retransmission status table, 82 retransmission cell history table,

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Mitsuru Mima 3--14-20 Higashinakano, Nakano-ku, Tokyo Inside Kokusai Electric Corporation

Claims (2)

[Claims] 1. A multimedia multiplexing communication system in which a plurality of types of media data blocked by a transmitting device are mixed and multiplexed and transmitted, and a receiving device receives a block data sequence and reproduces each media data. The transmitting device divides each media data into a plurality of block data and adds a check code for detecting a transmission error for each block data, and a mixed arrangement of the generated block data of each media data. Multiplexing means for transmitting the block data sequence generated by the mixed array, and transmitting means for transmitting the block data sequence generated by the mixed array. The receiving device receives the transmitted block data sequence, and Separation means for separating into block data for each media data, Error detecting means for detecting a transmission error based on the check code, and reproducing means for reproducing the media data from the separated block data for each media data. Retransmission request means for notifying the transmitting device of the detected block data is provided. The transmitting device is provided with retransmitting means for performing retransmission processing on the block data notified from the receiving device. Retransmission setting means for setting whether or not to perform retransmission processing for each piece of media data; and retransmission means for each piece of media data based on the setting of the retransmission setting means. And a retransmission control unit for restricting retransmission processing by the multimedia multiplexing communication system. 2. In a multimedia multiplexing communication method for dividing data for each medium into blocks, multiplexing and transmitting block data of these plural types of media, a retransmission process is performed for each type of media data. Multimedia transmission characterized in that, when a transmission error occurs in block data transmitted for media data for which retransmission processing is set, retransmission processing for the block data is performed. Multiplexed communication method.