JPH1056480A - Multimedia multiplex communication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To execute re-transmission processing conducted for occurrence of a transmission error in multimedia communication within a delay time in response to a characteristic for each medium. SOLUTION: In the case that transmission of multiplexed media data of a plurality of kinds of packet processing is conducted by the multimedia multiplex communication method, a time consumed for re-transmission processing for each kind of media data is set in advance to a transmitter side equipment 1, and when a transmission error is generated in transmitted media data packets, the processing of re-transmitting the packet to a receiver side equipment 30 is conducted within the set time. Each packet is added with a check code to detect a transmission error and the re-transmission processing is conducted based on that a confirmation reply of normal reception from the receiver side equipment 30 receiving sent data is not obtained within a prescribed time.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for multiplexing packetized multimedia data, and more particularly, to an apparatus and method for managing the time that can be spent for retransmission processing for each piece of media data.

[0002]

2. Description of the Related Art Conventionally, a plurality of different types of media data such as images, sounds, texts and the like have been communicated as separate data. However, as information communication becomes more diversified, a plurality of types of media data are multiplexed. The need for multimedia communication to communicate is growing. For example, when communicating image data together with audio data using a mobile phone such as a personal handy phone (PHS), conventionally, audio data and image data were transmitted separately without being multiplexed. Since images cannot be transmitted and calls cannot be made during image transmission, it is inconvenient for multimedia communication that requires immediacy. For this reason, it has been demanded that a plurality of different types of media data be multiplexed and communicated simultaneously.

A conventional main method of multiplexing audio data and image data is to allocate a fixed band (for example, 8 Kbps for audio data and 16 Kbps for image data) to each media data before transmission. Met. Here, such data communication is performed via a wireless or wired line. For example, a transmission error may occur in data due to disturbance such as fading. In particular, when data communication is performed using a mobile phone that performs wireless communication in an environment where the output is limited and there is a shield such as a building, the probability of occurrence of a transmission error becomes considerably high. For such a transmission error, a measure is taken to transmit the data with a check code added, detect a transmission error based on the check code, and retransmit the corresponding data.

However, in the above-mentioned conventional multiplexing method, although the immediacy of multimedia communication is obtained,
Since each media data was multiplexed into a transmission frame format and then transmitted with a check code added,
Retransmission processing for transmission errors could not be performed according to the characteristics of each medium. That is, for example, when multiplexing and transmitting audio data and image data,
As shown in FIG. 16, since the audio data and the image data are formed into one transmission frame, and a check code is added to each of these frames and transmitted, for example, even if a transmission error occurs only in a part of the image data, In order to recover the error, the entire frame (audio data + image data) including the error part is retransmitted, and the audio data part which does not need to be retransmitted is retransmitted.

[0005] Naturally, a certain amount of processing time is required for error recovery by such retransmission processing. However, due to the characteristics of media, audio data and image data can be tolerated for error recovery. Different delay time.
For example, audio data must undergo error recovery within at least 10 ms, while image data
The time that can be spent in the retransmission processing differs depending on the type of the medium, such that it is sufficient if the error is recovered within this time. The same applies to data other than voice and image, such as data such as electronic mail of personal computer communication, data of facsimile communication, and data of file transfer.
For this reason, if the time that can be spent in the retransmission processing is set relatively long, the probability that data can be reliably transmitted is improved, but some data exceeds the allowable time for error recovery. A situation has occurred in which the value of itself has been impaired.

[0006]

As described above, in the multimedia communication, the retransmission processing is performed for the occurrence of the transmission error, but the retransmission processing is performed according to the characteristics of each medium. Did not. In particular, when an attempt is made to realize multimedia multiplex communication using a communication means having a relatively low transmission speed and a relatively poor use environment such as a mobile phone, the effect of a transmission error is remarkable. Had become.

The present invention has been made in view of the above-mentioned conventional circumstances, and manages the time that can be spent for retransmission processing for each type of media with respect to each packetized media data. It is an object of the present invention to provide a multimedia multiplex communication method and system that realize a retransmission process corresponding to the multimedia multiplex communication method. Another object of the present invention is to provide a system and method for realizing highly efficient multimedia communication using a communication device having a relatively low transmission speed and a relatively poor use environment such as a mobile phone. And

[0008]

In order to achieve the above object, a multimedia multiplex communication method according to the present invention comprises:
In a multimedia multiplexing communication method for multiplexing and transmitting a plurality of types of packetized media data, a time that can be spent for retransmission processing is set in advance for each type of media data and transmitted to a transmitted media data packet. When an error occurs, retransmission processing for the packet is performed within the set time. Therefore, by setting the time that can be spent in the retransmission processing for each type of media, for example, 10 mS for audio data and 50 mS for image data, the retransmission processing according to the characteristics of the media is executed. You.

In the multimedia multiplex communication method according to the present invention, the set time is set by the number of retransmission processes performed within a predetermined time. That is, by limiting the number of retransmission processes that are repeated until a data packet is normally received for each medium, the time that can be substantially spent for the retransmission process is managed for each type of media. In the multimedia multiplexing communication method according to the present invention, a check code for detecting a transmission error is added to each packet, and the retransmission processing is performed by a receiving apparatus that receives transmitted data to confirm normal reception. The response is based on the failure to obtain a response within a predetermined time. That is, the detection of the transmission error is performed for each medium and each packet based on each check code, and the packet in which the transmission error has occurred is subjected to the retransmission processing under the above management.

Further, in the multimedia multiplex communication system according to the present invention, a packet generating means for packetizing each media data and a packet multiplexing means for mixing and arranging the generated packets of each media data in the transmitting apparatus. And transmission means for adding a check code for detecting a transmission error to each packet in the packet sequence generated by the mixed array, and transmitting the packet sequence. Error detecting means for detecting a transmission error based on the check code; receiving means for receiving the transmitted packet sequence; packet separating means for separating the normally received packet sequence into packets for each media data; Data playback means for playing back media data from a packet for each piece of media data. And transmits the mixed multiplexes packetized plurality of types of media data by Shin-side apparatus receives the packet sequence to reproduce the media data by the receiving device.

Further, the above-mentioned receiving apparatus is provided with retransmission request means for notifying the transmitting apparatus of a packet in which a transmission error has been detected. Retransmission processing means for retransmitting a packet to be transmitted, and retransmission management means for executing the retransmission processing within a time that can be spent for the retransmission processing set in advance for each type of media data. The retransmission processing is performed on the packet in which the error has occurred within the time corresponding to the media type.

[0012] In the multimedia multiplex communication system according to the present invention, the retransmission management means manages the number of retransmission processes performed within a predetermined time and saves time that can be substantially spent in the retransmission process. to manage. Further, in the multimedia multiplexing communication system according to the present invention, the transmitting device further includes a data compression unit for compressing the packetized media data, and the receiving device includes a data decompression device for decompressing the compressed media data. Means, and output means for outputting each piece of media information based on each piece of expanded and reproduced media data, so that data transmission efficiency is improved.

[0013]

Embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows an example of a multimedia multiplex communication system according to the present invention. The multimedia multiplex communication system includes a transmitting device 1 that wirelessly transmits image data, text data, and audio data as a multiplexed packet sequence, and wirelessly receives the multiplexed packet sequence. A receiving device 30 that reproduces image data, text data, and audio data;
Is provided. In this example, for the sake of simplicity, the transmitting device 1 is a device that controls a transmitting function, and the receiving device 30 is a device that controls a receiving function. However, each device has a transmitting function and a receiving function. May be configured as a transmission / reception device having both functions. Further, the multimedia multiplex communication system may be configured to include any two or more of these devices.

First, a camera 2 for capturing an image and inputting image data (video signal), text file means 3 for storing a text file, and voice data for capturing a voice signal (voice signal ) Is connected to a communication device 5 such as a personal handyphone that wirelessly transmits and receives a multiplexed packet sequence and necessary data. Also, the transmitting device 1 includes
A transmission-side encoding unit (transmission CODEC unit) 6 for compressing and encoding the input data of each medium; a transmission packet unit 7 for packetizing the encoded media data to form a multiplexed packet sequence; A communication control unit 8 for transmitting the transmitted packet sequence from the communication device 5 or receiving data such as an acknowledgment described later via the communication device 5 and controls these functional units 6 to 8 collectively and will be described later. And a main control unit 9 for performing retransmission management.

The transmitting-side encoding unit 6 includes an analog / digital converter (A / D converter) 10 for converting a video signal input from the camera 2 into digital image data, and a JPEG format for converting the digitized image data. An encoder 11 for compression-encoding by a method or the like, a text input unit 12 for receiving and inputting text data from the text file unit 3, an encoder 13 for compressing and encoding the input text data, and a microphone 4. An analog / digital converter 14 that converts an input audio signal into digital audio data, and an encoder 15 that compresses and encodes digitized audio data are provided.

The transmission packet unit 7 includes an encoder 1
1, a packet buffer (PAD buffer A) 16 for dividing and storing the image data compressed and encoded by a plurality of packets, and text data compressed and encoded by the encoder 13 into a plurality of packets for storage. Buffer (PAD buffer B) 17, a packet buffer (PAD buffer C) 18 that divides audio data compressed and encoded by the encoder 15 into a plurality of packets and stores the packets, and packet buffers 16 to 18. And a multiplexing buffer (MUX buffer) 19 for storing the extracted data packets of each medium as a multiplexed packet sequence. In addition, the communication control unit 8
Adds a check code (for example, CRC) for detecting a transmission error to each packet for each medium to be transmitted, in addition to the data transmission / reception processing described above.

The main control section 9 has a ROM 2 storing a control program and a definition table for retransmission processing to be described later.
0 and a processor (CPU) that executes the control program
A RAM 22, which is a work area for the CPU 21 and stores a retransmission management table to be described later, and a timer 23 for counting used for various controls such as a retransmission management process. The main control unit 9 executes a control program to execute packetization processing of each media data using the packet buffers 16 to 18, multiplex processing of a packet sequence using the multiplexing buffer 19, The retransmission processing of the packet according to the presence or absence of the acknowledgment, the processing of managing the packet retransmission processing for each medium, and the like are performed.

That is, the main control unit 9 and the packet buffers 16 to 18 constitute packet generation means, and the main control unit 9 and the multiplexing buffer 19 constitute packet multiplexing means. The retransmission processing means for retransmitting the packet is constituted by the means 8 and further includes a retransmission management table of the main control unit 9 and a timer 2 as will be described later.
3 constitutes retransmission management means for managing packet retransmission processing for each medium. Note that these functional units may be configured as circuits for each function, instead of being configured as software units under program control. Further, in the above device configuration, the A / D converters 10 and 1
4. The encoders 11, 13, and 15 can be shared, respectively, and the packet buffers 16 to 18 and the multiplexing buffer 19 can be configured by dividing the area into one memory.

On the other hand, the receiving device 30 includes a display unit 31 such as a liquid crystal display for displaying and outputting an image based on the reproduced image data, and a text file means 32 such as a hard disk device for storing the reproduced text data.
And a speaker 33 that outputs audio based on the reproduced audio data, and a communication device 34 such as a personal handyphone that wirelessly transmits and receives the multiplexed packet sequence and necessary data. The receiving device 30 receives the multiplexed packet sequence via the communication device 34 or transmits data such as an acknowledgment, which will be described later, to the communication device 34.
And a reception packet unit 36 for separating a received packet sequence for each media data
And a receiving-side decoding unit (reception CODEC unit) 37 for decompressing and decoding data separated for each medium, and a main control for controlling these functional units 35 to 37 as well as receiving management to be described later. And a built-in unit 38.

The communication control unit 35 performs a process of detecting the presence or absence of a transmission error based on the check code added to each packet, in addition to the above-described data transmission / reception process, and discards the packet in which the transmission error has occurred. Perform processing. The reception packet section 36 sequentially stores a multiplexing buffer (MUX buffer) 39 for storing data packets of each medium normally received in a multiplexed state, and image data packets taken out from the multiplexing buffer 39. A packet buffer (PAD buffer D) 40 for storing and maintaining the original data stream in a row, and a multiplexing buffer 39
Buffer (PAD buffer E) 41 for sequentially storing the text data packets taken out from the storage device and holding the original data stream in the order of the original data stream;
And a packet buffer (PAD buffer F) 42 for sequentially storing the audio data packets extracted from the data stream and holding them in the original data stream arrangement.

The receiving side decoding unit 37 includes a decoder 43 for expanding and decoding the image data taken out from the packet buffer 40 by a method corresponding to the time of compression, and converts the decoded image data into an analog video signal. A digital / analog converter 44 for converting and outputting to the display unit 31;
A decoder 45 for decompressing and decoding text data extracted from the packet buffer 41 by a method corresponding to compression, an output unit 46 for storing the decoded text data in the text file 32, and a packet buffer 4
A decoder 47 for decompressing and decoding the audio data extracted from the second audio data by a method corresponding to the compression, a digital / analog converter 48 for converting the decoded audio data into an analog audio signal and outputting the analog audio signal to the speaker 33; An encoder 13 for compressing and encoding text data, and a microphone 4
An analog-to-digital converter 14 that converts an audio signal input from a digital signal into digital audio data, and an encoder 15 that compresses and encodes the digitized audio data are provided.

The main control unit 38 stores a control program and a definition table for retransmission monitoring processing to be described later.
OM49 and a processor (C
PU) 50, a RAM 51 serving as a work area for the CPU 50 and storing a reception management table described later, and a timer 52 for counting used for various controls such as reception management processing.
And are provided. The main control unit 38 executes the control program to execute the packet buffer 40
４２42, a process of separating a multiplexed packet sequence into media data packets, a process of managing received packets, and the like.

That is, the main control unit 38 and the packet buffers 40 to 42 constitute a packet separating unit, and the acknowledgment unit for returning the acknowledgment of the normally received packet by the main control unit 38 and the communication control unit 8. Is configured. In this example, as will be described later, a packet without an acknowledgment is retransmitted assuming that a transmission error has occurred. The detected packet is notified. Further, these functional units may be configured as circuits for each function without being configured as software units under program control. Further, in the above device configuration, the D / A converter 44
And 48, and the decoders 43, 45, and 47 can be shared, respectively.
It is also possible to configure the 2 and multiplexing buffer 39 by dividing the area into one memory.

In this embodiment, the data input devices 2 to 3,
The communication device 5 or 34 and the reproduced data output devices 31 to 33 are configured separately from the transmission device 1 or the reception device 30, but a part or all of them are integrated into the device 1 or 30. It is also possible to configure each device in any embedded relationship. In the present embodiment, the camera 2 inputs still image analog data. However, it is also possible to input moving image data from the camera 2 and perform multiplex communication. Further, in this embodiment, three types of media data are multiplexed and communicated. However, any number of media data including other types of data and the like can be multiplexed and communicated, and the combination of the media types can be arbitrarily set. is there.

Next, the packetizing process will be described with reference to FIG.
Will be described in more detail. Note that, for clarity, image data will be described as an example.
The same applies to the packetization processing of text data and audio data 7, and 18. The data stream shown in FIG. 2 is image data that has been compressed and encoded by the encoder 11 after being fetched from the camera 2.
The image data stream output from 1 is divided into fixed-length data blocks, control data is added to each data block to form packets, and these packets are sequentially stored in the packet buffer 16. When no data stream is output from the encoder 11, no packet is generated.

That is, as shown in FIG. 3A, each packet has a format in which a transmission sequence number P (S) as control data and a virtual channel number VC are added to a data portion including a fixed-length data block. Configuration
The media type of each packet is identified by the virtual channel number VC, and each packet is identified as a transmission order by the transmission order number P (S). In this example,
The transmission sequence number P (S) cyclically uses a numerical value of “0” to “7”, and the virtual channel number VC that virtually defines the data communication path is the communication path of the image data packet. (A communication path between the packet buffer 16 and the packet buffer 40) is VC1, a communication path for a text data packet (a communication path between the packet buffer 17 and the packet buffer 41) is VC2, and a communication path (a packet The communication path between the buffer 18 and the packet buffer 42) is VC3.

The packet buffer 16 is a ring buffer for sequentially and sequentially storing packets in a first-in first-out manner. Each time a packet is stored, the value of the PUT pointer is updated one by one by the main controller 9 so that a valid packet is stored. It indicates that there is, and the value of the GET pointer is updated one by one each time a packet is taken out by the main controller 9 and stored in the multiplexing buffer 7. Therefore, the difference between the values described in the PUT pointer and the GET pointer indicates the number of packets stored in the packet buffer 16. In addition, as described later, the main controller 9 transmits the packet stored in the multiplexing buffer 7 to the receiving side device 30 every time the acknowledgment for the packet is received from the receiving side device 30. The discard pointers are updated one by one to indicate the packet positions where transmission was successful.

For example, in the state shown in FIG. 2, the fact that the PUT pointer is after packet C indicates that the next packet cut from the data stream is stored under packet C, and the GET pointer is B indicates that the packet to be stored next in the multiplexing buffer 7 is packet C because it is after B, and waits for the reception of an acknowledgment for packet A because the discard pointer is before packet A. It indicates that.

Here, the acknowledgment is a packet having no data part as shown in FIG.
(R) and a virtual channel number VC. The reception sequence number P (R) is a transmission sequence number P (S) given to a packet that can be normally received.
The virtual channel number VC is a virtual channel number VC assigned to the packet, and the receiving sequence number P (R) and the virtual channel number VC make it possible for any packet on any medium without causing a transmission error. Indicates whether or not the data has been normally received by the device 30. The acknowledgment packet is for notifying that the packet has been normally received from the receiving device 30 to the transmitting device 1, and is not stored in the packet buffer. Further, since the acknowledgment packet notifies a normally received packet, the fact that the acknowledgment packet cannot be received substantially notifies that a transmission error has occurred in the packet. Note that, in the present invention, when a transmitted packet cannot be received normally, the receiving device 30 transmits a retransmission request,
Retransmission processing may be performed.

Next, with reference to FIG. 4, the multiplexing process of the above-described packet sequence will be described. By the above-described packetization processing, a plurality of image packets are stored in the image packet buffer 16 in the order in the image data stream such as “image packet 1”, “image packet 2”,. Are stored in accordance with the sequence in the text data stream such as "text packet 1", "text packet 2",.
., A plurality of audio packets are stored according to the order in the audio data stream, such as “audio packet 1”, “audio packet 2”, and so on. As shown in FIG. 4, the main control unit 9 alternately takes out packets from the respective packet buffers 16 to 18 and sequentially stores them in the multiplexing buffer 19, and stores the packets in the multiplexing buffer 19 in the order of storing them. 8 to be transmitted. That is, "image packet 1", "text packet 1", "
A multiplexed packet sequence such as “audio packet 1”, “image packet 2”, “text packet 2”, “audio packet 2”... Is transmitted to the receiving device 30.

Such a multiplexed packet sequence is constituted by the communication control means 8, and the communication control means 8 sets a check code (for example, CRC) for detecting a transmission error for each packet when forming the packet sequence. Add. That is, as shown in FIG. 5, a check code is added to each packet included in the multiplexed packet sequence and transmitted, and the detection of a transmission error is performed by the communication control unit 35 of the receiving side device 30 for each packet. Will be Note that the packet in which the transmission error has been detected is discarded by the communication control unit 35 and becomes a target of retransmission processing.

Next, with reference to FIG. 6, a process of separating the multiplexed packet sequence into packets for each medium in the receiving device 30 will be described. This demultiplexing process is performed using the packet buffers 40 to 42 and the multiplexing buffer 39 under the control of the main control unit 38. It has been confirmed that the communication control unit 35 has no transmission error in the multiplexing buffer 39. Normally received packets are sequentially stored. Then, the main control unit 38 assigns each packet stored in the multiplexing buffer 39 to its virtual channel number VC.
, The packets are sequentially stored in the corresponding packet buffers 40 to 42, and the packets separated for each medium are held in the packet buffers 40 to 42 in the order according to the data stream at the time of transmission. The PUT pointer of each of the packet buffers 40 to 42 is updated each time a packet is stored to indicate that a valid packet is held, and the GET pointer indicates that the packet is sent to the decoder for reproduction output. Updated each time it is retrieved.

The multiplexed packet sequence as described above is transmitted from the transmitting device 1, received by the receiving device 30, and each medium is reproduced as the original data. Retransmission management is performed based on presence / absence. The retransmission management according to the present embodiment is performed by the number of times that retransmission processing can be performed for each packet of each medium in which a transmission error has occurred. As shown in FIG. 7A, parameters for performing management are defined. The table obtained is stored in the ROM 20 of the transmitting apparatus 1 in advance. The definition table stores a retransmission timer value and a retransmission count value for each piece of media data. The retransmission timer value defines a delay time until reception of an acknowledgment for performing retransmission processing. The number of times that the retransmission process can be repeatedly performed is defined by a numerical value.

The ROM 49 of the receiving device 30 stores
As shown in FIG. 7B, a table in which parameters for performing the discontinuation management of the retransmission processing are defined is stored in advance, and this definition table performs the retransmission processing for each media data on the receiving apparatus 30 side. The timer value for terminating the timer is stored as the reception monitoring timer value. here,
The parameters in the definition tables shown in FIGS. 7A and 7B can be rewritten as needed, and the number of retransmissions for each medium (that is, the time that can be spent for retransmission processing) described later is changed in the system environment or the like. It is also possible to change according to.

The above-mentioned retransmission management is performed by the main controller 9 in the RAM 22.
Is performed using the retransmission management table stored in. Here, the retransmission management table stored in the RAM 22 will be described with reference to FIG. The retransmission management table is provided for each of the packet buffers 16 to 18 of each medium. For each medium, the number of the transmission order number P (S) is transmitted, and the number of the acknowledgment is received. Further, a timer value allowed until an acknowledgment is obtained for each packet and a count value at the time of performing retransmission processing are managed. That is, in the retransmission management table, the transmission order number P cyclically assigned to the packet for each medium is stored.
A transmission P (S) pointer indicating (S), a reception P (R) pointer indicating the reception sequence number P (R) included in the acknowledgment packet, and a transmission sequence number from “0” to “7” For each of the packets, there is provided an area for a timer value that defines a delay time allowed before retransmission processing is performed and an area for the number of retransmissions that specifies the upper limit number of retransmission processing.

Next, the procedure of the retransmission management process will be specifically described with reference to FIG. This retransmission management process is also executed independently for each of the packet buffers 16 to 18. First, the PUT pointer and the GET pointer of the packet buffer are compared to determine whether there is a packet to be transmitted in the packet buffer (step S1). If there is a packet, the packet is stored in the multiplexing buffer 19. On the other hand, if there is no packet (step S2), the determination processing of step S1 is repeated. After storing the packet to be transmitted in the multiplexing buffer 19 as described above, the transmission P (S) pointer of the retransmission management table is updated (+1) (step S3), and the packet of the corresponding sequence number in the transmission management table is updated. The retransmission timer value defined in the definition table (FIG. 7A) is set and the timer 23 is started (step S4), and the retransmission count value defined in the definition table is retransmitted for the packet. The retransmission count value is set in the management table (step S5).

Next, the count value of the timer 23 is compared with the retransmission timer value in the transmission management table to determine whether there is a packet whose time has expired (step S6). Is repeated, if there is a packet whose time is up, the corresponding number of retransmissions in the retransmission management table is updated (−1) in order to perform retransmission processing on the packet (step S7). Then, the updated retransmission count value is
0 ", that is, whether or not the specified upper limit of the number of retransmissions has been reached (step S8). If the specified value has not yet been reached, the transmission management table is used to further retransmit the packet. (1) Returning the transmission P (S) pointer, and repeating the above processing (step S9), that is, returning the transmission P (S) pointer to a state where the packet has not been transmitted yet. Then, transmission processing (that is, retransmission processing) is performed.

On the other hand, when the value reaches the specified value, the time that can be spent for the retransmission processing has elapsed, so that the retransmission processing is aborted for the packet in which the transmission error has occurred. Is handled in the same manner as a normally received packet, the transmission P (S) pointer in the retransmission management table is updated to perform retransmission management for the next packet (step S10), and the discard pointer in the packet buffer is updated (step S10). S11).
Therefore, the retransmission processing is performed on the packet for each medium only within the time defined by the number of retransmissions, and the retransmission processing is performed with a delay time that is allowed according to the characteristics of each medium.

Here, the above-described retransmission management processing is performed only in a state where the acknowledgment from the receiving side apparatus 30 is not obtained, and when the acknowledgment is obtained, the processing shown in FIG. Then, the corresponding packet is excluded from the target of the retransmission processing. That is, when the acknowledgment packet is received, it is determined which medium is the response from the virtual channel number VC, and the corresponding retransmission management table is specified (step S21). Next, by adding a value obtained by adding 1 to the reception order number P (R) of the acknowledgment packet to the transmission P (S) pointer of the retransmission management table, the pointer is updated and the packet is subjected to transmission processing. (Step S22). And
The discard pointer of the corresponding packet buffer is updated (step S23), and the packet is removed from the packet buffer.

The above-mentioned acknowledgment is performed by the main control unit 38 of the reception-side apparatus 30 using the reception management table stored in the RAM 51, and is specifically executed according to the processing procedure shown in FIG. Here, the reception management table stored in the RAM 51 will be described with reference to FIG. The reception management table stores the packet buffers 40 to 4 of the respective media.
2 for each medium, and manages the number of the transmission sequence number P (S) that is expected to be received for each medium. The timer value allowed until the acknowledgment packet is transmitted is managed for each packet. That is, in the reception management table, the reception P (S) expected value indicating the transmission sequence number P (S) cyclically assigned to the packet for each medium, and the sequence number expected to be received are “0” to "7"
For each of the packets up to, a timer value area is provided which defines a delay time allowed until an acknowledgment packet is transmitted even though the packet is not normally received.

Next, the procedure for transmitting an acknowledgment will be specifically described with reference to FIG. This process is also executed independently for each of the packet buffers 40 to 42. First,
The count value of the timer 52 started in step S37 to be described later is compared with the timer value of the reception management table to determine whether there is a packet whose time has expired (step S31). In step S32, it is confirmed whether there is a packet that can be received normally (step S32). As a result, if there is still no normally received packet, the above process is repeated,
If there is a normally received packet, the transmission sequence number P (S) of the packet and the reception P in the reception management table
(S) It is determined whether the expected value matches (step S33).

If the result of this determination is that they match, an acknowledgment packet is transmitted to the transmitting device 1 (step S3).
4), the reception P (S) expected value in the reception management table is updated (+1) to prepare for reception of the next packet (step S3).
5) Output the data part of the normally received packet from the packet buffer to the corresponding encoder (step S3)
6). On the other hand, as a result of the determination (step S33), if the transmission sequence number P (S) of the normally received packet does not match the expected value of the reception P (S) in the reception management table, the communication error occurs. Since there is a packet discarded by the control means 35, the timer 52 is started to regulate the retransmission processing for the discarded (dropped) packet within a predetermined time (step S37), and the above-described step S31 is started. The subsequent processing is repeated.

When the count value of the timer 52 reaches the timer value of the reception management table (that is, the timer value defined in the ROM 49) (step S3).
1), the packet was retransmitted, but could not be received normally within a predetermined time, so that an acknowledgment packet is transmitted to the transmitting device 1 to terminate the receiving operation (step S38), and the packet is received. Management table reception P (S)
The expected value is updated (+1) to prepare for reception of the next packet (step S39). Therefore, the receiving device 30
The retransmission processing time for each medium is also regulated by the transmission processing of the response confirmation by.

FIGS. 13 to 15 show a second embodiment of the present invention. In the above-described embodiment, the time allowed for the retransmission processing for each packet of each medium is regulated by the number of retransmissions. In this embodiment, the time allowed for the retransmission processing is directly defined in advance. For this reason, the RAMs of the transmission side device 1 correspond to the packet buffers 16 to 18, respectively.
The retransmission management table stored in 22 is changed to the configuration shown in FIG. The retransmission management table is provided for each of the packet buffers 16 to 18 of each medium. For each medium, the number of the transmission order number P (S) is transmitted, and the number of the acknowledgment is received. Furthermore, a response delay timer value allowed until an acknowledgment is obtained for each packet, a retransmission monitoring timer value allowable when retransmission processing is performed, and an identification flag indicating that retransmission processing is being performed. Manage.

That is, in the retransmission management table, the transmission P (S) pointer indicating the transmission sequence number P (S) cyclically given to the packet for each medium, and the reception sequence number P ( R) Receive P indicating
(R) pointer, and further, the transmission sequence number is “0” to “”.
For each packet up to 7 ", a retransmission monitoring timer value that specifies an upper limit time during which retransmission processing is allowed, and a response delay that specifies a delay time allowed before retransmission processing is performed with respect to a confirmation response delay. An area for a timer value and a transmission identification flag indicating whether the transmission process for the packet is a retransmission process is provided.

The retransmission monitoring timer and the response delay timer are started when a packet for each medium is transmitted (first non-retransmission), and the transmission identification flag is set to “1” by the first transmission. You. By starting the response delay timer, it is managed whether an acknowledgment of the packet is obtained within a predetermined response delay time (ie, whether retransmission is performed), and by setting this transmission identification flag, When the packet is subsequently transmitted, it is identified as retransmission. Also, by starting the retransmission monitoring timer, the time during which retransmission processing is allowed for the packet is managed.

In order to perform retransmission management, the transmitting apparatus 1
The definition table previously stored in the ROM 20 of FIG.
4, the definition table stores a retransmission monitoring timer value that defines an upper limit time during which retransmission processing is permitted for each piece of media data. That is, the maximum time during which retransmission processing can be repeatedly performed is specified by the retransmission monitoring timer value. In the retransmission management processing, the timer value (time) spent on the retransmission processing is compared with the retransmission monitoring timer value. Is done. The reception management table stored in the RAM 51 of the receiving device 30 and the definition table stored in the ROM 49 are the same as those in the above-described embodiment.

Next, with reference to FIG. 15, the procedure of the retransmission management processing in this embodiment will be specifically described. This retransmission management process is also executed independently for each of the packet buffers 16 to 18. First, it is determined whether there is a packet to be transmitted in the packet buffer by comparing the PUT pointer and the GET pointer of the packet buffer (step S4).
1) If there is a packet, the packet is stored in the multiplexing buffer 19 (step S42), while if there is no packet, the determination processing of step S41 is repeated. Then, after storing the packet to be transmitted in the multiplexing buffer 19 as described above, the transmission P (S) pointer of the retransmission management table is updated (+1) (step S4).
3) Start the timer 23 for measuring the response delay time for the packet with the corresponding sequence number in the transmission management table (step S44).

Next, the transmission identification flag in the retransmission management table is checked (step S45), and the flag is set to "0".
If the packet is not a retransmission process, the timer 23 for counting the retransmission permissible time of the packet is started in order to manage the time when the retransmission process is performed later on the packet. (Step S46). At the same time as performing this processing, the transmission identification flag is set to “1”. On the other hand, if the transmission identification flag is "1", the transmission processing to be performed for the packet is the second or later (that is, retransmission), and the retransmission monitoring timer has already been set to the star. Move to.

Next, the count value of the timer 23 is compared with the response delay timer value in the transmission management table to determine whether there is a packet whose time has expired (step S4).
7) If there is no type-up packet, the above processing is repeated. On the other hand, if there is a packet whose time has expired, the timer 23 compares the count value of the timer 23 with the retransmission monitoring timer value of the transmission management table to confirm whether there is time to perform the retransmission processing on the packet. It is determined whether the process has been performed (Step S48).
As a result, if the time has not expired, the transmission P (S) pointer of the transmission management table is returned (−) in order to further repeat the retransmission of the packet.
1) The above processing is repeatedly performed (step S49). That is, by returning the transmission P (S) pointer, the packet is managed so as not to be transmitted yet, and the transmission processing (that is, the retransmission processing) is performed.

On the other hand, when the time is up, the time that can be spent for the retransmission processing has elapsed, so that
In order to abort the retransmission processing for the packet in which the transmission error has occurred, the transmission error packet is handled in the same manner as the normally received packet, and the transmission P (S) pointer in the retransmission management table is updated to determine the next packet. (Step S50), and updates the discard pointer of the packet buffer (step S5).
1). Therefore, the packet for each medium is retransmitted only within the time specified by the retransmission monitoring timer value,
Retransmission processing is performed with a delay time that is allowed according to the characteristics of each medium.

In the above embodiment, an example in which data input by a camera or a microphone is transmitted by packet communication has been described.
The media data stored in advance in the storage device may be packet-communicated, and the present invention can also be applied to multimedia data that has been packetized in advance. In the above embodiment, the media type of each packet is identified by the virtual channel number. However, media type information may be added to each packet.

[0053]

As described above, according to the present invention,
In communication of a plurality of types of multiplexed media data, retransmission processing for the occurrence of a transmission error is performed within a preset time for each medium, so that excessive delay time due to the retransmission processing is prevented. Thus, data communication according to the characteristics of each medium can be realized. In particular, the relay transmission speed is low and fixed (for example, PH
The present invention has a remarkable effect when applied to the case where multimedia communication is performed using a wireless communication means of 32 Kbps in S, and a simple videophone system, a videophone system, a remote instruction system, a construction attending system, a remote Application to a wide field such as a monitoring system is also possible.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a multimedia multiplex communication device according to an embodiment of the present invention.

FIG. 2 is a conceptual diagram illustrating packetization of a data stream.

FIG. 3 is a diagram illustrating the configuration of a data packet and an acknowledgment packet.

FIG. 4 is a conceptual diagram illustrating multiplexing of packets.

FIG. 5 is a diagram illustrating a transmission frame configuration of a multiplexed packet sequence.

FIG. 6 is a conceptual diagram illustrating separation of a multiplexed packet sequence.

FIG. 7 is a conceptual diagram illustrating a definition table of a transmitting device and a receiving device.

FIG. 8 is a conceptual diagram illustrating an example of a transmission management table provided in a transmitting device.

FIG. 9 is a flowchart illustrating an example of a procedure of a retransmission management process.

FIG. 10 is a flowchart illustrating an example of a procedure of an acknowledgment receiving process.

FIG. 11 is a conceptual diagram illustrating an example of a reception management table provided in a reception-side device.

FIG. 12 is a flowchart illustrating an example of a procedure of a confirmation response transmission process.

FIG. 13 is a conceptual diagram illustrating an example of a retransmission management table provided in a transmitting device.

FIG. 14 is a conceptual diagram illustrating a definition table of a transmitting device.

FIG. 15 is a flowchart illustrating another example of the procedure of the retransmission management process.

FIG. 16 is a diagram illustrating a conventional transmission frame configuration of a multiplexed packet sequence.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Sending apparatus, 2 ... Camera, 3 ... Text file, 4 ... Microphone, 5 ...
Communication means, 6: Transmission encoding section, 7: Transmission packet section, 8: Communication control means, 9: Main control section, 30: Receiving device, 31: Display section , 3
2 ... text file, 33 ... speaker, 3
4 ... communication means, 35 ... communication control means, 36
..Reception packet part, 37... Reception decoding part, 38
... Main control unit,

Claims (6)

[Claims] In a multimedia multiplexing communication method for multiplexing and transmitting a plurality of types of media data packetized, a time that can be spent for retransmission processing is set in advance for each type of media data, and transmitted. A multimedia multiplex communication method, wherein when a transmission error occurs in a media data packet, retransmission processing for the packet is performed within the set time. 2. The multimedia multiplex communication method according to claim 1, wherein the set time is set by the number of retransmission processes performed within a predetermined time. . 3. The multimedia multiplex communication method according to claim 1, wherein a check code for detecting a transmission error is added to each packet, and the retransmission processing includes receiving transmitted data. A multimedia multiplexing communication method is performed based on the fact that an acknowledgment of normal reception is not obtained within a predetermined time from a receiving-side device that performs communication. 4. A multimedia multiplexing communication system in which a plurality of types of media data packetized by a transmitting device are mixed and multiplexed and transmitted, and a receiving device receives a packet sequence and reproduces each media data. The transmitting apparatus includes a packet generating unit that packetizes each piece of media data, a packet multiplexing unit that mixes and arranges the generated packets of each piece of media data, and a packet error generated by the mixed array for each packet. Transmitting means for adding a check code for detecting a transmission error, wherein the receiving-side apparatus detects an error in transmission of each packet in the transmitted packet sequence based on the check code; Receiving means for receiving the received packet sequence, and transmitting the normally received packet sequence for each media data. A packet separating unit for separating the packet into packets, and a data reproducing unit for reproducing media data from the separated packets for each media data. Retransmission requesting means for notifying to the transmission device, the transmitting apparatus can spend retransmission processing means for retransmitting the corresponding packet based on the notification, and retransmission processing preset for each type of media data. And a retransmission management means for executing the retransmission processing within a time period. 5. The multimedia multiplexing communication system according to claim 4, wherein the retransmission management means manages a time that can be spent on the retransmission processing by the number of retransmission processing performed within a predetermined time. Multimedia multiplex communication system. 6. The multimedia multiplexing communication system according to claim 4, wherein the transmitting device further comprises data compression means for compressing the media data to be packetized, and wherein the receiving device comprises A multimedia multiplexing communication system, further comprising: data decompression means for decompressing expanded media data; and output means for outputting each piece of media information based on each decompressed and reproduced media data.