JP2007336382A - Image transmitting apparatus and image transmitting method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To keep image quality excellent while maintaining a real-time property even if data are lacked by easily applying redundant data to all image regions. <P>SOLUTION: In a transmitting apparatus 100, a high frequency component in a DCT processing result of a compression coding step is replaced with redundant data of the other pixel block in a redundant processing block 107 and transmitted. In a receiving apparatus, on the other hand, when a lack of data is detected, complementary processing is performed with the redundant data placed within the high frequency component in a lack complementary processing block 120. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image transmission apparatus, an image transmission method, and a program, for example, a technique suitable for use in decoding or displaying a compressed image for display or output.

  As personal computers become more sophisticated and popular in recent years, and the speed of communication media has increased, the diversity of data exchange between remote computers has expanded, and it has become normal to send and receive moving images over computer networks. It was. Along with this, a service for distributing a video stream in real time, such as Real Networks' Helix Server, has been put into practical use.

  However, if the communication path is congested or the communication status is bad for some other reason, not all of the image information that should have been delivered will be available, and this will result in real-time image transmission. The received image is greatly deteriorated. The following techniques exist as a method for interpolating a deteriorated received image.

  For example, an example in which a defective portion is interpolated using the immediately preceding frame is disclosed (for example, see Patent Document 1). In this method, an image interpolation unit is provided in the receiving apparatus, and interpolation processing is performed with reference to past image frames.

  There is a technique generally called FEC (Forward Error Correction). In this method, a redundant code is added in advance on the transmission side so that data can be restored on the reception side even if there is a packet loss during transmission. Redundant coding is known that uses Tornado code, LT code, and Reed-Solomon code. n packets are grouped, and m packets are made redundant based on the grouped packets (n <m). When packet loss occurs, the content of the lost packet can be restored using the remaining packets in the group (see, for example, Patent Document 2).

  There is also a method for selectively applying FEC. For example, a technique is disclosed in which priority is given to each part of a picture and a redundant code is added only to a place where the priority is set high (see, for example, Patent Document 3). In addition, a method of adding a redundant code only to frame data that has a large influence when lacking is disclosed (for example, see Patent Document 4).

JP-A-62-61485 JP 2001-45098 A JP 2000-134619 A JP 2004-48195 A

  However, in the conventional example, when scene switching occurs, interpolation processing is performed using unrelated data, which has a fundamental problem in image quality. In the conventional example, a band proportional to the error strength is required. When the band to be used is constant, the image quality is deteriorated, and when the image quality is constant, the band to be used is increased. Furthermore, the load for processing for creating redundant data and interpolation processing using the redundant data is increased, and real-time performance may be impaired. Further, in other conventional examples, error tolerance in portions where FEC is not applied is lowered, and image quality may be greatly deteriorated in those portions.

  In view of the above-described problems, the present invention can easily provide redundant data to the entire area of an image to maintain good image quality while maintaining real-time characteristics even when data loss occurs. The purpose is to be able to.

  An image transmission apparatus according to the present invention is an image transmission apparatus that compresses and encodes image data and transmits and receives the image data. The image transmission apparatus includes a replacement unit that replaces a high-frequency component of a DCT conversion processing result in a compression encoding process with redundant data. And an image receiving device having complementing means for performing complementation processing using the redundant data when data loss is detected.

  The image transmission method of the present invention is an image transmission method for compressing and encoding image data and transmitting / receiving the image data. In the image transmission device, a high-frequency component of a DCT conversion processing result in the compression encoding process is replaced with redundant data; The receiving apparatus includes a complementing step of performing a complementing process using the redundant data when data loss is detected.

  The program of the present invention is characterized in that each step of the above method is executed by a computer.

  According to the present invention, redundant data is inserted into transmission data, and data is complemented when data is lost, without impairing real-time performance of the entire image data and without unnecessarily compressing the network bandwidth. be able to.

(First embodiment)
The schematic configuration of the image transmission apparatus according to the first embodiment of the present invention will be described with reference to the block diagram shown in FIG.
In the image transmission apparatus according to the present embodiment, a transmission apparatus 100 and a reception apparatus 101 are connected via a network 102. The transmission apparatus 100 transmits the input image data that has been compression-encoded to the reception apparatus 101 via the network 102. Further, the receiving apparatus 101 decompresses and decodes the transmitted compression-encoded image data and outputs it.

First, the configuration of the transmission apparatus 100 will be described.
An image input interface 103 inputs source image data to be transmitted. Reference numeral 104 denotes a first buffer, which stores source image data input by the image input interface 103. Reference numeral 105 denotes a DCT processing block, which performs DCT conversion on the data stored in the first buffer 104.

  Reference numeral 106 denotes a second buffer, which stores the data DCT converted by the DCT processing block 105. Reference numeral 107 denotes a redundancy processing block, which performs redundancy processing on the data stored in the second buffer 106. Reference numeral 108 denotes a quantization processing block, which performs quantization processing on the data stored in the second buffer 106. Reference numeral 109 denotes an encoding processing block, which encodes the data quantized by the quantization processing block 108.

  Reference numeral 110 denotes a third buffer, which stores data encoded by the encoding processing block 109. Reference numeral 111 denotes a packetizing processing block, which generates a packet using the data stored in the third buffer 110 as a payload. A transmission interface 112 transmits the packet generated by the packetizing processing block 111 to the network 102.

Next, the configuration of the receiving apparatus 101 will be described.
A reception interface 113 receives a packet from the network 102. Reference numeral 114 denotes a depacketization processing block that expands a packet received by the reception interface 113.

  Reference numeral 115 denotes a fourth buffer, which stores the data developed from the packet by the depacketization processing block 114. Reference numeral 116 denotes a loss detection processing block, which detects packet loss from data stored in the fourth buffer 115. Reference numeral 117 denotes a decoding processing block, which decodes data stored in the fourth buffer 115.

  Reference numeral 118 denotes an inverse quantization processing block that dequantizes the data decoded by the decoding processing block 117. Reference numeral 119 denotes a fifth buffer, which stores the data inversely quantized by the inverse quantization processing block 118. Reference numeral 120 denotes a missing complement processing block, which performs missing complement processing on the data stored in the fifth buffer 119.

  Reference numeral 121 denotes an IDCT processing block, which performs inverse DCT conversion on the data stored in the fifth buffer 119. Reference numeral 122 denotes a sixth buffer, which stores the data obtained by the inverse DCT conversion by the IDCT processing block 121. An image output interface 123 outputs data stored in the sixth buffer 122.

Hereinafter, a series of processes in the block diagram shown in FIG. 1 will be described. In the present embodiment, an operation of transmitting an image of 640 pixels in the horizontal direction and 480 pixels in the vertical direction is illustrated.
The image input interface 103 stores the input image data in the first buffer 104. The DCT processing block 105 sequentially reads out the image data stored in the first buffer 104 in units of 8 × 8 pixels, performs DCT conversion, and sequentially stores the data resulting from the DCT conversion in the second buffer 106. .

FIG. 2 shows a data array of 8 × 8 pixel block DCT conversion results stored in the second buffer 106.
In FIG. 2, DC (i, j) 200 indicates the DC component of the 8 × 8 pixel block in the i-th horizontal direction and the j-th vertical direction. In this embodiment, since the pixel size of the input image is 640 pixels in the vertical direction and 480 pixels in the horizontal direction, i takes a value from 0 to 79, and j takes a value from 0 to 59. ACk (i, j) 201 to 207 indicate the k-th AC component of the 8 × 8 pixel block in the i-th horizontal direction and the j-th vertical direction.

  FIG. 3 shows the basis vectors after DCT transformation. The data after DCT conversion of each 8 × 8 pixel block is arranged in the order of zigzag scanning of the basis vectors in FIG. 3, and k takes a value from 1 to 63. The AC component having a larger value of k belongs to the high frequency region.

  When the DCT conversion processing by the DCT processing block 105 is completed and the data of the DCT conversion processing result is stored in the second buffer 106 in the manner shown in FIG. 2, the redundancy processing block 107 starts the redundancy processing. The redundancy processing block 107 replaces the two AC components on the highest frequency region side with the DC components of two 8 × 8 pixel blocks adjacent in the vertical direction. In the present embodiment, AC 62 (i, j) is replaced with DC (i, j−1), and AC 63 (i, j) is replaced with DC (i, j + 1).

  On the other hand, an 8 × 8 pixel block located at the top in the vertical direction (8 × 8 pixel block where j = 0) or an 8 × 8 pixel block located at the bottom in the vertical direction (8 × 8 pixels where j = 59) The block) has 8 × 8 pixel blocks adjacent in the vertical direction only on one side. In such a case, the redundancy processing block 107 replaces the DC component of the other 8 × 8 pixel block with the DC component of the other 8 × 8 pixel block instead of the non-existing 8 × 8 pixel block DC component. In the present embodiment, AC 62 (i, 0) is replaced with DC (i, 59), and AC 63 (i, 59) is replaced with DC (i, 0).

FIG. 4 shows a data array stored in the second buffer 106 after the redundancy processing by the redundancy processing block 107.
In FIG. 4, 400 is the DC component of the 8 × 8 pixel block of the subscript (i, j−1), and 401 is the DC component of the 8 × 8 pixel block of the subscript (i, j + 1).

  When the redundancy processing by the redundancy processing block 107 is completed and the DCT conversion processing result modified in the manner shown in FIG. 4 is stored in the second buffer 106, the quantization processing block 108 sequentially reads out the data, The process is applied. The data that has undergone the quantization process is transferred to the encoding process block 109, where the encoding process is performed. The data encoded by the encoding processing block 109 is stored in the third buffer 110.

  When encoding processing by the encoding processing block 109 is completed and encoded data is stored in the third buffer 110, the packetizing processing block 111 sequentially reads out the data and converts it into a communication packet. The packetizing processing block 111 adds a sequence number to each packet when packetizing. Note that the sequence number is added so that the image can be identified and the number of data in the image can be identified. One packet can include encoded data of a plurality of 8 × 8 pixel blocks, and the 8 × 8 pixel blocks in the packet are arranged in the order of the subscripts (i, j) described above. Included.

  When the packetizing processing block 111 completes a communication packet by adding headers necessary for transmission on the other network 102, the packetizing processing block 111 sequentially passes the packet to the transmission interface 112. The transmission interface 112 transmits the passed packet to the network 102.

  A packet transmitted from the transmission apparatus 100 is transmitted to the reception apparatus 101 on the network 102. The reception interface 113 receives a packet from the network 102 and passes the received packet to the depacketization processing block 114.

  When the depacketizing processing block 114 detects that data transmission of a new image has started by interpreting the sequence number in the packet, the depacketizing processing block 114 secures an area for the fourth buffer 115 and initializes the secured area. . Further, the depacketization processing block 114 checks whether or not a communication error has occurred in the passed packet. This check uses a technique such as CRC.

  A packet in which the occurrence of a communication error is not detected is stored in the fourth buffer 115 after removing the header added by the packetizing processing block 111 in the transmitting apparatus 100. When stored in the fourth buffer 115, it is stored at an address corresponding to the sequence number added to the packet. That is, the encoded data is arranged in the fourth buffer 115 in the order of the subscript (i, j) of the 8 × 8 pixel block.

  A packet in which the occurrence of a communication error is detected is discarded at this point. When the reception interface 113 starts receiving a packet including the image data and a certain time has elapsed, the defect detection processing block 116 starts operating. The fourth buffer 115 stores 8 × 8 pixel block encoded data in the order of subscripts (i, j). Also, the location in the fourth buffer 115 corresponding to the data included in the packet that has been detected and discarded due to the communication error and the data included in the packet that has arrived late due to communication delay is left initialized. ing.

  The defect detection processing block 116 finds a portion that remains initialized in units of one packet, and notifies the defect complement processing block 120 of the portion as a defect portion. Details of the operation of the defect complementation processing block 120 will be described later. After that, the decoding processing block 117 sequentially reads the encoded data stored in the fourth buffer 115 and performs decoding. The decoded data is transferred to the inverse quantization processing block 118, and the inverse quantization processing block 118 performs the inverse quantization processing.

  The data subjected to the inverse quantization process is stored in the fifth buffer 119. Note that the processing by the decoding processing block 117 and the processing by the inverse quantization processing block 118 are skipped for the missing portion of the data that remains in the fourth buffer 115 while being initialized. Also in the fifth buffer 119, data is arranged in the order of the subscript (i, j) of the 8 × 8 pixel block. If no data loss occurs, the data stored in the fifth buffer 119 is the same as that shown in FIG. When data loss occurs, for example, it is as shown in FIG.

  In FIG. 5, the data corresponding to the 8 × 8 pixel block whose subscripts (i, j) are (0,20), (1,20), (2,20), (3,20) are lost. Is shown. In FIG. 5, reference numerals 500 to 503 denote invalid areas that are “invalid”. The invalid areas 500 to 503 store data not related to the data of the image currently being processed, and are identified by the defect complementation processing block 120 based on information notified from the defect detection processing block 116. It is what is done.

  The missing complement processing block 120 performs missing data complementing processing based on the information notified from the missing detection processing block 116. The data complement of the 8 × 8 pixel block of the subscript (i, j) is the 8 × of the subscript (i, j) included in the data of the 8 × 8 pixel block of the subscript (i, j−1). The DC component of the 8 pixel block or the DC component of the 8 × 8 pixel block of the subscript (i, j) included in the data of the 8 × 8 pixel block of the subscript (i, j + 1) is subscript (i , J) is copied to the place where the DC component of the 8 × 8 pixel block is supposed to be. Then, the value 0 is written in the AC component.

  In the example shown in FIG. 5, the DC component of the 8 × 8 pixel block with the subscript (0, 20) is included in the data of the 8 × 8 pixel block with the subscript (0, 19) or the subscript (0, 21). The DC component of the 8 × 8 pixel block of the subscript (0, 20) included in is copied. And it complements by making all AC components into the value 0. Further, the DC component of the 8 × 8 pixel block of the subscript (1, 20) is included in the data of the 8 × 8 pixel block of the subscript (1, 19) or the subscript (1, 21). The DC component of the (1, 20) 8 × 8 pixel block is copied. And it complements by making all AC components into the value 0.

  Further, the DC component of the 8 × 8 pixel block of the subscript (2, 20) is included in the data of the 8 × 8 pixel block of the subscript (2, 19) or the subscript (2, 21). The DC component of the (2, 20) 8 × 8 pixel block is copied. Then, it complements by making all AC components into the value 0. Further, the DC component of the 8 × 8 pixel block of the subscript (3, 20) is included in the data of the 8 × 8 pixel block of the subscript (3, 19) or the subscript (3, 21). The DC component of the (3, 20) 8 × 8 pixel block is copied. And it complements by making all AC components into the value 0.

FIG. 6 shows a data array as a result of complementing the data deficiency shown in FIG.
In FIG. 6, 600 is a source of DC (0, 20), DC (1, 20), DC (2, 20), and DC (3, 20) to be complemented, and 601 to 604 are the missing complement processing block 120. It is a complemented data group.

  When the missing complement processing by the missing complement processing block 120 is completed and data is stored in the fifth buffer 119 in the manner shown in FIG. 6, for example, the IDCT processing block 121 sequentially reads the data and performs inverse DCT processing. Then, the result is written in the sixth buffer 122.

  The inverse DCT processing by the IDCT processing block 121 is performed without using the DC components of the 8 × 8 pixel blocks adjacent in the vertical direction that are redundant data added by the redundancy processing block 107 of the transmission apparatus 100. Then, the high frequency AC component originally placed in the portion where the DC components of the 8 × 8 pixel blocks adjacent in the vertical direction are placed is regarded as a value 0, and inverse DCT processing is performed. The restored image data stored in the sixth buffer 122 is transferred to the outside of the receiving apparatus by the image output interface 123.

  As described above, in the present embodiment, insertion of redundant data into transmission data and complementation of data in the event of loss occur without compromising real-time performance of the entire image data and pressing the network bandwidth unnecessarily. Can be done without.

(Second Embodiment)
A schematic configuration of an image transmission apparatus according to the second embodiment of the present invention will be described with reference to a block diagram shown in FIG.
In the image transmission apparatus of this embodiment, a transmission apparatus 700 and a reception apparatus 701 are connected via a network 702. The transmission apparatus 700 transmits the input image data that has been compression-encoded to the reception apparatus 701 via the network 702. Then, the receiving device 701 decompresses and decodes the transmitted compression-encoded image data and outputs it.

A configuration of transmitting apparatus 700 will be described.
Reference numeral 703 denotes an image input interface for inputting source image data to be transmitted. Reference numeral 704 denotes a first buffer, which stores source image data input by the image input interface 703. Reference numeral 705 denotes a DCT processing block that performs DCT conversion on the data stored in the first buffer 704.

  Reference numeral 706 denotes a second buffer, which stores the data DCT converted by the DCT processing block 705. Reference numeral 707 denotes a redundancy processing block which performs redundancy processing using data stored in the second buffer 706. Reference numeral 708 denotes a quantization processing block, which performs quantization processing on the data stored in the second buffer 706. Reference numeral 709 denotes an encoding processing block, which encodes the data quantized by the quantization processing block 708.

  Reference numeral 710 denotes a third buffer, which stores data encoded by the encoding processing block 709. Further, the redundancy processing block 707 stores the data subjected to redundancy processing. Reference numeral 711 denotes a packetizing processing block that packetizes the data stored in the third buffer 710. A transmission interface 712 transmits the packet generated by the packetizing processing block 711 to the network 702.

Next, the configuration of the receiving device 701 will be described.
A reception interface 713 receives a packet from the network 702. Reference numeral 714 denotes a depacketization processing block, which expands a packet received by the reception interface 713.

  Reference numeral 715 denotes a fourth buffer, which stores the data developed by the depacketization processing block 714. Reference numeral 716 denotes a decoding processing block for decoding data stored in the fourth buffer 715. Reference numeral 717 denotes an inverse quantization processing block, which dequantizes the data decoded by the decoding processing block 716. Reference numeral 718 denotes a loss detection / complementation processing block, which detects packet loss from data stored in the fourth buffer 715 and performs complement processing for the loss.

  Reference numeral 719 denotes a fifth buffer, which stores data inversely quantized by the inverse quantization processing block 717. Further, the data subjected to the defect complement processing by the defect detection / complement processing block 718 is stored. Reference numeral 720 denotes an IDCT processing block which performs inverse DCT conversion on the data stored in the fifth buffer 719. Reference numeral 721 denotes a sixth buffer, which stores data obtained by the inverse DCT conversion by the IDCT processing block 720. An image output interface 722 outputs data stored in the sixth buffer 721.

Hereinafter, a series of processes in the block diagram shown in FIG. 7 will be described. In this embodiment, an operation of transmitting an image of 640 pixels in the horizontal direction and 480 pixels in the vertical direction is illustrated.
The image input interface 703 stores the input image data in the first buffer 704. The DCT processing block 705 sequentially reads out the image data stored in the first buffer 704 in units of 8 × 8 pixels, performs DCT conversion, and sequentially stores the data resulting from the DCT conversion in the second buffer 706. To do.

  The data array of the 8 × 8 pixel block DCT conversion processing result stored in the second buffer 706 is the same as that shown in FIG. 2 of the first embodiment. When the DCT conversion processing by the DCT processing block 705 is completed and the data of the DCT conversion processing result is stored in the second buffer 706 in the manner shown in FIG. 2, the redundancy processing block 707 starts the redundancy processing and performs quantization. Processing block 708 begins the quantization process.

  The redundancy processing block 707 reads the DC component of each 8 × 8 pixel block from the second buffer 706 and copies it to a predetermined area of the third buffer 710. The quantization processing block 708 sequentially reads the data stored in the second buffer 706 and performs quantization processing. The data that has been subjected to the quantization processing is transferred to the encoding processing block 709, where the encoding processing is performed. Then, the data encoded by the encoding processing block 709 is stored in the third buffer 710.

FIG. 8 shows a state when the data encoded by the encoding processing block 709 is stored in the third buffer 710.
In FIG. 8, reference numeral 800 denotes an encoded data group stored by the encoding processing block 709, and reference numeral 801 denotes a DC component data group stored by the redundancy processing block 707.

  When the processing by the redundancy processing block 707 and the encoding processing by the encoding processing block 709 have been completed and the third buffer 710 is in the state shown in FIG. 8, the packetizing processing block 711 starts the packetizing processing.

  The packetizing processing block 711 sequentially reads encoded data including header information from the third buffer 710 and DC component data of each 8 × 8 pixel block stored by the redundancy processing block 709. Then, a communication packet is created by adding an identification of the image and a sequence number that makes it possible to identify the number of data in the image and other headers necessary for transmission on the network 102. .

  At this time, the encoded data is included in the payload data portion of the packet, and the DC component of the 8 × 8 pixel block is included in the extension region of the communication header portion. At this time, in the packet including the encoded data of the 8 × 8 pixel block of the subscript (i, j) in the payload data portion, DC (i, j−1) of the 8 × 8 pixel block and 8 × 8 × An extension header area includes DC (i, j + 1) of 8 pixel blocks. That is, a packet that transmits encoded data of a certain 8 × 8 pixel block includes DC components of 8 × 8 pixel blocks that are vertically adjacent in the vertical direction.

The case where RTP (: Realtime Transport Protocol) is used is illustrated.
Information shown in FIG. 9 is added as an RTP header. If the value of the X flag in the RTP header is 1, an extension header can be added in addition to the RTP fixed header shown in FIG. In the case of using RTP, this extension header part includes the DC components of each 8 × 8 pixel block.

The packet created by the packetizing processing block 711 is as shown in FIG.
In FIG. 10, 1000 is an IP header portion, and 1001 is a UDP header portion. Reference numeral 1002 denotes an RTP fixed header portion, and reference numeral 1003 denotes an RTP extension header portion. Reference numeral 1004 denotes a payload data portion. Reference numerals 1005 to 1007 denote encoded data groups included in the payload data portion 1004, and reference numerals 1008 to 1013 denote DC component data groups included in the RTP extension header portion 1003.

  When the packetizing processing block 711 creates communication packets, the packetizing processing block 711 sequentially passes the packets to the transmission interface 712. The transmission interface 712 transmits the passed packet to the network 702.

  A packet transmitted from the transmission device 700 is transmitted to the reception device 701 on the network 702. The reception interface 713 receives a packet from the network 702 and passes the received packet to the depacketization processing block 714. When the depacketizing processing block 714 detects that data transmission of a new image has started by interpreting the sequence number in the packet, the depacketizing processing block 714 secures an area of the fourth buffer 715 and initializes the secured area. .

  The depacketization processing block 714 checks whether or not a communication error has occurred in the passed packet. This check uses a technique such as CRC. For a packet in which no occurrence of a communication error is detected, the payload data portion 1004 and the RTP extension header portion 1003 in the packet are stored in the fourth buffer 715, respectively.

  Each data is stored in an address corresponding to a sequence number in the packet. The fourth buffer 715 includes the encoded data included in the payload data portion 1004 and the DC included in the RTP extension header portion 1003 in the order of the subscript (i, j) of the 8 × 8 pixel block. The ingredients are lined up. Note that a packet in which the occurrence of a communication error is detected is discarded at this point.

  The decoding processing block 716 sequentially reads the encoded data from the fourth buffer 715 and performs decoding. The decoded data is transferred to the inverse quantization processing block 717, and the inverse quantization processing block 717 performs an inverse quantization process. Then, the data subjected to the inverse quantization process is stored in the fifth buffer 719.

  For the data that remains initialized in the fourth buffer 715, the processing by the decoding processing block 716 and the processing by the inverse quantization processing block 717 are not performed. Each time a packet is received and encoded data is stored in the fourth buffer 715, the processing of the decryption processing block 716 is started, and then the processing of the inverse quantization processing block 717 is started.

  On the other hand, when a certain period of time has elapsed since the reception interface 713 started to receive a packet containing the image data, the defect detection complement processing block 718 starts its operation. The fourth buffer 715 stores 8 × 8 pixel block encoded data and a DC component in the order of subscripts (i, j). The data stored in the fourth buffer 715 corresponding to the data included in the packet that has been detected and discarded due to the communication error, and the data included in the packet that has been delayed due to communication delay remains uninitialized. Has been.

  The missing detection complement processing block 718 finds a portion that is left initialized in units of one packet, recognizes that a packet that includes the data is missing, and starts complement processing.

  Data supplementation of the 8 × 8 pixel block of the subscript (i, j) is performed by using a packet including the encoded data of the 8 × 8 pixel block of the subscript (i, j−1), or the subscript (i, j). j + 1) 8 × 8 pixel block of subscript (i, j) in the fourth buffer 715 included in the RTP extension header portion 1003 of the packet that contained the encoded data of the 8 × 8 pixel block Is copied to the place where the DC component of the 8 × 8 pixel block of the subscript (i, j) in the fifth buffer 719 should be. Then, the value 0 is written in the AC component.

FIG. 11 is a diagram exemplifying complementing processing in the present embodiment.
11A shows the state of the fourth buffer 715, FIG. 11B shows the state of the fifth buffer 719 before complement processing, and FIG. 11C shows the state after complement processing. The state of the fifth buffer 719 is shown.

  By looking at the state of the fourth buffer 715 shown in FIG. 11A, the missing detection complement processing block 718 has the subscripts (i, j) of (0, 22), (1, 22), (2, It is detected that the encoded data of the 8 × 8 pixel block of 22) and (3, 22) has not arrived.

  In FIG. 11A, reference numerals 1100 to 1103 are areas marked “invalid”, which are places that remain in the initialized state. When the decoding processing block 716 and the inverse quantization processing block 717 perform processing, the state of the fifth buffer 719 before the complementing processing by the defect detection supplementing processing block 718 is as shown in FIG.

  The four 8 × 8 pixel blocks whose subscripts (i, j) are (0,22), (1,22), (2,22), and (3,22) are opposite to the decoding processing block 716. Processing by the quantization processing block 717 is not performed. Therefore, invalid data are arranged in the areas 1104 to 1107. The missing detection complement processing block 718 is an 8 × 8 subscript (i, j) of (0, 22), (1, 22), (2, 22), and (3, 22) in the fourth buffer 715. The DC component 1108 of the pixel block is copied to the corresponding location in the fifth buffer 719. The AC component values are all 0.

  By doing so, the fifth buffer 719 is in a state as shown in FIG. Reference numerals 1109 to 1112 denote data groups complemented by the defect detection complement processing block 718.

  When the defect complementation processing by the defect detection complement processing block 718 is completed and data is stored in the fifth buffer 719, the IDCT processing block 720 sequentially reads the data and performs inverse DCT processing. Then, the result is written in the sixth buffer 721. The restored image data stored in the sixth buffer 721 is transferred to the outside of the receiving device 701 by the image output interface 722.

  As described above, in the present embodiment, insertion of redundant data into transmission data and complementation of data in the event of loss occur without compromising real-time performance of the entire image data and pressing the network bandwidth unnecessarily. Can be done without.

(Other embodiments according to the present invention)
Each means constituting the image transmission apparatus and each step of the image transmission method in the embodiment of the present invention described above can be realized by operating a program stored in a RAM or ROM of a computer. This program and a computer-readable recording medium recording the program are included in the present invention.

  Further, the present invention can be implemented as, for example, a system, apparatus, method, program, or recording medium. Specifically, the present invention may be applied to a system including a plurality of devices. The present invention may be applied to an apparatus composed of a single device.

  Note that the present invention supplies a software program that implements the functions of the above-described embodiments directly or remotely to a system or apparatus. This includes the case where the system or the computer of the apparatus is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, and the like.

  Examples of the recording medium for supplying the program include a floppy (registered trademark) disk, a hard disk, an optical disk, and a magneto-optical disk. Further, there are MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, there is a method of connecting to a homepage on the Internet using a browser of a client computer. The computer program itself of the present invention or a compressed file including an automatic installation function can be downloaded from the homepage by downloading it to a recording medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  As another method, the program of the present invention is encrypted, stored in a recording medium such as a CD-ROM, distributed to users, and encrypted from a homepage via the Internet to users who have cleared predetermined conditions. Download the key information to be solved. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. Furthermore, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can be realized by the processing.

  As another method, the program read from the recording medium is first written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Then, based on the instructions of the program, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.

It is a block diagram which shows the example of schematic structure of the image transmission apparatus in the 1st Embodiment of this invention. In the 1st Embodiment of this invention, it is a figure which shows the example of a data array in the 2nd buffer which stores a DCT conversion process result. In the 1st Embodiment of this invention, it is a figure which shows the base vector after DCT conversion. In the 1st Embodiment of this invention, it is a figure which shows the example of a data arrangement | sequence in the 2nd buffer after a redundancy process. In the 1st Embodiment of this invention, it is a figure which shows the example of a data array in the 5th buffer at the time of defect | deletion generation | occurrence | production. In the 1st Embodiment of this invention, it is a figure which shows the example of a data array in the 5th buffer after a complementation process. It is a block diagram which shows the example of schematic structure of the image transmission apparatus in the 2nd Embodiment of this invention. In the 2nd Embodiment of this invention, it is a figure which shows the example of a data arrangement | sequence in the 3rd buffer after an encoding process. In the 2nd Embodiment of this invention, it is a figure which shows a RTP header. In the 2nd Embodiment of this invention, it is a figure which shows the structural example of a communication packet. It is a figure which shows a complementation process in the 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Transmission apparatus 101 Reception apparatus 102 Network 103 Image input interface 104 1st buffer 105 DCT processing block 106 2nd buffer 107 Redundancy processing block 108 Quantization processing block 109 Encoding processing block 110 3rd buffer 111 Packetization processing block 112 transmission interface 113 reception interface 114 depacketization processing block 115 fourth buffer 116 missing detection processing block 117 decoding processing block 118 inverse quantization processing block 119 fifth buffer 120 missing complement processing block 121 IDCT processing block 122 sixth buffer 123 Image output interface

Claims (11)

An image transmission apparatus that compresses and encodes image data and transmits / receives the image data,
An image transmission device having a replacement means for replacing the high-frequency component of the DCT conversion processing result of the compression encoding process with redundant data;
An image transmission apparatus comprising: an image receiving apparatus having a complementing unit that performs a complementing process using the redundant data when data loss is detected. The image transmitting apparatus includes an input unit that inputs the image data, a compression encoding unit that compresses and encodes the image data, an adding unit that adds redundant information to the image data, and the image data to a network. Output means for outputting,
The image receiving device includes: an image input unit that inputs the image data from a network; a detection unit that detects a loss of the image data; a loss complementation unit that complements the loss of the image data; and a decoding / decompression of the image data The image transmission apparatus according to claim 1, further comprising: a decoding / decompressing unit that performs image output, and an image output unit that outputs the image data.   The high-frequency component of the DCT conversion processing result replaced with the redundant data by the replacement means is the DC component of the DCT conversion processing result of the pixel group adjacent to the upper vertical direction of the image and the DCT of the pixel group adjacent to the lower vertical direction of the image. The image transmission apparatus according to claim 1, wherein the image transmission apparatus includes two DC components of the conversion processing result. The compression encoding means temporarily stores the data of the DCT conversion processing result in the storage means,
4. The image transmission apparatus according to claim 2, wherein the adding unit adds redundant data by the storage unit. An image transmission apparatus including an image transmission apparatus that compresses and encodes image data and transmits the image data to a network, and an image reception apparatus that receives the compression-encoded image data.
Image transmission characterized in that redundant data is stored in an extension area when generating a communication packet in the image transmission device, and complementing processing is performed using the redundant data when data loss is detected in the image reception device apparatus. The image transmitting apparatus includes an input unit that inputs the image data, a compression encoding unit that compresses and encodes the image data, an adding unit that adds redundant information to the image data, and the image data to a network. Output means for outputting,
The image receiving device includes an image input unit that inputs the image data from a network, a detection unit that detects a loss of the image data, a complement unit that complements the loss of the image data, and a decoding and decompression of the image data. 6. The image transmission apparatus according to claim 5, further comprising a decoding / decompression unit and an image output unit that outputs the image data.   The redundant data stored in the extension area of the communication packet and the compression-encoded data in the payload of the communication packet are data related to different pixel blocks. Image transmission device.   The redundant data stored in the extension area of the communication packet is data related to a pixel block adjacent in the vertical direction among pixel blocks related to compression-coded data in the payload of the communication packet. The image transmission device according to claim 5. The compression encoding means temporarily stores the data of the DCT conversion processing result in the storage means,
9. The image transmission apparatus according to claim 6, wherein the adding unit generates the redundant data in the storage unit and passes the redundant data to a communication packet generating unit. In an image transmission method in which image data is compressed and encoded and transmitted / received,
In the image transmission device, a replacement step of replacing the high-frequency component of the DCT conversion processing result in the compression encoding process with redundant data;
An image transmission method comprising: a complementing step of performing a complementing process using the redundant data when data loss is detected in an image receiving device.   A program that causes a computer to execute each step of the method according to claim 10.

Images