JP5007759B2 - Encoding apparatus, encoding method, decoding apparatus, and decoding method - Google Patents

Description

  The present invention relates to an encoding device, an encoding method, a decoding device, and a decoding method. For example, image information in which a plurality of unit image information is continuous (hereinafter referred to as continuous image information) is used for satellite broadcasting, cable, and the like. It is suitable for transmission via a network medium such as a TV or the Internet, or when processing on a storage medium such as an optical disk, a magnetic disk, or a flash memory.

  In recent years, continuous images have been handled as digital information. At that time, orthogonal transform such as discrete cosine transform or motion compensation is used for the purpose of transmitting or storing information with high efficiency and using redundancy unique to the continuous image information. An apparatus conforming to an encoding method such as MPEG (Moving Picture Experts Group) that encodes (compresses) an image is becoming widespread in broadcasting stations and general households.

  In particular, the MPEG2 (ISO / IEC 13818-2) encoding method is defined as a general-purpose image encoding method, and covers both interlaced scanning images and progressive scanning images, and standard resolution images and high-definition images. It is now widely used as a wide range of professional and consumer applications. In this MPEG2 encoding method, for example, a standard resolution interlaced scan image having 720 × 480 pixels is 4 to 8 Mbps, and a high resolution interlaced scan image having 1920 × 1088 pixels is 18 to 22 Mbps. ] Can be realized with high encoding efficiency (compression rate) and good image quality.

  The MPEG2 encoding method has been used mainly for high-quality encoding suitable for broadcasting, but it has a lower code amount (bit rate) than the MPEG1 encoding method, that is, an encoding method with higher encoding efficiency. Did not correspond. In the future, the need for such an encoding method is expected to increase with the spread of mobile terminals, for example, and the MPEG4 encoding method has been standardized accordingly. As for the MPEG4 encoding method for images, the standard was approved as an international standard in December 1998 as ISO / IEC 14496-2.

  Furthermore, in recent years, MPEG4AVC or H.264 has been developed by a joint video team between a VCEG organization and an MPEG organization. An encoding method called H.264 (hereinafter referred to as a JVT encoding method) has been standardized. In this JVT encoding system, a larger amount of calculation is required for encoding or decoding than conventional encoding systems such as MPEG2 and MPEG4, but it is known that higher encoding efficiency is realized. ing.

  Here, FIG. 8 shows a schematic configuration of an encoding apparatus that executes an encoding process according to any of the various encoding methods described above. In FIG. 8, an encoding apparatus 100 includes an image rearrangement buffer 102, an adder 103, an orthogonal transform unit 104, a quantization unit 105, a lossless encoding unit 106, an accumulation buffer 107, and an inverse quantization unit. 108, an inverse orthogonal transform unit 109, a frame memory 110, a motion prediction / compensation unit 111, and a rate control unit 112.

  In this case, the encoding apparatus 100 accumulates the continuous image information in the image rearrangement buffer 102, and arranges the continuous image information for each unit image information (frame or field data) according to a GOP (Group of Pictures) structure. Change.

  Here, the image rearrangement buffer 102 supplies unit image information to be subjected to intra prediction encoding among the continuous image information to the orthogonal transform unit 104. The orthogonal transform unit 104 performs orthogonal transform processing such as discrete cosine transform or Karhunen-Labe transform on the unit image information, and supplies the obtained orthogonal transform coefficient to the quantization unit 105.

  The quantization unit 105 performs a quantization process according to the control of the rate control unit 112 on the orthogonal transform coefficient supplied from the orthogonal transform unit 104, and obtains quantization information (quantized orthogonal transform coefficient). ) To the lossless encoding unit 106 and the inverse quantization unit 108. The lossless encoding unit 106 performs lossless encoding processing such as variable length encoding or arithmetic encoding on the quantized information, and stores the obtained encoded information (encoded quantized information) in the accumulation buffer 107. accumulate.

  The inverse quantization unit 108 performs inverse quantization processing on the quantization information and supplies the obtained orthogonal transform coefficient to the inverse orthogonal transform unit 109. The inverse orthogonal transform unit 109 performs an inverse orthogonal transform process on the orthogonal transform coefficient, and accumulates the obtained unit image information as reference image information in the frame memory 110 as necessary.

  On the other hand, the image rearrangement buffer 102 supplies unit image information to be subjected to inter prediction encoding among the continuous image information to the motion prediction / compensation unit 111. The motion prediction / compensation unit 111 performs motion prediction / compensation processing using the unit image information and the reference image information extracted from the frame memory 110, and supplies the obtained predicted image information to the adder 103. The adder 103 supplies a difference between the predicted image information and the corresponding unit image information to the orthogonal transform unit 104 as difference information.

  This difference information is subjected to various processes in the same manner as in the case of intra coding. As a result, the difference information is accumulated as coding information in the accumulation buffer 107 and also as reference image information in the frame memory 110 as necessary. The

  In addition, the motion compensation / prediction unit 111 supplies the motion vector information obtained together with the predicted image information to the lossless encoding unit 106 by performing motion prediction / compensation processing. The lossless encoding unit 106 performs lossless encoding processing on the motion vector information, and generates the obtained encoded motion vector information as a header portion of the corresponding encoded information.

  In this way, the encoding apparatus 100 sequentially generates encoded information for each unit image information by executing the encoding process on the continuous image information, and then the encoded information is transmitted via the accumulation buffer 107. Are output sequentially.

Next, FIG. 9 shows a schematic configuration of a decoding apparatus that executes a decoding process according to an encoding method corresponding to the encoding apparatus 100. In Fig 9, the decoding apparatus 120 includes a storage buffer 121, a lossless decoding unit 122, an inverse quantization unit 123, an inverse orthogonal transform unit 124, an adder 125, a picture rearrangement buffer 126, a motion The prediction / compensation unit 127 and the frame memory 128 are included.

  In this case, the decoding device 120 temporarily stores encoded information that is sequentially input in the accumulation buffer 121 and then supplies the encoded information to the lossless decoding unit 122. When the lossless decoding unit 122 is encoded information that has been subjected to intra prediction encoding, the lossless decoding unit 122 performs a decoding process such as variable length decoding or arithmetic decoding on the encoded information, and obtains quantization information obtained. Is supplied to the inverse quantization unit 123.

  The inverse quantization unit 123 performs an inverse quantization process on the quantization information supplied from the lossless decoding unit 122 and supplies the obtained orthogonal transform coefficient to the inverse orthogonal transform unit 124. The inverse orthogonal transform unit 124 performs inverse orthogonal transform processing on the orthogonal transform coefficients to generate original image information before encoding processing (hereinafter referred to as restored image information), and rearranges the images. Store in the buffer 126.

  On the other hand, when the lossless decoding unit 122 is encoded information that has undergone inter-prediction encoding, the encoded information and the encoded motion vector information inserted in the header portion of the encoded information are respectively The decoding processing is performed, and the obtained quantization information is supplied to the inverse quantization unit 123 and the motion vector information is supplied to the motion prediction / compensation unit 127. The quantized information is subjected to various processes in the same manner as when decoding the intra-coded information, and as a result, is supplied to the adder 125 as difference information.

  Further, the motion prediction / compensation unit 127 generates predicted image information based on the motion vector information and the reference image information stored in the frame memory 128, and supplies this to the adder 125. The adder 125 combines the reference image information and the difference information, and stores the obtained restored image information in the image rearrangement buffer 126.

  In this way, the decoding apparatus 120 sequentially generates the restored image information by executing the decoding process on the sequentially input encoded information, and then stores the restored image information in the image rearrangement buffer 126. For example, continuous reproduction is performed by sequentially outputting to a display unit (not shown).

  By the way, in the MPEG2 encoding system, since it is defined that only I (Intra) picture and P (Predictive) picture are used as inter prediction encoding targets, the decoding order in the decoding process is uniquely determined. come.

  Therefore, in the decoding apparatus 120, when the restored image information generated by executing the decoding process on each piece of encoded information compliant with the MPEG2 encoding method is continuously reproduced, the output of the restored image information is output. Even if the time is not adjusted through the image rearrangement buffer 126, an image based on each restored image information can be appropriately displayed on the display unit.

  On the other hand, in the JVT encoding method, a B (Bidirectional) picture can be used in addition to the I and P pictures as an inter prediction encoding target. Many degrees of freedom of selection of pictures that can be selected are defined.

  However, in the JVT encoding method, the output timing of the restored image information is not clearly defined even though the decoding order in the decoding process is not uniquely determined in accordance with such a specification.

  Therefore, when the decoding apparatus 120 continuously reproduces the restored image information generated by executing the decoding process on each piece of encoded information compliant with the JVT encoding method, the resource of the image rearrangement buffer 126 is limited. As a result, a situation occurs in which the encoded information is still being decoded regardless of the output timing of the restored image information corresponding to the encoded information, and as a result, the continuity is interrupted. There was a problem.

  The present invention has been made in view of such a situation, and an object thereof is to provide an encoding device, an encoding method, a decoding device, and a decoding method that can be continuously reproduced by a decoding device.

In order to solve such a problem, the present invention is an encoding device, which is an encoding unit that encodes unit image information to generate encoded information, and a plurality of unit image information including unit image information. from the maximum value based on the maximum value of the encoding delay from running encoding process to the output of the encoded information, the encoded information from the executing encoding processing with respect to the unit image information by subtracting the encoding delay until the output, and calculating means for calculating a decode delay from when decoding the encoded information to the output of the decoded image information generated by the decoding process, calculated by the calculation means Transmission means for transmitting the decoding delay as decoding delay information together with the encoded information generated by the encoding means.

The present invention relates to a coding method, execution and encoding step of the unit image information by encoding processing to generate encoded information, each coding on multiple unit image information including the unit image information encoding from to from the maximum value based on the maximum value of the encoding delay before outputting the encoded information, perform the encoding process to the output of the encoded information to the unit image information decoded by subtracting the delay, a calculation step of calculating a decode delay from when decoding the encoded information to the output of the decoded image information generated by the decoding process, the decoding delay that is calculated by the calculation step A transmission step of transmitting together with the encoded information generated by the encoding means as the delay information.

The present invention also relates to a decoding apparatus, coded information from running the encoded information unit image information is processed coded, each coding on multiple unit image information including the unit image information from the maximum value based on the maximum value of the encoding delay until the output by subtracting the encoding delay from running encoding process to the output of the encoded information to the unit image information obtained, and receiving means for receiving and decoding delay information indicating the decode delay from when decoding the encoded information to the output of the decoded image information generated by the decoding process, the decoding delay information received by the receiving means And decoding means for decoding the encoded information and generating decoded image information.

The present invention relates to a decoding method, encoding information from running the encoded information unit image information is processed coded, each coding on multiple unit image information including the unit image information from the maximum value based on the maximum value of the encoding delay until the output by subtracting the encoding delay from running encoding process to the output of the encoded information to the unit image information obtained, a receiving step of receiving the decode delay information indicating the decode delay from when decoding the encoded information to the output of the decoded image information generated by the decoding process, the decoding delay information received by the receiving step And a decoding step of decoding the encoded information to generate decoded image information.

According to the present invention, it is anticipated that encoded information generated as a result of the encoding process is sequentially decoded on the decoding side based on unit image information that spends the most time on the decoding process on the decoding side. Thus, since the time period required from the start time of the decoding process to the output time is calculated, each unit image information can be continuously reproduced without causing an underflow almost certainly.

1 is a block diagram showing an image reproduction system according to a first embodiment. It is a functional block diagram which shows the processing content of a code | symbol control part. It is a chart used for description of calculation of output time. It is a flowchart which shows an output time notification process procedure. It is a block diagram which shows the image reproduction system by 2nd Embodiment. It is a table | surface used for description of control of re-output. It is a flowchart which shows the re-output control processing procedure. It is a block diagram which shows the structure of an encoding apparatus. It is a block diagram which shows the structure of a decoding apparatus.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

<Example 1>
(1) Configuration of Image Reproduction System 1 In FIG. 1, reference numeral 1 denotes an image reproduction system according to the first embodiment as a whole, and a predetermined transmission is performed through an encoding device 2 and a decoding device 3 compliant with the JVT encoding method. It is configured by connecting each other through a path.

  The encoding device 2 includes an encoding unit 10 having the same configuration as the encoding device 100 described above with reference to FIG. 8 and a code control unit 11 that controls the encoding unit 10 and is supplied from the outside. 8 or the continuous image information D1 read from a recording medium (not shown) such as an internal HDD (Hard Disk Drive) or the like via the encoding unit 10 controlled by the code control unit 11 in FIG. By executing the encoding process in the same manner as described above, encoded information D2 (D2a, D2b,... Or D2n) is sequentially generated for each unit image information (frame or field data) D1a to D1n. Thereafter, the encoded information D2 is sequentially output.

  On the other hand, the decoding device 3 includes a decoding unit 20 having the same configuration as the decoding device 120 described above with reference to FIG. 9 and a decoding control unit 21 that controls the decoding unit 20. By executing the decoding process on the encoded information D2 sequentially input via the decoding unit 20 as described above with reference to FIG. 9 via the decoding unit 20 controlled by the decoding control unit 21. Restored image information D3 (D3a, D3b,..., Or D3n) is sequentially generated, and thereafter, the restored image information D3 is continuously output to a display unit (not shown) and continuously reproduced.

(2) Configuration of Code Control Unit 11 The code control unit 11 of the encoding device 2 is input to the encoding unit 10 based on a program or table information compliant with the JVT encoding method stored in the inside in advance. The GOP structure for the continuous image information D1 and conditions relating to encoding processing such as the encoding order (hereinafter referred to as encoding conditions) are determined, and the encoding unit 10 is controlled according to the encoding conditions.

  In addition to this configuration, the code control unit 11 anticipates that the encoded information D2 is sequentially decoded on the decoding side (decoding device 3), and restores the decoding result of the encoded information D2. The output timing of the image information D3 is calculated, and an output timing notification process for notifying this output timing to the decoding device 3 before generating the restored image information D3 is executed.

  Here, when the processing contents of the output timing notification process in the code control unit 11 are functionally classified, for example, as shown in FIG. 2, the decoding process for the encoded information D2 is started and then generated by the process. A delay calculation unit 11a that calculates a period until the decoded image information D3 is output (hereinafter referred to as a decode delay), and a header addition unit 11b that adds the decode delay as a header of the corresponding encoded information D2. Can be classified. The processing of the delay calculation unit 11a and the header addition unit will be specifically described using an example shown in FIG.

In FIG. 3, the column “EI (Encoder Input)” represents the unit image information D1a to D1n input to the encoding unit 10 by the picture type assigned according to the GOP structure. It shows the order of picture types before digitization processing. The column of “EO (Encoder Output)” is encoded information D2a˜ that is generated as a result of executing an encoding process according to a predetermined encoding order for each unit image information D1a˜D1n input to the encoder 10. This represents the picture type of D2n, that is, the order of the picture types after the encoding process.

  The column “Ed (Encoder Delay)” is a period from the start of executing the encoding process to the unit image information D1a, D1b,..., Or D1n until the output of the encoded information D2 generated by the process. (Hereinafter referred to as “encode delay”), which are calculated based on encoding conditions. Based on this encoding delay ("Ed" field), the output timing of the encoded information D2a to D2n ("EO" field) is adjusted as necessary. Specifically, for example, for the first unit image information D1a (“I00” in the “EI” field) that is an I picture type, the encoded information D2a (in the “EO” field) from the time when it is input (t = 0). If the period until it is generated as “I00”) is less than the corresponding encoding delay (“6” [seconds] in the “Ed” column), the storage buffer 107 (FIG. 8) of the encoding unit 10 And is output at the same time as the encoding delay has elapsed (t = 6).

  The delay calculation unit 11a calculates a decoding delay (“Dd (Decoder Delay)”) for each piece of encoded information D2 based on the encoding delay, and the encoding delay (“Ed”). ) Of the encoded information D2a to D2n so that the decoding delay with respect to the encoded information D2 ("B13") having the maximum encoding delay (10 [seconds]) is minimized (0 [seconds]). Calculate the delay.

  Specifically, since the maximum value of the encode delay is 10 [seconds], the delay calculation unit 11a subtracts the value of the encode delay corresponding to each piece of encoded information D2a to D2n from this 10 [second]. Then, the decoding delay value (the value of each term in the column “Dd”) for the encoded information D2a to D2n is calculated.

  The delay calculation unit 11a generates each decode delay calculated in this way as decode delay information D10, and sends this to the header adding unit 11b.

  The header adding unit 11b replaces the corresponding decoding delay as a header each time the encoded information D2 is sequentially generated in the encoding unit 10 based on the decoding delay information D10 supplied from the delay calculating unit 11a. It is added to the encoded information D2.

  In this case, for example, the encoded information D2a (“I00” in the “EO” column) to which the decoding delay is added as a header is output at the same time as the encoding delay elapses, and is transmitted to the decoding device 3 via the transmission path. Entered. In FIG. 3, for convenience of explanation, the transmission time of the transmission path between the encoding device 2 and the decoding device 3 is not considered, and the input timing of the encoded information D2 in the decoding device 3 is the code It is regarded as equivalent to the output timing of the encoding information D2 from the encoding unit 10 (column of “EO”).

  The decoding device 3 adjusts the output timing of the restored image information D3, which is the decoding result of the encoded information D2, based on the decoding delay of the encoded information D2. That is, for example, the decoding delay of the first encoded information D2a (4 [seconds] in the “Dd” column) is determined by the decoding device 3 based on the header added to the encoded information D2a. It is recognized by the decoding control unit 21 before the decoding process. Thereafter, the encoded information D2a is generated as restored image information D3a in the decoding unit 20, and at this time, the elapsed period from the input time (t = 6) is less than the corresponding encoding delay (4 [seconds]). The display unit is adjusted as necessary by accumulation in the image rearrangement buffer 126 (FIG. 9), and is output at the same time as the decoding delay has elapsed (4 [seconds] (t = 10)). Is displayed.

  Further, the remaining encoded information D2b to D2n is generated as restored image information D3b to D3n in the decoding unit 20 in the same manner as the encoded information D2a, and the restored image information D3b to D3n is In the image rearrangement buffer 126 (FIG. 9), the picture order is rearranged in the same order as that before the encoding process ("DO (Decoder Output)" column), and the output timing is adjusted based on the corresponding encoding delay. It is output later, and as a result, it is continuously reproduced.

  In this way, the code control unit 11 executes the output time notification process, thereby causing the decoding device 3 to transmit a decoding delay assuming that the encoded information D2 is decoded by the decoding device 3 via the header. It is designed to be recognized in advance.

  Here, the output timing notification processing as described above is sequentially executed according to the output timing notification processing procedure RT1 shown in FIG.

  That is, for example, when a predetermined operation for executing an encoding process is performed via an input unit (not shown), the code control unit 11 starts this output timing notification processing procedure RT1 from step SP0 and continues. The maximum encoding delay is recognized from the encoding delays calculated based on the encoding condition in step SP1 (corresponding to 10 [seconds] in the “Ed” column in FIG. 3).

  In step SP2, the code control unit 11 loads, for example, the encoded information D2a existing in the lossless encoding unit 106 (FIG. 8) in the encoding unit 10 into the accumulation buffer 107 (FIG. 8). It is determined whether or not it has succeeded, and if a negative result is obtained, the process returns to step SP2 to capture the encoded information D2a again.

  On the other hand, if a positive result is obtained in step SP2, the code control unit 11 performs encoding delay (step S3) from the encoding delay recognized in step SP1 to the encoded information D2a fetched in step SP2. In FIG. 3, the decoding delay (corresponding to 4 [seconds] in the “Dd” column) for the encoded information D2a is calculated by subtracting 6 [seconds] in the “Ed” column. The calculated decoding delay is added to the encoded information D2a as a header.

  Then, the code control unit 11 determines whether or not all the encoded information D2 has been taken into the accumulation buffer 107 (FIG. 8) in step SP5. If a negative result is obtained, the code control unit 11 returns to step SP2 and returns to the above-described case. When the process is repeated and a positive result is obtained, the process proceeds to the subsequent step SP6, and this output timing notification processing procedure RT1 is terminated.

  In this manner, the code control unit 11 can execute the output timing notification process according to the output timing notification processing procedure RT1.

  In the above configuration, the encoding device 2 expects that the encoded information D2 obtained as a result of the encoding process according to the JVT encoding method will be decoded on the decoding device 3 side. The output time (decode delay) for the decoding result of D2 (each restored image information D3) is calculated, and the calculated output time is added as a header.

  Therefore, in this encoding device 2, it is possible to cause the decoding device 3 before the decoding process to recognize the decoding delay assuming that the encoded information D2 is decoded by the decoding device 3 via the header. Even if the encoded information D2 is encoded according to the JVT encoding method in which the decoding order is not uniquely determined, the continuity of output in the restored image information D3 can be ensured.

  In this case, in the encoding device 2, the encoding information D2 that is the maximum period of time (encoding delay) from the start of executing the encoding process to the output of the encoding information D2 (“ The output timing (decoding delay) for each restored image information D3 is calculated so that the decoding result of B13 ") is output immediately.

  Therefore, in this encoding apparatus 2, the decoding result of each encoded information D2 is based on the output timing (encode delay) on the decoding side with respect to the encoded information D2 that spends the most time on the decoding side. Since the output time (decode delay) for (restored image information D3) can be calculated, the underflow is almost certainly performed by assigning (offset) the output time during decoding in the decoding device 3. It can be adjusted without generating.

  According to the above configuration, the encoded information D2 is decoded in anticipation that the encoded information D2 obtained as a result of the encoding process according to the JVT encoding method is decoded on the decoding device 3 side. The output time (decode delay) for each restored image information D3 at the time is calculated, and the calculated output time is added as a header, so that the decoding order is not uniquely determined and the encoding is performed according to the JVT encoding method. Even in the encoded information D2, the continuity of output in the restored image information D3 can be ensured, and thus the decoding device 3 can continuously reproduce.

(3) Other Embodiments In the above-described first embodiment, the case where the JVT coding system is applied has been described. However, the present invention is not limited to this, and at least a B picture is predicted. Various other encoding schemes that are defined so as to be used as a conversion target can be applied.

  In the first embodiment described above, the decoding result of the encoded information in anticipation that a plurality of pieces of encoded information D2 generated as a result of executing the encoding process are sequentially decoded on the decoding side. As a timing calculation means for calculating the output timing for the encoding information, the decoding delay for the decoding result of the encoding information is calculated so that the decoding result of the encoding information D2 that becomes the maximum encoding delay among the encoding delays is immediately output. However, the present invention is not limited to this, and instead of the decoding delay, the period from the generation of the decoding result of the encoded information to the output timing is calculated, or the buffer on the decoding side The decoding delay for the decoding result of the encoded information is calculated based on the calculation result assuming the occupation rate of the information amount stored in It may be. If the output time is calculated based on the calculation result, it is possible to calculate the output time so that the decoding side surely performs continuous reproduction.

  Further, in the above-described first embodiment, the decoding delay information supplied from the delay calculation unit 11a is used as a time notification unit that notifies the decoding side of the output time before the decoding result of the corresponding encoded information is obtained. Although the case where the output timing (decode delay) is added to the corresponding encoded information D2 as a header based on D10 has been described, the present invention is not limited to this, and for example, the decode delay is not added as a header. The information D10 may be directly output to the decoding side before the encoding process.

<Example 2>
(1) Configuration of Image Reproduction System 51 In FIG. 5, in which parts corresponding to those in FIG. 1 are assigned the same reference numerals, 51 denotes an image reproduction system according to the second embodiment as a whole, which conforms to the JVT encoding method. The encoding device 52 and the decoding device 53 are connected to each other via a predetermined transmission path.

  The encoding device 52 includes an encoding unit 10 and a code control unit 61 that controls the encoding unit 10, and the code control unit 61 does not execute the output timing notification process described above. Others are the same as in the case of the first embodiment described above. In this case, the encoding device 52 performs an encoding process on the continuous image information D1 via the encoding unit 10 controlled by the code control unit 61, so that no decoding delay is added as a header. The encoded information D2 is sequentially generated, and thereafter the encoded information D2 is sequentially output.

  On the other hand, the decoding device 53 includes a decoding unit 20 and a decoding control unit 71 that controls the decoding unit 20, and for encoded information D2 sequentially input via a transmission path. The restored image information D3 is sequentially generated by executing a decoding process via the decoding unit 20 controlled by the decoding control unit 71, and then the restored image information D3 is displayed on a display unit (not shown). Are continuously output so that they are output sequentially.

(2) Configuration of Decoding Control Unit 71 In practice, the decoding control unit 71 of the decoding device 53 temporarily stores sequentially input encoded information D2 in the accumulation buffer 121 (FIG. 9) in the decoding unit 20. accumulated, based on the header or the like included in the encoded information D2, the decoding order and the encoded information D2, conditions relating to the decoding of the start phase or the like of the decoding process (hereinafter, the decoding conditions this And the decoding unit 20 is controlled according to the decoding conditions.

  In addition to this configuration, the decoding control unit 71 monitors the storage state of the restored image information D3 stored in the image rearrangement buffer 126 (FIG. 9) after the decoding process, and fails (underflow) as the storage state. Is detected, a re-output control process for re-outputting the restored image information D3 output immediately before the detection (failure) is executed. This re-output control process will be specifically described using the example shown in FIG.

  In FIG. 6, as in FIG. 3, the “EI” field indicates the order of picture types before the encoding process, the “EO” field indicates the order of picture types after the encoding process, and “DO” indicates after decoding. The “Ed” column indicates the encoding delay, and the “Dd” column indicates the decoding delay. In FIG. 6, as in FIG. 3, the transmission time of the transmission path between the encoding device 52 and the decoding device 53 is not considered, and the input timing of the encoded information D2 in the decoding device 53 is The output timing of the encoded information D2 from the encoding unit 10 (the column “EO”) is regarded as equivalent.

  The column “Sud (Start-up delay)” indicates a period (hereinafter referred to as a start-up delay) from when each piece of encoded information D2 is input to when the decoding process starts. The startup delay is calculated by the code control unit 61 in the encoding device 52 based on the encoding condition, and is added as a header.

When the decoding control unit 71 first inputs the first encoded information D2a (“I00” in the “EO” column), the start-up delay for the encoded information D2a (“6” [second] in the “Sud” column) The encoding information D2a is immediately sent to the lossless encoding unit 122 and the decoding process is started (“0” [seconds] in the “Dd” column). As a result, the decoding control unit 71 is adapted to be able to shorten the preparation time when continuous playback (start time).

  In this case, in the decoding control unit 71, a shift period (delay amount that must be adjusted) between the decoding process start timing to be originally started and generated by such control, specifically 6 [seconds] in FIG. In the following, this period is referred to as an adjustment delay amount), and the restored image information D3a, which is the result of the decoding process of the encoded information D2a, is transmitted via the image rearrangement buffer 126 for this adjustment delay amount. Is output again (corresponding to the term “t = 8” in the “DO” column), so that a part of the adjustment delay amount (“2” of “t = 8” in the “Dd” column) Equivalent to “second”).

  The decoding control unit 71 decodes the encoded information D2b, D2c,... (Sequentially input following the encoded information D2a (“P01”, “P02”... In the “EO” column) according to the startup delay. The stored state of the restored image information D3b, D3c... (“P01”, “P02”... In the “DO” column) stored in the image rearrangement buffer 126 (FIG. 9) after execution of the decoding process is monitored. To do.

  In this state, every time the restored image information D3 stored in the image rearrangement buffer 126 fails (underflows) (“t = 18” and “t = 30” in FIG. 6), the decoding control unit 71 performs the failure. By re-outputting the restored image information D3 (“P05” and “P10” in the “DO” column) output immediately before the time, the remaining adjustment delay amount is periodically offset.

  In this way, the decoding control unit 71 can ensure continuity of output in the restored image information D3 by executing the re-output control process.

  Here, the output timing notification process as described above is sequentially executed according to the re-output control process procedure RT2 shown in FIG.

  That is, for example, when a predetermined operation for executing a decoding process is performed via an input unit (not shown), the decoding control unit 71 starts this re-output control processing procedure RT2 from step SP10 and continues. In step SP11, the input of the first encoded information D2a is awaited. When the encoded information D2a is input, the process proceeds to step SP12.

  In step SP12, the decoding control unit 71 immediately starts the decoding process without following the startup delay for the encoded information D2a, and in the subsequent step SP13, outputs the restored image information D3a that is the decoding process result. In step SP14, the restored image information D3a is re-output to cancel a part of the adjustment delay amount.

  Subsequently, in step SP15, the decoding control unit 71 starts decoding of the next restored image information D2 (D2b, D2c,... D2n) and restores the restored image information D3 (D3b, D3c,. D3n) is output, and in the subsequent step SP16, it is determined whether or not the restored image information D3 stored in the image rearrangement buffer 126 (FIG. 9) has failed. If a negative result is obtained, the process returns to step SP15. The above process is repeated.

  On the other hand, when a positive result is obtained, the decoding control unit 71 re-outputs the restored image information D3 (“P05” and “P10” in the “DO” column) output in step SP15 in subsequent step SP17. Accordingly, a part (or all) of the remaining adjustment delay amount is canceled, and the process proceeds to the subsequent step SP18.

  Then, the decoding control unit 71 determines whether or not all the adjustment delay amounts have been canceled in step SP18. If a negative result is obtained, the decoding control unit 71 returns to step SP15 and repeats the above-described processing. If an affirmative result is obtained, the process proceeds to the subsequent step SP19, and the re-output control processing procedure RT2 is terminated.

  In this way, the code control unit 11 can execute the re-output control process according to the re-output control process procedure RT2.

  In the above configuration, the decoding device 53 uses the image rearrangement buffer 126 (the reordered image information D3 sequentially generated as a result of executing the decoding process on the encoded information D2 encoded according to the JVT encoding method. When the restored image information D3 stored temporarily in FIG. 9) and accumulated in the image rearrangement buffer 126 (FIG. 9) fails, the restored image information D3 output immediately before the failure is re-output. I did it.

  Therefore, the decoding apparatus 53 can ensure the continuity of the output in the restored image information D3 even if the encoded information D2 is encoded according to the JVT encoding method in which the decoding order is not uniquely determined. it can.

  In this case, the decoding device 53 does not follow the decoding start time (startup delay) defined in the first encoded information D2a stored in the storage buffer 121 (FIG. 9), and immediately decodes the encoded information D2a. When the restored image information D3 stored in the image rearrangement buffer 126 (FIG. 9) fails, the time difference (delay amount for adjustment) generated from the decoding start time caused by this The reconstructed image information output immediately before is canceled by re-outputting.

  Accordingly, the decoding device 53 can shorten the preparation period (start-up time) for continuous reproduction, and can reduce the shift period (adjustment delay amount) caused by the shortening by periodically dispersing it. However, the continuity of output in the restored image information can be ensured.

  According to the above configuration, the restored image information D3 sequentially generated as a result of performing the decoding process on the encoded information D2 encoded according to the JVT encoding method is temporarily stored in the image rearrangement buffer 126 (FIG. 9). When the restored image information D3 accumulated and stored in the image rearrangement buffer 126 (FIG. 9) fails, the restored image information D3 output immediately before the failure is re-outputted. Even if the encoded information D2 is encoded according to the JVT encoding method whose decoding order is not uniquely determined, the continuity of the output in the restored image information D3 can be ensured, and thus the continuous reproduction is possible. Can do.

(3) Other Embodiments In the second embodiment described above, the case where the JVT coding system is applied has been described. However, the present invention is not limited to this, and at least a B picture is predicted. Various other encoding schemes that are defined so as to be used as a conversion target can be applied.

In the second embodiment, the storage buffer 121 is used as storage means for temporarily storing each piece of encoded information and restored image information sequentially generated as a result of the decoding process on the encoded information. Although the case where the rearrangement buffer 126 is used separately has been described, the present invention is not limited to this, and the storage buffer 121 and the image rearrangement buffer 126 may be shared. In this case, the number of buffers and the accumulation mode can be changed as appropriate.

  In the second embodiment described above, the decoding control unit 71 serving as output control means performs a decoding start time (startup) determined in the first encoded information D2a stored in the storage buffer 121 (FIG. 9). The encoding information D2a is immediately started to be decoded without following the delay), and the deviation period (adjustment delay amount) of the decoding start time caused thereby is accumulated in the image rearrangement buffer 126 (FIG. 9). The case where the restored image information D3 to be canceled is canceled by re-outputting the restored image information output immediately before the failure has been described. In addition, the storage buffer 121 The storage order of the encoded information D2a to D2b stored in (i.e., corresponding to the "EO" field) corresponds to the order before the encoding (i.e., the "EI" field). And in the case were different (e.g. corresponding to the section in Fig. 6, "t = 32"), may be re-output the restored image information D3 corresponding to the encoded information D2 that has its different. In this way, the decoding delay that increases most when the order is different can be complemented by re-output, and the continuity of output in the restored image information D3 can be further ensured.

  The present invention can be used when continuous image information is transmitted via a network medium such as satellite broadcasting, cable TV, or the Internet, or when it is processed on a storage medium such as an optical disk, a magnetic disk, or a flash memory. .

  DESCRIPTION OF SYMBOLS 1, 51 ... Image reproduction system 2, 52 ... Encoding apparatus 3, 53 ... Decoding apparatus, 10 ... Encoding part 11, 61 ... Code control part, 11a ... Delay calculation part, 11b... Header addition unit, 20... Decryption unit, 21 and 71.

Claims (6)

Encoding means for encoding unit image information to generate encoded information;
From the maximum value based on the maximum value of the encoding delay from running the encoding process, respectively to the output of the coded information to the multiple unit image information including the unit image information, the unit by subtracting the encoding delay from running the encoding process to the output of the encoded information to the image information, decoding generated by the decryption processing after decoding the encoded information Calculating means for calculating a decoding delay until image information is output;
As decode delay information the decode delay calculated by the calculating means, the encoding apparatus and a transmitting means for transmitting with the encoded information generated by said encoding means. The transmission means is
The encoding apparatus according to claim 1, wherein the decoding delay information is added to the encoding information as a header. An encoding step of encoding unit image information to generate encoded information;
From the maximum value based on the maximum value of the encoding delay from running the encoding process, respectively to the output of the coded information to the multiple unit image information including the unit image information, the unit by subtracting the encoding delay from running the encoding process to the output of the encoded information to the image information, decoding generated by the decryption processing after decoding the encoded information A calculation step for calculating a decoding delay until image information is output;
The decode delay calculated in the calculation step as the decode delay information, a coding method and a transmission step of transmitting with the encoded information generated by said encoding means. Among the unit image information encoding processing encoded information, run each of the above encoding process for multiple unit image information including the unit image information encoding delay before outputting the encoded information the maximum value from the maximum value on the basis, obtained by subtracting the encoding delay from running the encoding process to the output of the coded information to the unit image information, the code a receiving means for receiving and decoding delay information indicating the decode delay from when decoding the reduction information to the output of the decoded image information generated by the decoding process,
By using the decode delay information received by said receiving means, said encoding information and the decoding process the decoding apparatus and a decoding means for generating the decoded image information. The decoding means includes
Based on the decoding delay shown in the decode delay information, the decoding apparatus according to claim 4 for adjusting the output timing of the decoded image information. Among the unit image information encoding processing encoded information, run each of the above encoding process for multiple unit image information including the unit image information encoding delay before outputting the encoded information the maximum value from the maximum value on the basis, obtained by subtracting the encoding delay from running the encoding process to the output of the coded information to the unit image information, the code a receiving step of receiving the decode delay information indicating the decode delay from when decoding the reduction information to the output of the decoded image information generated by the decoding process,
By using the decode delay information received by the receiving step, the decoding method and a decoding step of the coded information by the decoding process generates the decoded image information.

Images