JP2006319556A - Image decoding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image decoding device capable of extracting and decoding the code data of the code amount appropriate to browsing by taking both of the image quality for a reproduced image and a processing time required for decoding into consideration. <P>SOLUTION: The image decoding device comprises a code stream storage means 210 for accumulating code streams, a code stream extraction means 201 for extracting a predetermined code data from the code streams, a code stream decoding means 220 for decoding code streams extracted by the code stream extraction means, a code amount detection means 205 for detecting the code amount giving a specified layer or the image quality, a decoding speed measurement means 208 for measuring a time required for decoding and a decoding speed, a decoding speed storage means 207 for storing the decoding speed measured by the decoding speed measurement means, and an extracted packet determining means 206 for determining a code data to be extracted from code streams based on the code amount detected by the code amount detection means and the decoding speed stored at the decoding speed storage means. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image decoding apparatus for hierarchical code data of still images.

  Currently, the still image coding algorithm JPEG is widespread mainly on the Internet. On the other hand, JPEG2000, which has been standardized against the background of the demand for further performance improvement and function addition as the next generation coding method, is gradually being used. It is coming. In the following, an outline of the basic scheme of the JPEG2000 encoding algorithm will be described according to the standard (see Non-Patent Document 1).

  First, an input image signal is subjected to two-dimensional wavelet transform by a wavelet transform unit, and is divided into a plurality of subbands. Here, the two-dimensional wavelet transform is realized as a combination of the one-dimensional wavelet transform. That is, a process of sequentially performing vertical one-dimensional wavelet transform for each column and a process of sequentially performing horizontal one-dimensional wavelet transform for each line. The one-dimensional wavelet transform is composed of a low-pass filter, a high-pass filter, and a down sampler having predetermined characteristics.

  The two-dimensional wavelet transform coefficient generated in this way is expressed by expressing the low-frequency component as L, the high-frequency component as H, the horizontal conversion by the first character, and the vertical sub-scanning conversion by the second character. Expressed as LL, HL, LH, HH. These band-divided components are called subbands. Here, the horizontal and vertical low-frequency components (LL components) are recursively subjected to wavelet transform.

  Next, the wavelet transform coefficient of each subband is quantized by the quantization step size set for each subband.

  Next, after the wavelet transform coefficients after quantization of each subband are divided into fixed-size areas called code blocks, the code block consisting of multi-value data is converted into a binary bit plane representation, and each bit plane is converted to Dividing into three types of encoding passes (Significant Propagation Decoding Pass, Magnitude Refinement Pass, and Cleanup Pass).

  The binary signal output from the three coding passes is subjected to context modeling for each coding pass and subjected to entropy coding.

  In parallel with the entropy encoding process, the code amount and distortion information for each encoding pass are calculated in each code block.

Finally, rate control that adjusts the code amount to a target code size or less while minimizing image quality degradation (encoding distortion) using the Lagrange multiplier method is generally applied. In this method, the truncation point in each code block i (which truncates the lower encoding pass) is ni, the code amount up to each truncation point is R (i, ni), and the encoding distortion is D (i, ni). When using Lagrange's multiplier method, the following formula Σ (R (i, ni) + λD (i, ni))
The variable λ is adjusted until the total code amount R in the entire screen generated by the cut-off point that minimizes is satisfied within the range of the target code amount Rmax. Here, the coding distortion is a mean square error of a reproduced image generated when decoding up to a certain coding pass.

  When the variable λ is obtained, the code data up to the truncation point corresponding to the variable λ is collected from all the code blocks, and the number of coding passes in each code block is added as additional information to form the final code data. . In this way, code data that minimizes coding distortion can be generated under the target code amount Rmax.

  It is also possible to configure the code stream by dividing the coding pass into a plurality of layers. For this purpose, the target code amount Dmax is set a plurality of times from a small value to a large value, and a plurality of truncation points are set by the above procedure. Next, the code data between the cut-off points is divided into layers, and the code data of the same resolution level, the same component, and the same layer are collected as a packet. A packet header storing the number of bytes is added to each packet. By arranging higher-order bit-plane packets earlier in the code stream than lower-bit-plane packets, it is possible to realize a configuration in which more important packets are placed earlier in the code stream. At the time of decoding, if a packet satisfying a desired number of bytes is extracted from the head of the code stream, a code stream with a minimum coding distortion can be obtained under a desired code amount.

  The JPEG2000 international standard as described above can be obtained through a standardization organization such as ISO or ITU-T. The latest information on JPEG2000 can be obtained by referring to http://www.jpeg.org.

ISO / ITU 15444-1: 2000

  In the code data extraction method described above, it is possible to extract and decode code data of a desired code amount, but it is not known how much time it takes to process, and how much image quality the reproduced image to be displayed is. There was a problem that I did not know what.

  The present invention has been made to solve the above-described problems, and a decoding-side user can display by determining an extraction code amount in consideration of a desired image quality or processing time designated on the decoding side. An object of the present invention is to provide a comfortable browsing of images by preventing waiting for an unnecessarily long time or an extremely poor reproduction image quality.

  An image decoding apparatus according to the present invention is an image decoding apparatus that decodes a code stream configured by being divided into a plurality of layers, the code stream storing means for storing the code stream, and stored in the code stream storing means. A code stream extraction unit for extracting predetermined code data from the code stream; a code stream decoding unit for decoding the code stream extracted by the code stream extraction unit; and a code amount that provides a specified layer or image quality. Code amount detection means, decoding speed measurement means for measuring the time and decoding speed required for decoding, decoding speed storage means for storing the decoding speed measured by the decoding speed measurement means, and detection by the code amount detection means Code amount and the decoding speed stored in the decoding speed storage means. Those with an extraction packet determining means for determining the encoded data to be extracted from the code stream Te.

  An image decoding apparatus according to another invention is an image decoding apparatus for decoding a code stream configured by being divided into a plurality of layers, the code stream storing means for storing the code stream, and the code stream storing means Code stream extraction means for extracting predetermined code data from the code stream stored in the code stream, a code amount for giving a specified layer or a specified image quality and a code stream / code quantity information transmission means for transmitting the code stream, and designation Code amount detection means for detecting a code amount that gives a specified layer or image quality, a code amount that gives a specified layer or a specified image quality, and a code stream / code amount information receiving means for receiving a code stream; Code stream decoding means for decoding the code stream and necessary for decoding A decoding speed measuring means for measuring the transmission time; a decoding speed storing means for storing the decoding speed; a transmission speed measuring means for measuring the transmission speed; a transmission speed storing means for storing the transmission speed; Extraction packet determination means for determining code data is provided.

  According to the present invention, it is possible to extract and decode code data having a code amount appropriate for browsing by taking into consideration both the image quality of a reproduced image and the processing time required for decoding.

Embodiment 1 FIG.
In the image decoding apparatus according to the first embodiment, an example in which code data having a code amount appropriate for a user to view a still image is extracted from a code stream stored in a storage device inside the decoding apparatus and decoded is shown. . The encoding / decoding method will be described using JPEG2000. It is assumed that encoding is not performed in real time, but is executed in advance and a code stream is accumulated.

First, in order to describe the configuration of the code stream, an encoding procedure for generating a code stream will be described using the block diagram showing the configuration of the image encoding device shown in FIG.
The image encoding apparatus shown in FIG. 1 includes a wavelet transform unit 101 that recursively performs two-dimensional wavelet transform on an input image signal, and a wavelet transform coefficient generated by the wavelet transform unit 101 with a preset quantization step size. Quantization means 102 for performing quantization processing, entropy coding means 103 for decomposing quantized wavelet transform coefficients into bit planes and binary arithmetic coding, and converting entropy-coded code data into a predetermined format And a rate distortion that calculates a coding distortion D that occurs when decoding up to that coding pass every time a coding pass in each code block is finished. The characteristic calculation means 105 and a code for storing the code stream And a de stream storage means 110.

  An image signal input from an image input device (for example, an image scanner, a digital camera, a network, a storage medium, or the like) not shown in the figure is converted into a one-dimensional wavelet transform by the wavelet transform unit 101 in the vertical direction and the horizontal direction. It is applied two-dimensionally in both directions and is divided into subbands. Here, the one-dimensional wavelet transform is constituted by a filter bank of a low-pass filter and a high-pass filter.

  FIG. 2 is an example in which a two-dimensional wavelet transform is recursively performed twice. In the figure, the first number represents the decomposition level, and the following two alphabetic characters L or H represent the types of filters in the horizontal and vertical directions. L represents a low-pass filter, and H represents the result of applying a high-pass filter. In addition, “recursively” performing wavelet transform twice means that when 1LL, 1HL, 1LH, and 1HH are generated by the first wavelet transform, the second wavelet transform is performed on the 1LL. To generate 2LL, 2HL, 2LH, and 2HH.

  The quantization processing means 102 quantizes the wavelet transform coefficient with the quantization step size set for each subband.

  The entropy encoding unit 103 divides the wavelet transform coefficients of each subband into fixed-size rectangular areas called code blocks, and then converts each code block made up of multi-value data into a binary bit plane. Usually, the size of this code block is set to a size of 64 × 64, 32 × 32 or the like.

  Here, the decomposition of the bit plane will be described in detail with reference to FIG. In FIG. 3, (a) represents an example of a 4 × 4 code block. These data are converted into a 1-bit signal representing positive and negative values and an absolute value representation, and the result of binary representation of the data in the vertical direction is arranged in units of rows (b). Next, (c) is a collection of bits having the same bit number with respect to (b). Here, when the least significant bit (LSB) is the 0th bit and the most significant bit (MSB) is the 3rd bit, the 0th bit plane is a collection of the 0th bit. A collection of 1 bits is a first bit plane, a collection of 2 bits is a second bit plane, and a collection of 3 bits is a third bit plane. In addition to this, a sign bit plane is created as a collection of bits representing positive and negative.

  In the entropy encoding means 103, each bit in the bit plane is divided into three types of encoding pass, significant propagation decoding pass (Significant Propagation Decoding Pass), and magnitude refinement pass (Magnitude Refinement Pass) according to the context. And divide it into Cleanup Pass.

  Next, context modeling for arithmetic coding is performed for each coding pass. However, the bit planes that are all 0s from the MSB plane are not subjected to context modeling and encoding, and only the number of all 0 bit planes is written in the header. For the bit plane in which 1 appears first, all bits are classified as a cleanup pass, while the other bit planes are classified into three types of encoding passes as described above.

  FIG. 4 shows an example in which the number of bit planes in the code block is 6 and the number of effective bit planes is 4. When the contest modeling is completed, entropy coding using arithmetic codes is performed. Entropy coding is performed on all bit planes, and the generated code data is temporarily stored in the formatting means 104.

  When entropy encoding of all code blocks is completed, the encoding pass of each code block is divided into units called layers. For example, the higher the contribution level to the SNR improvement of the reproduced image, the higher the upper layer, the coding pass is divided into several layers, and the code data of the same layer of the code block of the same resolution, the same component, and the close spatial position Are combined to form a unit called a packet.

  FIG. 5 shows a conceptual diagram of the code stream. Here, for the sake of simplicity, in the case where the number of components is one and the packet is not divided by the spatial position, that is, the packets are classified based on only the layer and the resolution level. An example of arrangement will be shown. Wavelet transform coefficients 2LL are resolution levels 0, 2HL, 2LH, 2HH are resolution levels 1, 1HL, 1LH, and 1HH are resolution levels 2. In this example, since each resolution level is divided into five layers, a total of 15 packets Kr, s (resolution level r = 0-2, layer s = 0-4) are defined.

  Next, a packet header storing attributes such as the number of bytes of the packet is added to each packet, and the packets are arranged in a predetermined order. A code stream called an EOC marker indicating the end of the code stream is added to the main header storing information such as the image size and the number of wavelet transform levels at the head of the code stream, thereby forming one code stream.

  Next, a layer determination method will be described. The rate distortion characteristic calculation means 105 calculates the coding distortion D that occurs when decoding up to the coding pass every time the coding pass in each code block is finished. In this embodiment, encoding distortion is defined as a mean square error of a reproduced image when decoding up to a certain encoding pass. Further, every time encoding of a certain coding pass is completed in each code block, the accumulated code amount R up to that coding pass is counted.

When the encoding of all the code blocks is completed and the relationship between the encoding distortion and the code amount can be calculated in all the code blocks, all the code blocks are passed through the encoding path having the minimum code amount below the target total distortion amount Dmax. The process of finding for is started. This optimization problem can be realized by using Lagrange's undetermined multiplier method. Specifically, when the code amount up to the encoding pass ni in a certain code block i is R (i, ni) and the encoding distortion is D (i, ni),
Σ (D (i, ni) + λR (i, ni))
Is found, and λ is obtained so that the total distortion amount Dsum at that time becomes the target distortion amount Dmax. As shown in FIG. 6, in the graph of the code amount R and the encoding distortion D, a point where the slope −λ is a tangent is found, and the sum Dsum of the encoding distortion at that point becomes Dmax. Is the same as adjusting

Next, a specific example of the λ adjustment method will be described with reference to FIG. First, λmax giving the lower limit value of the slope −λ and λmin giving the upper limit value are defined (step 701 in FIG. 7), and λ that satisfies λ = (λmin + λmax) / 2 is calculated (step 702 in FIG. 7). An encoding pass ni that minimizes D (i, ni) + λR (i, ni) in the code block i is obtained, and its code amount is set to D (i, ni). ΣD (i, ni) = Dsum is calculated for all code blocks (steps 703 to 707 in FIG. 7),
If Dsum ≧ Dmax, λmax = λ
If Dsum <Dmax, λmin = λ
(Steps 708 to 710 in FIG. 7).

  When new λmin and λmax are obtained, λ = (λmin + λmax) / 2 is calculated in the same manner as before, and the same is repeated thereafter. The above processing is repeated until λmax−λmin ≦ ε (ε is a positive constant) (steps 702 to 703 in FIG. 7). In this way, the encoding pass ni for the finally obtained λ becomes the encoding pass to be obtained.

  The formatting means 104 can create a layer having a minimum total code amount below a desired encoding distortion Dmax by collecting code data of an encoding pass higher than the encoding pass ni in each code block.

  Here, in the first layer, the above-described processing is executed with Dmax being a large value, and gradually decreasing values are calculated to calculate a coding pass that becomes a boundary between the layers. For example, as shown in Table 1, when Dmax is set, the layer structure of PSNR (= 10 × log (255 × 255 / Dmax)) widely used as an objective image quality evaluation scale is It is formed to be higher.

  Next, the image decoding apparatus according to the first embodiment will be described with reference to FIG. In this image decoding device, code data having an appropriate code amount is partially extracted from the code stream stored in the image decoding device in consideration of both the image quality of the reproduced image and the processing time required for decoding, and decoded. . It is assumed that the coding distortion achieved in each layer is known in advance on the decoding side.

  As shown in FIG. 8, the image decoding apparatus according to Embodiment 1 decodes a code stream extraction unit 201 that partially extracts a code stream necessary for display from the code stream storage unit 210 and the code stream. A code stream decoding unit 220 for generating a reproduced image, a code amount detecting unit 205 for detecting the number of bytes of a desired packet of the code stream, an extracted packet determining unit 206 for determining a packet to be decoded, and a decoding speed. Decoding speed storage means 207 for storing, and decoding speed measuring means 208 for calculating the decoding speed from the processing time required for entropy decoding, inverse quantization, and inverse wavelet transform and the amount of processed code are provided.

  The code stream decoding unit 220 includes an entropy decoding unit 202 that performs binary arithmetic decoding on the code stream to reproduce the wavelet transform coefficient quantized value, and dequantizes the wavelet transform coefficient quantized value with a preset quantization step size. It comprises an inverse quantization means 203 for processing, and a wavelet inverse transform means 204 that performs inverse wavelet transform of wavelet transform coefficients to generate reproduced image data.

  Hereinafter, the operation will be described with reference to the flowchart of FIG. 9 focusing on the code stream extraction method. There are two types of codestream extraction methods, an image quality priority mode and a processing time priority mode, which are designated in advance by the user. In the first embodiment, the time priority mode will be described.

  First, the standard processing time T is set by the code stream extraction means 201 (step 902 in FIG. 9). For example, it is a standard time required from the time the user designates an image to the display, such as 3 seconds. Next, the standard decoding layer number N, which is a parameter for setting the image quality, is set (step 903 in FIG. 9). If N = 2, the standard number of layers to be decoded is two layers. In the code stream of Table 1, the layers determined by Dmax = 65 (PSNR = 30) are decoded.

Next, the extracted packet determination unit 206 reads the predicted decoding rate p [bytes / sec] from the decoding rate storage unit 207 (step 904 in FIG. 9). This value is updated each time an image is decoded, but an appropriate value is set in advance as an initial value for the first image. Next, the code amount detection unit 205 detects the code amount b [bytes] of the N layer of the code stream stored in the code stream storage unit 210 (step 905 in FIG. 9). The code amount of each packet is obtained by reading the value written in the packet header of the code stream. For example, the code amount of layer 0 in the code stream of FIG. 5 is obtained by reading the code amount of each packet written in the packet header of the packets K _0,0 , K _0,1 , K _0,2 .

Next, the extracted packet determining means 206 calculates a predicted processing time t = b / p [sec], which is a predicted value of the time required for processing (step 906 in FIG. 9). Here, the predicted processing time t is compared with the standard processing time T (step 907 in FIG. 9). If the predicted processing time t is shorter than the standard processing time T, N layers are extracted from the code stream and decoded (steps 909 and 910 in FIG. 9). Conversely, if the predicted processing time t is longer than the standard processing time T, only the code amount Tp [byte] that can be decoded by the standard processing time is extracted and decoded (steps 908 and 910 in FIG. 9). In Embodiment 1, when a code is extracted, the code data boundary to be extracted is set as a packet boundary. That is, when code data of a certain code amount is extracted, the code amount that actually exceeds the code amount is the most. It is assumed that the data is extracted up to the nearest packet boundary. For example, in the code stream of FIG. 5, when the number of bytes Tp [byte] to be extracted from the beginning to the middle of the packet K _0,2 is the extracted packet, K0,0, K _0,1 , K _0,2 It becomes.

  The decoding speed measuring means 208 receives a decoding start signal output by the entropy decoding means at the start time of the process and a decoding end signal output by the wavelet inverse transform means at the end time of the process, and decodes from the difference between the reception times. Is calculated (step 911 in FIG. 9). Then, the decoding speed p [byte / sec] for the code stream is calculated from the code amount of the extracted code stream and the processing time Δt (step 912 in FIG. 9) and written in the decoding speed storage means 207. At the time of decoding the next image, the decoding speed for the current code stream is applied as the predicted value.

  The above is the processing procedure for one code stream, and if an image to be decoded next is designated, the process is executed for the next image after reading the predicted decoding speed (step 904 in FIG. 9).

  In the first embodiment, the layer division has been described such that the coding distortion has a predetermined value. However, the layering is not necessarily the coding distortion. For example, the subjective evaluation scale (MOS) is a predetermined value. It is possible to divide up to an encoding pass as a layer. In addition, although the method of specifying the number of layers as a measure of the image quality to be extracted has been shown, the number of layers is not necessarily limited, and for example, a mean square error, PSNR, or the like may be used.

  If code data is simply decoded by the number of standard decoding layers, the decoding time may be significantly increased in some cases. In the first embodiment, in such a case, an effect that the decoding time is shortened can be obtained by limiting the code amount to a range that can be decoded in the standard processing time.

  On the other hand, if only the amount of code that requires the standard processing time is decoded, in some cases, even code data that can obtain an image quality higher than necessary may be decoded. In the first embodiment, in such a case, the decoding time is shortened by decoding only the code data of the standard decoding layer number.

  As described above, in the first embodiment, it is possible to extract and decode code data having a code amount appropriate for browsing by considering both the image quality of a reproduced image and the processing time required for decoding. .

Embodiment 2. FIG.
In the first embodiment, the time priority mode, which is a mode in which code data is extracted with the highest priority that the decoding processing time is within the standard processing time, has been described. In the second embodiment, code distortion is desired. An example of an image quality priority mode in which code data is extracted with the highest priority being set to be equal to or less than the value (achieving a desired image quality) will be described. Since the code stream generation method, the code stream configuration, and the block diagram of the decoding apparatus are the same as those in the first embodiment, the description thereof will be omitted, and the code stream extraction operation in the decoding process will be described with reference to the flowchart of FIG.

  First, the standard processing time T is set by the code stream extraction means 201 (step 1002 in FIG. 10). For example, it is a standard time required from the time the user designates an image to the display, such as 3 seconds. Next, the standard decoding layer number N, which is a parameter for setting the image quality, is set (step 1003 in FIG. 10). If N = 2, the standard number of layers to be decoded is two layers. In the code stream of Table 1, the layers determined by Dmax = 65 (PSNR = 30) are decoded.

Next, the code stream extraction unit 201 reads the predicted decoding rate p [bytes / sec] from the decoding rate storage unit 207 (step 1004 in FIG. 10). This value is updated each time an image is decoded, but an appropriate value is set in advance as an initial value for the first image. Next, the code amount detection unit 205 detects the code amount b [bytes] of the N layer of the code stream stored in the code stream storage unit 210 (step 1005 in FIG. 10). For example, in order to calculate the code amount of layer 0 in the code stream of FIG. 5, the code amount of each packet written in the packet header of the packets K _0,0 , K _0,1 , K _0,2 is read. Is obtained.

  Next, the extracted packet determining means 206 calculates a predicted processing time t = b / p [sec], which is a predicted value of the time required for processing (step 1006 in FIG. 10). Here, the predicted processing time t is compared with the standard processing time T (step 1007 in FIG. 10). If the predicted processing time t is shorter than the standard processing time T, only the code amount Tp [byte] that can be decoded by the standard processing time is extracted and decoded (steps 1008 and 1010 in FIG. 10). Conversely, if the predicted processing time t is longer than the standard processing time T, N layers are extracted and decoded (steps 1009 and 1010 in FIG. 10). In the second embodiment, when a code is extracted, a code data boundary to be extracted is set as a packet boundary. In the case of extracting code data of a certain code amount, it is actually assumed that the nearest packet boundary exceeding the code amount is extracted.

  The decoding speed measuring means 208 receives a decoding start signal output by the entropy decoding means at the start time of the process and a decoding end signal output by the wavelet inverse transform means at the end time of the process, and decodes from the difference between the reception times. Is calculated (step 1011 in FIG. 10). Then, the decoding speed p [byte / sec] for the code stream is calculated from the code amount of the extracted code stream and the processing time Δt (step 1012 in FIG. 10), and written in the decoding speed storage means 207. At the time of decoding the next image, the decoding speed for the current code stream is applied as the predicted value.

  The above is the processing procedure for one code stream, and if the next image to be decoded is designated, the processing is executed from the predicted decoding speed reading (step 1004 in FIG. 10) for the next image.

  In the second embodiment, the layer division is described such that the coding distortion becomes a predetermined value. However, the layer division is not necessarily the coding distortion. For example, the subjective evaluation scale (MOS) is a predetermined value. It is possible to divide up to an encoding pass as a layer. In addition, although the method of specifying the number of layers as a measure of the image quality to be extracted has been shown, the number of layers is not necessarily limited, and for example, a mean square error, PSNR, or the like may be used.

  If only the number of standard decoding layers is always decoded, the decoding time will be significantly shorter than the standard processing time, and only low-quality images can be played back compared to the decoded image with code data corresponding to the standard processing time. Occurs. In the second embodiment, in such a case, by extracting code data that can be decoded in the standard processing time, an effect of reproducing an image with higher image quality within the standard processing time can be obtained.

  On the other hand, if only the code amount that requires the standard processing time is to be decoded, only code data that cannot achieve the image quality obtained from the standard decoding layer may be decoded. In this embodiment, in such a case, an effect that an image with higher image quality is reproduced is obtained by decoding only the code data of the standard number of layers.

  As described above, in the second embodiment, it is possible to extract and decode code data having a code amount appropriate for viewing by considering both the image quality of the reproduced image and the processing time required for decoding. .

Embodiment 3 FIG.
In the first and second embodiments, it is assumed that the code stream is in the decoding apparatus. In the third embodiment, the code stream is stored in the code stream storage apparatus connected to the decoding apparatus and the network. The case will be described. It is assumed that encoding is not performed in real time, but is executed in advance and a code stream is accumulated. The code stream generation process is the same as that of the first embodiment shown in FIG.

  FIG. 11 is a block diagram showing the configuration of the image decoding apparatus according to Embodiment 3 of the present invention, and the operation from code stream extraction to decoding will be described using this block diagram. As shown in FIG. 11, the image decoding apparatus according to Embodiment 2 includes a code stream storage unit 1101 that stores a code stream, and a code stream extraction that partially extracts necessary code data from the code stream storage unit 1101. Means 1110, code amount information of the designated layer, code stream / code amount information transmission means 1111 for transmitting the extracted code stream, code amount detection means 1112 for detecting the code amount of the designated layer, Code stream / code amount information receiving means 1120 for receiving the code amount information of the specified layer and the extracted code stream, code stream decoding means 1130 for decoding the code stream and generating a reproduced image, and a packet to be decoded Extraction packet determining means 1125 for determining the decoding speed, and decoding speed Decoding speed storage means 1126 for storing the decoding speed, decoding speed measuring means 1127 for calculating the decoding speed from the processing time required for entropy decoding, inverse quantization, and inverse wavelet transform and the amount of code processed, and the transmission speed Transmission rate storage means 1128 to be placed, and transmission rate measurement means 1129 for measuring the transmission rate of the code stream.

  The code stream decoding unit 1130 includes an entropy decoding unit 1121 that performs binary arithmetic decoding of the code stream to reproduce the wavelet transform coefficient quantized value, and dequantizes the wavelet transform coefficient quantized value with a preset quantization step size. An inverse quantization unit 1122 for processing and a wavelet inverse transform unit 1123 for generating a reproduction image data by performing an inverse wavelet transform on the wavelet transform coefficient.

  Hereinafter, the operation will be described with reference to the flowchart of FIG. 12 focusing on the code stream extraction method. There are two types of codestream extraction methods, an image quality priority mode and a processing time priority mode, which are designated in advance by the user. In the third embodiment, the time priority mode will be described.

  First, the standard processing time T is set by the code stream extraction means 1110 (step 1202 in FIG. 12). For example, it is a standard time required from the time the user designates an image to the display, such as 3 seconds. Next, the standard decoding layer number N, which is a parameter for setting the image quality, is set (step 1203 in FIG. 12). If N = 2, the standard number of layers to be decoded is two layers. In the code stream of Table 1, the layers determined by Dmax = 65 (PSNR = 30) are decoded.

  The extracted packet determination unit 1125 reads the predicted decoding rate p1 [bytes / sec] from the decoding rate storage unit 1126 (step 1204 in FIG. 12). Similarly, the predicted transmission rate p2 [bytes / sec] is read from the transmission rate storage means 1128 (step 1205 in FIG. 12). These values are updated every time the image is decoded, but appropriate values are set in advance as initial values for the first image.

  The code stream / code amount information receiving unit 1120 requests code amount information of the number of standard layers from the code stream / code amount information transmitting unit 1111 connected by the network. Further, the code stream / code amount information transmitting unit 1111 issues a request to the code amount extracting unit 1112 to acquire the code amount, and the code amount extracting unit 1112 uses the code stream stored in the code stream storage unit 1101. The code amount of the standard layer number is read (step 1206 in FIG. 12). The code amount of each packet is obtained by reading the code amount of each packet header written in the packet header of the code stream.

  The code amount extraction unit 1112 outputs the code amount information to the code stream / code amount information transmission unit 1111, and further encodes the code stream / code amount information transmission unit 1111 to the code stream / code amount information reception unit 1120 via the network. Quantity information is transmitted (step 1207 in FIG. 12). The received code amount information is sent to the extracted packet determining means 1125.

  The extracted packet determination means 1125 uses the code amount b for the number of standard layers, the predicted decoding speed p1, and the predicted transmission speed p2, and the predicted value of the time required for processing, the predicted processing time t = b / p1 + b / p2 [sec]. Is calculated (step 1208 in FIG. 12). Then, the predicted processing time t is compared with the standard processing time T (step 1209 in FIG. 12).

  If the predicted processing time t is shorter than the standard processing time T, N layers are extracted from the code stream and received (step 1211 in FIG. 12). When receiving, first, the code stream / code amount information receiving unit 1120 requests the code stream / code amount information transmitting unit 1111 for packets of N layers. The code stream / code amount information transmission unit 1111 issues a request to the code stream extraction unit 1110 to extract a designated packet, and receives the code stream. The code stream / code amount information transmitting unit 1111 transmits the code stream extracted via the network, and the code stream / code amount information receiving unit 1120 receives the code stream.

  On the contrary, if the predicted processing time t is longer than the standard processing time T, a packet corresponding to the code amount T · p1 · p2 (p1 + p2) [byte] that can be decoded in the standard processing time is extracted and received (FIG. 12). Step 1210). In the present embodiment, when a code is extracted, a code data boundary to be extracted is set as a packet boundary. In the case of extracting code data of a certain code amount, it is actually assumed that the nearest packet boundary exceeding the code amount is extracted.

  The received code stream is processed in the order of entropy decoding means 1121, inverse quantization means 1122, and wavelet inverse transform means 1123, and reproduced image data is output (step 1212 in FIG. 12).

  The decoding speed measuring means 1126 receives a processing start signal output from the entropy decoding means at the start time of the process and a processing start signal output from the wavelet inverse transform means at the end time of the process, and decodes from the difference between the reception times. Is calculated (step 1213 in FIG. 12). Then, the decoding speed p1 [byte / sec] for the code stream is calculated from the code amount of the extracted code stream and the decoding time Δt1, and written to the decoding speed storage means 1126 (step 1214 in FIG. 12).

  In the transmission rate measuring means 1129, the code stream / code amount information receiving means 1120 receives the reception start signal output at the reception start time of the code stream and the reception end signal output at the reception end time, and the time required for reception from the time difference between them. Δt2 is calculated (step 1215 in FIG. 12). Then, the transmission rate p2 [byte / sec] for the code stream is calculated from the code amount of the extracted code stream and the transmission time Δt2, and written to the decoding rate storage means (step 1216 in FIG. 12). At the time of decoding the next image, the decoding speed and transmission speed for the current code stream are applied as predicted values.

  The above is the processing procedure for one code stream. If an image to be decoded next is specified, the process is executed for the next image after reading the predicted decoding speed (step 1204 in FIG. 12).

  In the third embodiment, the layer division has been described in which the coding distortion has a predetermined value. However, the layering is not necessarily the coding distortion. For example, the subjective evaluation scale (MOS) is a predetermined value. It is possible to divide up to an encoding pass as a layer. In addition, although the method of specifying the number of layers as a measure of the image quality to be extracted has been shown, the number of layers is not necessarily limited, and for example, a mean square error, PSNR, or the like may be used.

  In the third embodiment, the code amount information is transmitted every time an image to be decoded is designated. However, the code amount information may be transmitted and stored in advance on the decoding side.

  If code data is simply decoded by the number of standard decoding layers, the decoding time may be significantly increased in some cases. In the third embodiment, in such a case, an effect that the decoding time is shortened can be obtained by limiting the code amount to a range that can be decoded in the standard processing time.

  On the other hand, if only the amount of code that requires the standard processing time is decoded, in some cases, even code data that can obtain an image quality higher than necessary may be decoded. In the third embodiment, in such a case, the decoding time is shortened by decoding only the code data of the standard decoding layer number.

  As described above, in the third embodiment, it is possible to extract and decode code data having a code amount appropriate for browsing by considering both the image quality of a reproduced image and the processing time required for decoding. .

Embodiment 4 FIG.
In the third embodiment, the time priority mode, which is a mode in which code data is extracted with the highest priority when the decoding processing time is within the standard processing time, has been described. In the fourth embodiment, code distortion is desired. An example of an image quality priority mode in which code data is extracted with the highest priority being set to be equal to or less than the value (achieving a desired image quality) will be described. Since the code stream generation method, the code stream configuration, and the block diagram of the decoding apparatus are the same as those in the third embodiment, description thereof will be omitted and the code stream extraction operation in the decoding process will be described with reference to the flowchart of FIG.

  First, the standard processing time T is set (step 1302 in FIG. 13). For example, it is a standard time required from the time the user designates an image to the display, such as 3 seconds. Next, the standard decoding layer number N, which is a parameter for setting the image quality, is set (step 1303 in FIG. 13). If N = 2, the standard number of layers to be decoded is two layers. In the code stream of Table 1, up to the layer determined by Dmax = 65 is decoded.

  The predicted decoding speed p1 [bytes / sec] is read from the decoding speed storage means 1226 (step 1304 in FIG. 13). Similarly, the predicted transmission rate p2 [bytes / sec] is read from the transmission rate storage means 1128 (step 1305 in FIG. 13). These values are updated every time the image is decoded, but appropriate values are set in advance as initial values for the first image.

  The code stream / code amount information receiving unit 1120 requests code amount information of the number of standard layers from the code stream / code amount information transmitting unit 1111 connected by the network. Further, the code stream / code amount information transmitting unit 1111 issues a request to the code amount extracting unit 1112 to acquire the code amount, and the code amount extracting unit 1112 uses the code stream stored in the code stream accumulating unit 1101 as a standard. The code amount of the number of layers is read (step 1306 in FIG. 13). The code amount of each packet is obtained by reading the code amount of each packet header written in the packet header of the code stream.

  The code amount extraction unit 1112 outputs the code amount information to the code stream / code amount information transmission unit 1111, and further encodes the code stream / code amount information transmission unit 1111 to the code stream / code amount information reception unit 1120 via the network. Quantity information is transmitted (step 1307 in FIG. 13). The received code amount information is sent to the extracted packet determining means 1125.

  The extracted packet determination unit 1125 uses the code amount b, the predicted decoding speed p1, and the predicted transmission speed p2 for the number of standard layers, and the predicted value of the time required for processing, the predicted processing time t = b / p1 + b / p2 [sec]. Is calculated (step 1308 in FIG. 13). Then, the predicted processing time t and the standard processing time T are compared (step 1309 in FIG. 13).

  If the predicted processing time t is longer than the standard processing time T, N layers are extracted from the code stream and received (step 1311 in FIG. 13). When receiving, first, the code stream / code amount information receiving unit 1120 requests the code stream / code amount information transmitting unit 1111 for packets of N layers. The code stream / code amount information transmission unit 1111 issues a request to the code stream extraction unit 1110 to extract a designated packet, and receives the code stream. The code stream / code amount information transmitting unit 1111 transmits the code stream extracted via the network, and the code stream / code amount information receiving unit 1120 receives the code stream.

  Conversely, if the predicted processing time t is shorter than the standard processing time T, a packet corresponding to the code amount T · p1 · p2 (p1 + p2) [byte] that can be decoded in the standard processing time is extracted and received (FIG. 13). Step 1310). In the present embodiment, when a code is extracted, a code data boundary to be extracted is set as a packet boundary. In the case of extracting code data of a certain code amount, it is actually assumed that the nearest packet boundary exceeding the code amount is extracted.

  The received code stream is processed in the order of entropy decoding means 1121, inverse quantization means 1122, and wavelet inverse transform means 1123, and reproduced image data is output (step 1312 in FIG. 13).

  The decoding speed measuring means 1126 receives a processing start signal output from the entropy decoding means at the start time of the process and a processing start signal output from the wavelet inverse transform means at the end time of the process, and decodes from the difference between the reception times. Is calculated (step 1313 in FIG. 13). Then, the decoding speed p1 [byte / sec] for the code stream is calculated from the code amount of the extracted code stream and the decoding time Δt1, and written to the decoding speed storage means 1126 (step 1314 in FIG. 13).

  In the transmission rate measuring means 1129, the code stream / code amount information receiving means 1120 receives the reception start signal output at the reception start time of the code stream and the reception end signal output at the reception end time, and the time required for reception from the time difference between them. Δt2 is calculated (step 1315 in FIG. 13). Then, the transmission rate p2 [byte / sec] for the code stream is calculated from the code amount of the extracted code stream and the transmission time Δt2, and written in the decoding rate storage means (step 1316 in FIG. 13). At the time of decoding the next image, the decoding speed and transmission speed for the current code stream are applied as predicted values.

  The above is the processing procedure for one code stream, and if the next image to be decoded is designated, the processing is executed from the predicted decoding speed reading (step 1304 in FIG. 13) for the next image.

  In the fourth embodiment, the layer division has been described in which the coding distortion has a predetermined value. However, the layering is not necessarily the coding distortion. For example, the subjective evaluation scale (MOS) is a predetermined value. It is possible to divide up to an encoding pass as a layer. In addition, although the method of specifying the number of layers as a measure of the image quality to be extracted has been shown, the number of layers is not necessarily limited, and for example, a mean square error, PSNR, or the like may be used.

  In the fourth embodiment, the code amount information is transmitted every time an image to be decoded is designated. However, the code amount information may be transmitted and stored in advance on the decoding side.

  If decoding is performed for the number of standard decoding layers, the decoding time is significantly shorter than the standard processing time, and only low-quality images can be reproduced compared to a decoded image using code data corresponding to the standard processing time. . In the fourth embodiment, in such a case, by extracting code data that can be decoded in the standard processing time, an effect of reproducing an image with higher image quality within the standard processing time can be obtained.

  On the other hand, if only the code amount that requires the standard processing time is to be decoded, only code data that cannot achieve the image quality obtained from the standard decoding layer may be decoded. In the fourth embodiment, in such a case, an effect that a higher quality image is reproduced can be obtained by decoding only the code data of the standard number of layers.

  As described above, in the fourth embodiment, it is possible to extract and decode code data having a code amount suitable for browsing by considering both the image quality of a reproduced image and the processing time required for decoding. .

It is a block diagram which shows the structure of an image coding apparatus. It is a figure which shows a subband when wavelet transformation is carried out to decomposition level 2. It is a figure explaining a bit plane. It is a figure explaining decomposition | disassembly from a bit plane to an encoding pass. It is a conceptual diagram of a code stream. It is a figure which shows the determination method of a layer boundary. It is a flowchart which shows the adjustment method of (lambda). It is a block diagram which shows the structure of the image decoding apparatus which concerns on Embodiment 1 and 2 of this invention. It is a flowchart of the code stream extraction process in Embodiment 1 of this invention. It is a flowchart of the code stream extraction process in Embodiment 2 of this invention. It is a block diagram which shows the structure of the image decoding apparatus which concerns on Embodiment 3 of this invention. It is a flowchart of the code stream extraction process in Embodiment 3 of this invention. It is a flowchart of the code stream extraction process in Embodiment 4 of this invention.

Explanation of symbols

  201 code stream extraction means, 202 entropy decoding means, 203 inverse quantization means, 204 wavelet inverse transform means, 205 code amount detection means, 206 extracted packet determination means, 207 decoding speed storage means, 208 decoding speed measurement means, 210 code stream Storage means, 220 Code stream decoding means, 1101 Code stream storage means, 1110 Code stream extraction means, 1111 Code stream / code amount information transmission means, 1112 Code amount detection means, 1120 Code stream / code amount information reception means, 1121 Entropy decoding Means, 1122 inverse quantization means, 1123 wavelet inverse transform means, 1125 extraction packet determination means, 1126 decoding speed storage means, 1127 decoding speed measurement means, 1128 transmission Degree storage unit, 1129 transmission rate measuring means 1130 code stream decoding unit.

Claims (6)

An image decoding device that decodes a codestream configured by being divided into a plurality of layers,
Code stream storage means for storing the code stream;
Code stream extraction means for extracting predetermined code data from the code stream stored in the code stream storage means;
Code stream decoding means for decoding the code stream extracted by the code stream extraction means;
Code amount detection means for detecting a code amount that gives a specified layer or image quality;
Decoding speed measuring means for measuring the time required for decoding and the decoding speed;
Decoding speed storage means for storing the decoding speed measured by the decoding speed measuring means;
An image decoding apparatus comprising: an extracted packet determining unit that determines code data to be extracted from the code stream based on a code amount detected by the code amount detecting unit and a decoding rate stored in the decoding rate storage unit. The image decoding device according to claim 1,
The extracted packet determination means includes
Calculate the prediction processing time from the decoding speed for the code stream processed in the past and the code amount that gives the specified layer or image quality,
If the calculated prediction processing time is shorter than the specified processing time, the code data of the code amount that gives the specified layer or image quality is extracted,
An image decoding apparatus, wherein if the calculated prediction processing time is longer than the designated processing time, code data having a code amount that requires the designated processing time is extracted. The image decoding device according to claim 1,
The extracted packet determination means includes
Calculate the prediction processing time from the decoding speed for the code stream processed in the past and the code amount that gives the set layer or image quality,
If the calculated prediction processing time is longer than the specified processing time, the code data of the code amount that gives the specified layer or image quality is extracted,
If the calculated processing time is shorter than the designated processing time, code data having a code amount that requires the designated processing time is extracted. An image decoding device that decodes a codestream configured by being divided into a plurality of layers,
Code stream storage means for storing the code stream;
Code stream extraction means for extracting predetermined code data from the code stream stored in the code stream storage means;
A code stream / code amount information transmission means for transmitting a code amount and a code stream that gives a designated layer or a designated image quality;
Code amount detection means for detecting a code amount that gives a specified layer or image quality;
A code stream / code quantity information receiving means for receiving a code stream and a code stream that gives a designated layer or a designated image quality;
Code stream decoding means for decoding the extracted code stream;
Decoding speed measuring means for measuring the time required for decoding;
Decoding speed storage means for storing the decoding speed;
A transmission speed measuring means for measuring the transmission speed;
Transmission rate storage means for storing the transmission rate;
An image decoding apparatus comprising: extracted packet determining means for determining code data to be extracted from the code stream. The image decoding device according to claim 4, wherein
The extracted packet determination means includes
Calculate the processing time from the processing speed for the code stream processed in the past, the transmission speed and the code amount that gives the specified layer or image quality,
If the calculated processing time is shorter than the specified processing time, extract the code data of the code amount that gives the specified layer or image quality,
If the calculated processing time is longer than the designated processing time, code data having a code amount that requires the designated processing time is extracted. The image decoding device according to claim 4, wherein
The extracted packet determination means includes
Calculate the processing time from the processing speed for the code stream processed in the past, the transmission speed and the code amount that gives the specified layer or image quality,
If the calculated processing time is longer than the specified processing time, extract the code data of the code amount that gives the specified layer or image quality,
If the calculated processing time is shorter than the designated processing time, code data having a code amount that requires the designated processing time is extracted.

Images