JP2004343710A - Device and method for decoding moving image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain an excellent reproduction image quality with almost no visual disturbance by efficiently decoding all of or a part of moving image encoding data corresponding to the processing capability of a moving image decoder. <P>SOLUTION: The moving image decoder is so constituted that each frame of the moving image data is disassembled to a plurality of subbands and the moving image encoding data, which are generated by bit-plane-encoding the coefficient of the subband for each prescribed unit, are decoded. It is provided with a decoding processing time measurement part (105), which acquires the information for determining the magnitude of the difference between the times required for the decoding processing of the moving image encoding data at the prescribed unit, an undecoding bit plane decision part (107) for deciding the bit plane to be undecoded on the basis of the acquired information, a bit plane decoding part, which restores the coefficient of the subband from the encoding data other than the bit plane to be undecoded at the prescribed unit, and an inverse discrete wavelet converting part (104) which composites the restored coefficient of the subband and generates the frame data. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a moving image decoding apparatus and method for obtaining a reproduced image by decoding all or a part of moving image data in which each frame is independently encoded, a program realizing the method, and a program And a storage medium for storing.

  In general, moving image data encoding methods can be broadly classified into those that use correlation between frames and those that do not. Each method has advantages and disadvantages, and which method is suitable depends on the application used. For example, Motion JPEG is a method in which each frame of moving image data is treated as one still image and independently encoded, and is an example of an encoding method that does not use correlation between frames. Independent encoding for each frame facilitates moving image editing such as division, concatenation, and partial rewriting of moving images, and selects and decodes the number of decoding frames according to the processing capacity of the decoding side. There is an advantage that it is possible.

  2. Description of the Related Art In recent years, in an encoding method for independently encoding moving image data for each frame, a method of encoding each frame by combining wavelet transform and bit plane encoding has attracted attention. Such a moving picture coding method can perform decoding by changing the spatial resolution step by step using the mechanism of subband decomposition in the wavelet transform, and can also perform decoding by changing the number of decoding bit planes. There is a great feature that the pixel accuracy can be changed step by step.

  JPEG2000 (ISO / IEC 15444), which is an image coding system whose standardization is being advanced in ISO / IEC JTC1 / SC29 / WG1, is also configured by a combination of wavelet transform and bit plane coding. Part 3 of the standard defines a file format when JPEG2000 is applied to encoding of each frame of a moving image under the name of Motion JPEG2000.

  Such a moving picture coding method represented by Motion JPEG2000 has advantages such as flexibility of decoding resolution and decoding pixel precision as described above, but has a high load of coding / decoding processing by bit plane coding. There is a disadvantage that. In particular, when a video recorded by a dedicated moving picture recording device is reproduced by a personal computer, there is a problem that all data cannot be decoded and displayed in real time depending on the performance of the computer.

  With respect to such a problem, there is disclosed a method of determining a desired decoding processing time for decoding a frame, allocating a decoding processing time to an encoding processing unit, and decoding in bit plane units within the allocated decoding processing time. (For example, see Patent Document 1).

JP-A-11-288307

  However, in a moving picture decoding apparatus that terminates the decoding processing at a predetermined time as disclosed in Patent Document 1, the point at which the decoding processing is terminated (the number of decoding bit planes) is likely to change for each frame, and thus the moving picture is reproduced as a moving picture. In such a case, there is a problem that a temporal change in the shape of the distortion occurs, which causes visual disturbance as flicker.

  The present invention has been made in view of the above-described problems, and efficiently or efficiently decodes all or a part of encoded video data in accordance with the processing capability of a video decoding device. It is an object of the present invention to provide a moving image decoding apparatus and method capable of obtaining a reproduction image quality.

  In order to achieve the above object, each frame of the moving image data is decomposed into a plurality of sub-bands, and the sub-band coefficients are converted from a higher-order bit to a lower-order bit for each predetermined unit in a bit plane or a sub bit plane unit. The video decoding apparatus of the present invention, which decodes the encoded video data generated by encoding, comprises a time allocated to a decoding process of the predetermined unit of video encoded data and a time required for an actual decoding process. Decoding processing time information obtaining means for obtaining information for determining the magnitude of the difference between the non-decoding and non-decoding bit planes or sub-bit planes to be determined based on the information obtained by the decoding processing time information obtaining means. A bit plane determined by the non-decoded bit plane determining means, Bit plane decoding means for restoring sub band coefficients in predetermined units from encoded data, and sub band synthesis for synthesizing the plurality of sub band coefficients obtained by the bit plane decoding means to generate frame data Means.

  Further, each frame of the moving image data is decomposed into a plurality of sub-bands, and the coefficients of the sub-bands are generated by encoding the bits of the sub-band from the upper bits to the lower bits for each predetermined unit in bit plane or sub-bit plane units. The moving picture decoding method of the present invention for decoding moving picture encoded data determines a magnitude of a difference between a time allocated to decoding processing of the predetermined unit of moving picture encoded data and a time required for actual decoding processing. Decoding processing time information obtaining step of obtaining information to perform, based on the information obtained by the decoding processing time information obtaining step, a non-decoding bit plane, or a non-decoding bit plane determining step of determining a sub bit plane, From encoded data other than the bit plane or sub-bit plane determined in the non-decoding bit plane determining step, A bit plane decoding step of restoring band coefficients in the predetermined unit; and a subband combining step of combining the coefficients of the plurality of subbands obtained in the bitplane decoding step to generate frame data. Features.

  According to the present invention, it is possible to efficiently decode all or a part of encoded video data in accordance with the processing capability of the video decoding device and obtain good reproduction image quality with little visual interference.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the dimensions, materials, shapes, relative arrangements, and the like of the components exemplified in the present embodiment should be appropriately changed depending on the configuration of the apparatus to which the present invention is applied and various conditions. Is not limited to those examples.

<Outline of encoded video data>
First, moving picture coded data to be decoded by the moving picture decoding apparatus according to the present embodiment will be described.

  FIG. 1 is a diagram illustrating a configuration of a moving picture encoding apparatus 200 that generates encoded moving picture data to be decoded by the moving picture decoding apparatus according to the present embodiment. In the moving picture coding apparatus 200, each frame constituting a moving picture is independently coded by a coding method combining wavelet transform and bit plane coding. As shown in FIG. 1, a moving image data input unit 201, a discrete wavelet transform unit 202, a coefficient quantization unit 203, a bit plane encoding unit 204, a code string forming unit 205, a secondary storage device 206, and a signal line 207 are provided. ing.

  Next, the flow of the encoding process in the video encoding device 200 will be described while describing the operation of each component of the video encoding device 200 shown in FIG. Here, at a rate of 30 frames per second, monochrome moving image data of which the luminance value of one pixel is 8 bits is taken into the moving image encoding device 200 for 4 seconds (total of 120 frames) and encoded. It will be described as what is. That is, in the moving picture coding apparatus 200, the moving picture data of 30 frames per second inputted from the moving picture data input unit 201 is coded in frame units, and the coded data is finally stored in the secondary storage device 206. Is what you do.

  First, moving image data for 4 seconds is input from the moving image data input unit 201 at a rate of 30 frames per second. The moving image data input unit 201 is, for example, an imaging unit such as a digital camera, and can be realized by an imaging device such as a CCD and various image adjustment circuits such as gamma correction and shading correction. The moving image data input unit 201 sends the input moving image data to the discrete wavelet transform unit 202 frame by frame. In the following description, for convenience, each frame data is given a number from 1 in the order of input, and each frame data is identified by a number such as, for example, frame 1, frame 2,. In each frame, the horizontal pixel position (coordinate) is x, the vertical pixel position is y, and the pixel value at the pixel position (x, y) is represented by P (x, y).

  One frame of image data input from the moving image data input unit 201 is appropriately stored in an internal buffer (not shown) by the discrete wavelet transform unit 202, and two-dimensional discrete wavelet transform is performed. The two-dimensional discrete wavelet transform is realized by applying a one-dimensional discrete wavelet transform to each of the horizontal and vertical directions. FIG. 2 is a conceptual diagram for explaining subbands of an encoding target image processed by two-dimensional discrete wavelet transform.

  That is, a one-dimensional discrete wavelet transform is first applied in the vertical direction to the image to be encoded as shown in FIG. 2A, and the low-frequency sub-band L and the high-frequency sub-band L are applied as shown in FIG. Decomposes into band H. Next, by applying a one-dimensional horizontal discrete wavelet transform to each subband, the subbands are decomposed into four subbands LL, HL, LH, and HH as shown in FIG. 2C.

  The discrete wavelet transform unit 202 of the video encoding device 200 further repeatedly applies a two-dimensional discrete wavelet transform to the sub-band LL obtained by the above-described two-dimensional discrete wavelet transform. As a result, the encoding target image can be decomposed into seven subbands LL, LH1, HL1, HH1, LH2, HL2, and HH2. FIG. 3 is a diagram illustrating seven sub-bands obtained by two two-dimensional discrete wavelet transforms.

  In the moving picture coding apparatus 200, the coefficient in each subband is represented as C (Sb, x, y). Here, Sb represents the type of subband, and is one of LL, LH1, HL1, HH1, LH2, HL2, and HH2. (X, y) represents coefficient positions (coordinates) in the horizontal and vertical directions when the coefficient position at the upper left corner in each subband is (0, 0).

  The moving image coding apparatus 200 includes two methods as a one-dimensional discrete wavelet transform for N one-dimensional signals s (n) (where n is an integer of 0 to N−1) in the discrete wavelet transform unit 202. One is conversion using an integer type 5 × 3 filter shown in equations (1) and (2), and the other is conversion using a real number type 5 × 3 filter shown in equations (3) and (4).

h (n) = s (2n + 1)
−floor {(s (2n) + s (2n + 2)) / 2} (1)

l (n) = s (2n)
+ Floor {(h (n-1) + h (n) +2) / 4} (2)

  h (n) = s (2n + 1)-(s (2n) + s (2n + 2)) / 2 (3)

  l (n) = s (2n) + (h (n-1) + h (n)) / 4 (4)

  Here, h (n) is a coefficient of the high frequency sub-band, l (n) is a coefficient of the low frequency sub-band, and floor {R} is a maximum integer value not exceeding the real number R. In addition, s (n) at both ends (n <0 and n> N-1) of the one-dimensional signal s (n) required in the calculation of the equations (1) and (2) and the equations (3) and (4). Is obtained from the value of the one-dimensional signal s (n) (n = 0 to N-1) by a known method.

  Which of the integer type 5 × 3 filter and the real type 5 × 3 filter is to be applied can be designated in frame units by a filter selection signal input from outside the device via the signal line 207. For example, when the filter selection signal input from the signal line 207 is “0”, the frame of interest is decomposed by an integer 5 × 3 filter, and when the filter selection signal is “1”, the frame of interest is a real number. Decomposition is performed by a type 5 × 3 filter.

  The coefficient quantization unit 203 quantizes the coefficient C (Sb, x, y) of each subband generated by the discrete wavelet transform unit 202 using a quantization step delta (Sb) determined for each subband. I do. Here, when the quantized coefficient value is represented as Q (Sb, x, y), the quantization process performed by the coefficient quantization unit 203 can be represented by Expression (5).

Q (Sb, x, y) = sign {C (Sb, x, y)}
× floor {| C (Sb, x, y) | / delta (Sb)} (5)

  Here, sign {I} is a function representing the sign of the integer I, and returns 1 if I is positive and -1 if I is negative. Floor {R} represents the maximum integer value not exceeding the real number R. However, the above-described quantization processing is applied only when a real type 5 × 3 filter is selected and used in the discrete wavelet transform unit 202, and is applied to an integer type by a filter selection signal input from a signal line 207. When the 5 × 3 filter is selected, the coefficient C (Sb, x, y) is output as a quantized coefficient value. That is, in this case, Q (Sb, x, y) = C (Sb, x, y).

  The bit plane encoding unit 204 encodes the coefficient value Q (Sb, x, y) quantized by the coefficient quantization unit 203. Note that various methods have been proposed as encoding methods, such as a method of facilitating random access by dividing the coefficients of each subband into blocks and separately encoding the coefficients, but in order to simplify the description, To be encoded in subband units.

Coding of the quantized coefficient value Q (Sb, x, y) of each subband (hereinafter, simply referred to as “coefficient value”) is performed by encoding the coefficient value Q (Sb, x, y) in the subband. This is performed by expressing the absolute value in a natural binary number and performing binary arithmetic coding with priority on the bit plane direction from the upper digit to the lower digit. A description will be given by describing the n-th bit from the bottom in the case where the coefficient value Q (Sb, x, y) of each subband is expressed in natural binary notation as Qn (x, y). Note that a variable n representing a digit of a binary number is referred to as a bit plane number, and the bit plane number n has an LSB (least significant bit) as the 0th digit.
FIG. 4 is a flowchart illustrating a processing procedure for encoding subband Sb in bit plane encoding section 204. As shown in FIG. 4, first, the absolute value of the coefficient in the subband Sb to be encoded is checked, and the maximum value Mabs (Sb) is obtained (step S601). Next, the number of digits N _BP (Sb) required when Mabs (Sb) is represented by a binary number in order to represent the absolute value of the coefficient in the sub-band Sb is obtained using Expression (6) (Step S602). ).

N _BP (Sb) = ceil {log2 (Mabs (Sb) +1)} (6)

  Note that ceil {R} represents the smallest integer value equal to or greater than the real number R.

Next, the number of significant digits N _BP (Sb) is substituted for the bit plane number n (step S603). Then, 1 is subtracted from the bit plane number n to obtain n-1 and substitute for n (step S604).

  Further, the n-th bit plane is encoded using a binary arithmetic code (step S605). When each bit in the bit plane is encoded, it is classified into several states (contexts) from the encoded information, and is encoded using different appearance probability prediction models. The moving picture coding apparatus 200 uses MQ-Coder as an arithmetic code to be used. A procedure for encoding a binary symbol generated in a certain state (context) using the MQ-Coder, or an initialization procedure and an end procedure for arithmetic encoding processing, are described in International Standard ISO / ISO for Still Images. Since it is described in detail in the IEC15444-1 recommendation and the like, the description is omitted here.

Also, in the moving picture coding apparatus 200, the arithmetic coder is initialized at the start of coding of each bit plane, and termination processing of the arithmetic coder is performed at the end. Immediately after "1" which is coded at the beginning of each coefficient, the sign of the coefficient is represented by 0 or 1, and arithmetically coded. Here, the value is 0 for a positive value and 1 for a negative value. For example, when the coefficient is −5 and the number of significant digits N _BP (Sb) of the subband Sb to which the coefficient belongs is 6, the absolute value of the coefficient is represented by a binary number 00001, and the upper digit is obtained by encoding each bit plane. To the lower digit. Then, at the time of encoding the second bit plane (in this case, the fourth digit from the top), the first “1” is encoded, and immediately after that, the sign “1” is arithmetically encoded.

  Next, it is determined whether the bit plane number n is 0 (step S606). As a result, if n is 0, that is, if encoding of the LSB plane has been performed in step S605 (YES), the encoding process of the subband Sb ends. Otherwise (NO), the process moves to step S604.

  By the above-described processing, all the coefficients of the sub-band Sb can be encoded, and a code string CS (Sb, n) corresponding to each bit plane n is generated. The generated code sequence CS (Sb, n) is sent to the code sequence forming unit 205, and is temporarily stored in a buffer (not shown) in the code sequence forming unit 205.

  In the code sequence forming unit 205, when the encoding of the coefficients of all the subbands is completed by the bit plane coding unit 204 and all the code sequences are stored in the internal buffer, the code sequences stored in the internal buffer in a predetermined order Read out. Then, necessary additional information is inserted to form a code string corresponding to one frame and output to the secondary storage device 206.

  The final code string generated by the code string forming unit 205 includes a header and coded data hierarchized into three levels: level 0, level 1, and level 2. The header includes information such as the number of pixels in the horizontal and vertical directions of the image, the number of applications of the two-dimensional discrete wavelet transform, information specifying the selected filter, and the quantization step delta (Sb) of each subband. Is stored.

The coded data of level 0 is composed of a code string of CS (LL, _NBP (LL) -1) to CS (LL, 0) obtained by coding the coefficients of the subband LL. The level 1 is, LH1, HL1, encodes the coefficients of each subband HH1 code string obtained by _CS from _{(LH1, N BP (LH1)} -1) CS (LH1,0), CS (HL1, N _BP (HL1) -1) from CS (HL1,0), and _{CS (HH1, N BP (HH1} ) consists -1) with CS (HH1,0). Furthermore, level 2, LH2, HL2, encodes the coefficients of each subband HH2 code string obtained by _CS from _{(LH2, N BP (LH2)} -1) CS (LH2,0), CS (HL2, N _BP (HL2) -1) from CS (HL2,0), and _{CS (HH2, N BP (HH2} ) consists -1) with CS (HH2,0).

  FIG. 5 is a diagram illustrating a detailed structure of a code string corresponding to one frame data generated by the code string forming unit 205.

  In the code string configured as shown in FIG. 5, a decoded image having the original quarter resolution can be obtained by decoding the header and the coded data of level 0 on the decoding side. Further, by adding and decoding the level 1 encoded data, a restored image having the original 1/2 resolution can be obtained. Furthermore, when decoding is performed in addition to the encoded data of level 2, it is possible to decode the image by gradually increasing the resolution so that a restored image of the original resolution can be obtained.

  On the other hand, when only some upper bit planes of the bit plane encoded data of each level are decoded, a coarse decoded image is gradually decoded. The transform coefficients of each subband can be restored with higher accuracy, and the decoded image quality can be improved.

  The secondary storage device 206 is, for example, a storage device such as a hard disk or a memory, and stores therein the code string generated by the code string forming unit 205. In the secondary storage device 206, the code strings of the respective frames output from the code string forming unit 205 are connected to form encoded data of moving image data. FIG. 6 shows the structure of the encoded video data stored in the secondary storage device 206. In the first header, additional information as a moving image, for example, the number of frames, a reproduction frame rate, and the like are stored.

<First embodiment>
FIG. 7 is a block diagram illustrating a configuration of the video decoding device 100 according to the first embodiment of the present invention. The same reference numerals are used for blocks common to the above-described video encoding device 200. As shown in FIG. 7, the moving image decoding apparatus 100 according to the first embodiment includes a secondary storage device 206, a code string reading unit 101, a bit plane decoding unit 102, an inverse discrete wavelet transform unit 104, and a moving image data output. A decoding unit 105, a decoding processing time measuring unit 106, and a non-decoding bit plane determining unit 107.

  Hereinafter, an operation procedure of the video decoding device 100 according to the first embodiment will be described with reference to FIG.

  The encoded video data to be decoded by the video decoding device 100 of the first embodiment is the encoded data generated by the video encoding device 200 described above. In addition, when generating encoded video data, it is assumed that an integer type 5 × 3 filter is used for all frames. That is, a signal for selecting an integer type 5 × 3 filter is input from the signal line 207 of the moving picture coding apparatus 200 described above to perform coding of moving picture data.

  The decoding of the encoded moving image data is performed in units of frames in the encoded data. Therefore, the code string reading unit 101 reads the encoded data of the frame of interest from the encoded data stored in the secondary storage device 206, and stores the encoded data in an internal buffer (not shown). Reading of encoded data in units of frames is performed in order, such as frame 1 and frame 2.

  The bit plane decoding unit 102 reads the encoded data stored in the internal buffer of the code string reading unit 101 in the order of subbands, and decodes the quantized transform coefficient data Q (Sb, x, y). The processing in the bit plane decoding unit 102 forms a pair with the bit plane encoding unit 204 shown in FIG.

  That is, the bit plane coding unit 204 performs binary arithmetic coding of each bit of the absolute value of the coefficient from the upper bit plane to the lower bit plane in a predetermined context. On the other hand, the bit plane decoding unit 102 similarly decodes the binary arithmetic coded data from the upper bit plane to the lower bit plane in the same context as when encoding, and restores each bit of the coefficient. Further, with respect to the sign of the coefficient, the arithmetic code is decoded using the same context at the same timing as the encoding.

However, at this time, the non-decoding bit plane determining unit 107 indicates the number of lower bit planes ND (Sb) not to be decoded for each subband, and the bit plane decoding unit 102 performs decoding processing on the specified number of lower bit planes. Do not do. For example, when decoding the coefficient of the sub-band HH2, when the number ND (HH2) of the non-decoded bit planes of the sub-band HH2 given from the non-decoded bit plane determining unit 107 is 2, the coefficient is read from the code string reading unit 101. restore the coefficients of subband decoded from the encoded data of the coefficients of the subband _{HH2 CS (HH2, N BP (} HH2) -1) to CS (HH2,2), CS (HH2,1 ) and CS The two bit planes (HH2, 0) are not decoded.

  The inverse discrete wavelet transform unit 104 performs an inverse transform of the wavelet transform process in the discrete wavelet transform unit 202 in FIG. 1 to restore frame data. In the video decoding device 100 according to the first embodiment, since the video coded data generated by using the integer type 5 × 3 filter in all frames is to be decoded, the above equation (1) is used. The inverse transformation corresponding to (2) is performed.

  When playing back and displaying a moving image, each frame data is displayed at a predetermined time. On the other hand, the output from the inverse discrete wavelet transform 104 is not synchronized with the time to be displayed because it depends on the time required for the decoding process. For this reason, it is necessary to store the decoded frame data in a buffer and adjust the display time. For example, the moving image data output unit 105 is realized by an interface of a network line, and a buffer storage process for adjusting the display time is performed outside the moving image decoding device 100 according to the first embodiment. Alternatively, the processing may be performed inside the moving image data output unit 105.

  As a specific example of the moving image data output unit 105, a case where a display is connected to the moving image data output unit 105 to display moving images will be described. FIG. 16 is a diagram showing one mode of the moving image data output unit 105 and connection to a display. In the figure, reference numeral 1601 denotes a buffer capable of storing a plurality of frame data, 1602 denotes a display interface, and 1603 denotes a display. The buffer 1601 sequentially stores restored image data output from the inverse discrete wavelet transform unit 104 at variable time intervals. When the restored image data is stored, the frame number of the restored image data to be stored and the storage address are held so that they can be sequentially taken out later. The display interface 1602 sequentially takes out the restored image data from the buffer 1601 at a fixed time interval (for example, every 1/30 second if the moving image is 30 frames per second) according to the frame rate of the moving image data, and displays it on the display 1603. To be displayed. The extracted frame data is deleted from the buffer 1601. As described above, the buffer 1601 plays a role of adjusting the difference in the time interval between the input of the restored image data from the inverse discrete wavelet transform unit 104 and the data extraction by the display interface.

  The moving picture decoding apparatus according to the present embodiment performs control so that the average of the decoding processing time of one frame becomes the target decoding time T by the processing described later. However, since the time required for data restoration of each frame varies, the number of frame data stored in the buffer 1601 changes. Therefore, in order to prevent the frame data from being unprepared in the buffer even when the display time has elapsed, reproduction display is started after a predetermined time from the start of video decoding, or after a predetermined number of frames are accumulated in the buffer. Adjust the timing of video playback start, such as starting playback. In addition, when the buffer capacity is limited, and when a predetermined number or more of frame data is stored in the buffer, it is necessary to control to stop the decoding process of the frame data.

  Then, the moving image data output unit 105 outputs the restored image data output from the inverse discrete wavelet transform unit 104 to the outside of the device. The moving image data output unit 105 can be realized by, for example, an interface to a network line or a display device.

  The decoding processing time measuring unit 106 measures the time Dt required for each frame from the start of reading the encoded frame data by the code string reading unit 101 to the output of the restored frame data by the moving image data output unit 105. Output to decoding bit plane determining section 107. However, as illustrated in FIG. 16, when the moving image data output unit 105 includes a buffer for storing the frame data therein and adjusts the output to the outside of the apparatus at a constant interval, the time Dt is set to the buffer 1601. This is the time until the frame data is stored.

  Non-decoding bit plane determining section 107 determines a non-decoding bit plane for each subband based on the decoding processing time of one frame output from decoding processing time measuring section 106. The non-decoding bit plane determining unit 107 includes therein a variable Q (hereinafter, referred to as a “Q factor”) serving as an index value for determining the number of non-decoding bit planes, and a non-decoding bit of each subband in each Q factor. It holds a table indicating the number of planes, a target decoding processing time T, and a time difference ΔT. FIG. 8 shows an example of a table representing the correspondence between the Q factor and the number of non-decoded bit planes of each subband.

  FIG. 9 is a flowchart showing the flow of the decoding processing of the encoded video data by the video decoding apparatus 100. As shown in FIG. 9, first, the Q factor and the time difference ΔT are reset to 0 at the start of the decoding of the encoded video data, that is, before the decoding of the encoded data of the frame 1 starts (step S701).

  Next, the non-decoding bit plane determining unit 107 reads out the number of non-decoding bit planes of each subband from the table based on the Q factor, and sets the number in the bit plane decoding unit 102 (step S702).

  Subsequently, one frame is decoded by the processing of the inverse discrete wavelet transform unit 104 from the code string reading unit 101, and the frame data is output to the moving image data output unit 105 (step S703).

  The decoding processing time measurement unit 106 measures the time Dt required for the decoding processing of one frame performed in step S703, and passes it to the non-decoding bit plane determination unit 107 (step S704).

  The non-decoding bit plane determining unit 107 obtains the difference between the target decoding time T of one frame and the actually applied decoding processing time Dt, and adds the difference to the held time difference ΔT (step S705).

  Next, the Q factor is updated according to the value of ΔT (step S706). If ΔT is larger than a predetermined threshold Uq (Uq> 0), 1 is subtracted from Q to reduce the value. ΔT becomes larger than the predetermined threshold value when the total sum of the decoding time actually taken with respect to the total sum of the target time is small, and the non-decoding is performed by reducing the value of Q in order to improve the decoding image quality. Reduce the number of bit planes. Conversely, if ΔT is smaller than a predetermined threshold Lq (Lq <0), 1 is added to Q to increase the value. ΔT becomes smaller than the predetermined threshold Lq when the total sum of the decoding times actually taken with respect to the total sum of the target times is large, and by increasing the value to shorten the decoding time of one frame. Increase the number of non-decoded bit planes. However, the range of the value of Q is from 0 to 9, and is 0 when it is smaller than 0 and 9 when it is larger than 9 by the above-described update processing. If ΔT is within the range between the thresholds Lq and Uq (Lq ≦ ΔT ≦ Uq), the sum of the decoding times actually taken with respect to the sum of the target times is just within a good range. Leave the value unchanged.

  It is determined whether or not the decoded frame is the last frame. If it is not the last frame (NO), the process proceeds to step S702, where the next frame is decoded and the last frame is decoded. (YES) ends the decoding processing of the encoded video data (step S707).

  As described above, the Q factor that is the index value of the number of non-decoding bit planes is determined from the accumulated value of the difference between the time required for decoding one frame and the target decoding time, and the non-decoding for each subband is determined according to the Q factor. By changing the number of bit planes, it is possible to control the decoding processing time while minimizing visual defects in the reproduced image.

<Second embodiment>
In the first embodiment, the description has been made on the assumption that all the moving image encoded data to be decoded are subjected to subband decomposition by an integer type 5 × 3 filter. In the moving picture decoding apparatus according to the second embodiment, a case will be described in which moving picture coded data subjected to subband decomposition using a real number type 5 × 3 filter is to be decoded. That is, a signal for selecting the real number type 5 × 3 filter is input from the signal line 207 to the moving picture coding apparatus 200 described above, and each frame of the moving picture is coded. When encoding, the quantization step delta (Sb) of each subband is set to be the same for all frames.

  FIG. 10 is a block diagram illustrating a configuration of a video decoding device 300 according to the second embodiment of the present invention. The same reference numerals are used for blocks common to the above-described video encoding device 200 and the video decoding device 100 of the first embodiment. As shown in FIG. 10, the video decoding device 300 according to the second embodiment includes a secondary storage device 206, a code string reading unit 904, a bit plane decoding unit 102, a coefficient inverse quantization unit 901, an inverse discrete wavelet transform. A video data output unit 105; a decoding processing time measuring unit 106; and a non-decoded bit plane determining unit 903.

  Hereinafter, the operation procedure of the video decoding device 300 according to the second embodiment will be described with reference to FIG.

  First, the coded data of the target frame is read from the coded video data stored in the secondary storage device 206 by the coded sequence reading unit 904 in the same manner as the coded sequence reading unit 101 of the first embodiment. It is stored in an internal buffer (not shown). At this time, the quantization step delta (Sb) of each sub-band Sb is read from the header of the coded data of the read frame and stored in an internal buffer (not shown).

  The bit plane decoding unit 102 decodes the quantized transform coefficient data Q (Sb, x, y) from the encoded data stored in the internal buffer of the code sequence reading unit 904 in the same manner as in the first embodiment. I do. Also in the video decoding device 300 of the second embodiment, the decoding process is not performed on ND (Sb) lower-order bit planes specified by the non-decoding bit plane determining unit 903.

  The coefficient inverse quantization unit 901 calculates the quantization step delta (Sb) determined for each sub-band and the quantized coefficient value decoded by the bit plane decoding unit 102 from Q (Sb, x, y). The coefficient C (Sb, x, y) of each subband is restored.

  The inverse discrete wavelet transform unit 902 performs inverse transform of the wavelet transform process in the discrete wavelet transform unit 202 in FIG. 1 to restore frame data. In the moving picture decoding apparatus 300 of the second embodiment, the moving picture coded data generated by using the real number type 5 × 3 filter is to be decoded in every frame. The inverse transformation corresponding to (4) is performed.

  Then, the moving image data output unit 105 outputs the restored image data output from the inverse discrete wavelet transform unit 902 to the outside of the device.

  As in the first embodiment, the decoding processing time measuring unit 106 performs, for each frame, from the start of reading the encoded frame data by the code string reading unit 904 to the output of the restored frame data by the moving image data output unit 105. Is measured and output to the non-decoding bit plane determining unit 903.

  The non-decoding bit plane determining unit 903 determines a non-decoding bit plane for each subband based on the decoding processing time of one frame output from the decoding processing time measuring unit 106. The non-decoding bit plane determining unit 903 holds therein a table indicating the number ND (Sb) of non-decoding bit planes of each subband, a target decoding processing time T, a time difference ΔT, and a subband index SI. FIG. 11 shows an example of a table that holds the number ND (Sb) of non-decoded bit planes for each subband Sb.

  FIG. 12 is a flowchart illustrating a flow of a decoding process of moving image encoded data by the image decoding device 300. As shown in FIG. 12, first, the subband index SI and the time difference ΔT are reset to 0 at the start of the decoding of the encoded video data, that is, before the decoding of the encoded data of the frame 1 is started (step S1101).

  Next, the number of non-decoded bit planes ND (Sb) for each subband held in the non-decoded bit plane determination unit 903 is initialized to 0 (step S1102).

  Next, the number of non-decoded bit planes ND (Sb) stored in the non-decoded bit plane determining unit 903 is read and set to the bit plane decoding unit 102 (step S1103).

  Subsequently, one frame is decoded by the processing of the inverse discrete wavelet transform unit 902 from the code string reading unit 904, and the frame data is output to the moving image data output unit 105 (step S1104).

  The decoding processing time measuring unit 106 measures the time Dt required for the decoding processing of one frame performed in step S1104, and passes it to the non-decoded bit plane determining unit 903 (step S1105).

  The non-decoding bit plane determining unit 903 obtains the difference between the target decoding time T of one frame and the decoding processing time Dt actually taken, and adds the difference to the held time difference ΔT (step S1106).

  Next, the table holding the number ND (Sb) of non-decoded bit planes and the subband index SI are updated according to the value of ΔT (step S1107).

  FIG. 13 is a flowchart showing the flow of the process performed in step S1107. First, it is determined whether ΔT is larger than a predetermined threshold Uq (Uq> 0) set in advance (step S1201). If it is larger (YES), 1 is subtracted from the subband index SI (step S1202). Then, it is determined whether or not SI is -1 (step S1203). If it is -1, SI is set to 6 (step S1204). Next, 1 is subtracted from the number ND (S (SI)) of non-decoded bit planes of the subband S (SI) corresponding to the subband index SI (step S1205). Since ΔT becomes larger than the predetermined threshold value when the total sum of the decoding time actually taken with respect to the total sum of the target time is small, the decoding image quality is improved by reducing the number of non-decoding bit planes. The correspondence between the subband index SI and the subband is as shown in FIG. For example, if SI is 2, the corresponding subband is HL2, and 1 is subtracted from the value of ND (HL2). Then, ND (S (SI)) is compared with 0 (step S1206). If ND (S (SI)) is smaller than 0, ND (S (SI)) is set to 0 (step S1207). ), Return SI to 0 (step S1213).

  On the other hand, if ΔT ≦ Uq as a result of the comparison in step S1201 (NO), ΔT is compared with a predetermined threshold Lq (Lq <0) set in advance (step S1208), and if ΔT> Lq (NO). Ends the processing. If ΔT ≦ Lq (YES), 1 is added to ND (S (SI)) (step S1209). Since ΔT becomes smaller than the predetermined threshold value when the total sum of the decoding times actually taken with respect to the total sum of the target times is long, the decoding time of one frame can be reduced by increasing the number of non-decoding bit planes. Shorten. Subsequently, 1 is also added to SI (step S1210), and SI is compared with 7 (step S1211). If SI is 7 (YES), SI is set to 0 (step S1212).

  By the above processing, when ΔT is larger than the predetermined value or smaller than the predetermined value, the number ND (Sb) of non-decoded bit planes of one subband is changed by one level.

  Returning to the processing of FIG. 12, it is determined whether or not the frame subjected to the decoding processing is the last frame (step S1108). If it is not the last frame (NO), the processing moves to step S1103, and the next frame , And if it is the last frame (YES), the decoding process of the encoded video data ends.

  As described above, by changing the number of non-decoding bitplanes for each subband from the accumulated value of the difference between the time required for the decoding process of one frame and the target decoding time, visual defects of the reproduced image are suppressed as much as possible. The decoding processing time can be controlled.

<Third embodiment>
In the video decoding devices of the first and second embodiments, the non-decoding part is determined in units of bit planes, but each bit in the bit plane is classified into a category based on a coded part around a bit of interest. It is also possible to divide the data into a plurality of paths (sub-bit planes) and determine a non-decoding part in units of paths. Hereinafter, an embodiment will be described in which a non-decoding portion is determined in units of paths.

The process of generating the encoded video data to be decoded by the video decoding device of the third embodiment is basically the same as the process of the video encoding device 200 shown in FIG. 1 described above. However, the bit plane encoding unit 204 uses a different bit plane encoding method. One bit plane is divided into a plurality of passes and encoded as described above. For simplicity of description, a specific method of dividing into paths is not described here, but encoding is performed by a method similar to the bit plane encoding method in JPEG2000 described in the ISO / IEC15444-1 recommendation. In JPEG2000, encoding is performed by decomposing into three passes, except for the most significant bit plane. Therefore, the number of effective bits of a certain subband Sb be a _N BP _(Sb), it is encoded by the (N BP (Sb) -1) × 3 + 1 paths. The code generated by each pass is referred to as CSP (Sb, n). Here, n is the number of the path, the first path number is ( _NBP (Sb) -1) × 3), and the last path number is 0.

  In the same manner as the code string is formed by arranging the coded data in bit plane units by the code string forming unit 205, the coded data of the paths is arranged to form a code string. FIG. 15 shows an example of the structure of the encoded moving image data to be decoded by the moving image decoding device of the third embodiment generated in this manner. As compared with the encoding targets of the first and second embodiments shown in FIG. 5, the elements constituting the encoded data are the bit encoded data CS (Sb, n) and the encoded data CSP (Sb, n). n is a bit plane or path number. Furthermore, the moving picture coded data to be decoded in the first and second embodiments includes all the bit plane coded data output from the bit plane coding unit 204. An example is shown in which discarded encrypted data is performed. For the subbands HH1, LH2, and HL2, the encoded data of the last pass is discarded, and for HH2, the encoded data of the last two passes are discarded.

  Note that the configuration of the video decoding device of the third embodiment is the same as the configuration of the video decoding device 100 of the first embodiment shown in FIG. Only the operation of the unit 107 is different.

  The bit plane decoding unit 102 decodes the coded data CSP (Sb, n) of the path and extracts the bits of each path by a decoding process paired with the bit plane coding of the moving image coding device 200 described above. Restore the subband coefficients. At this time, in the moving picture decoding apparatus 100 according to the first embodiment, the non-decoding bit plane determining unit 107 instructs the number of lower bit planes ND (Sb) not to be decoded. The number of paths NDP (Sb) is specified, and the bit plane decoding unit 102 does not decode the encoded data of the lower NDP (Sb) path. That is, CSP (Sb, 0) is not decoded from CSP (Sb, NDP (Sb) -1).

  The non-decoding bit plane determining unit 107 specifies the number of non-decoding paths NDP (Sb) by the same processing as instructing the bit plane decoding unit 102 of the number of non-decoding bit planes ND (Sb) in the first embodiment. I do.

  As described above, in the video decoding device 100 of the third embodiment, since the non-decoding part can be set in a coding unit finer than the bit plane, finer adjustment of the decoding image quality and decoding time can be performed. It becomes possible.

<Fourth embodiment>
In the first to third embodiments, the decoding range is adjusted based on the value ΔT obtained by accumulating the difference between the target decoding time T of each frame and the actual decoding processing time Dt. As described as a specific example of the moving image data output unit 105 in the description of the first embodiment, the moving image data output unit 105 is configured by a buffer and a display interface, and a moving image is displayed by connecting a display. In this case, the same processing can be performed by monitoring the number of frame data stored in the buffer without actually measuring the decoding processing time. Hereinafter, such an embodiment will be described.

  The process of generating the encoded video data to be decoded by the video decoding device of the fourth embodiment is the same as the process of the video encoding device 200 shown in FIG. 1 described above.

  FIG. 17 is a block diagram illustrating a configuration of a video decoding device 400 according to the fourth embodiment of the present invention. The same reference numerals are used for blocks common to the moving picture coding apparatus 200 and the moving picture decoding apparatus 100 of the first embodiment. As shown in FIG. 17, a moving image decoding device 400 according to the fourth embodiment includes a secondary storage device 206, a code string reading unit 101, a bit plane decoding unit 102, an inverse discrete wavelet transform unit 104, a buffer 1601, and a display. An interface 1602, a non-decoded bit plane determining unit 1702, and a buffer status monitoring unit 1701 are provided, and are connected to the display 1603. As shown in the figure, the moving picture data output unit 105 of the moving picture decoding apparatus 100 according to the fourth embodiment of the present invention comprises a buffer 1601 and a display interface 1602 as shown in FIG. The decoding processing time measuring unit 106 is replaced with a buffer state monitoring unit 1701 and the non-decoded bit plane determining unit 107 is replaced with a non-decoded bit plane determining unit 1702.

  Hereinafter, the operation procedure of the video decoding device 400 according to the fourth embodiment will be described with reference to FIG. The operations of the code string reading unit 101, the bit plane decoding unit 102, and the inverse discrete wavelet transform unit 104 are the same as the operations of the video decoding device 100 according to the first embodiment. The basic operations of the buffer 1601 and the display interface 1602 are also the same as those described in the first embodiment as a specific example in which the moving image data output unit 105 is configured by the buffer and the display interface. However, in the moving picture decoding apparatus 400 according to the fourth embodiment, in order to avoid a state in which data to be originally displayed at a predetermined time is not prepared in the buffer 1601, a decoding frame is set after a lapse of time mT from the start of decoding. Display of data is started. T is the target decoding processing time for one frame, as in the above embodiment, and m is any positive integer. That is, after a lapse of time mT from the start of the decoding process, the display interface 1602 sequentially retrieves the restored frame data from the buffer 1601 at fixed time intervals and displays the data on the display. Assuming that the time for preparing one frame of decoded data in the buffer 1601 is negligible, other than the decoding processing time, the target decoding time T is 1/30 second for a moving image of 30 frames per second. The interval at which the restored frame data is extracted from the buffer 1601 is also 1/30 second.

  FIG. 18 shows the operation of the main video decoding apparatus 400 in a time series from the start of decoding. As shown in FIG. 18, the first frame is displayed when a time mT has elapsed from the decoding start time (time 0), and the frames are displayed one after another at time T intervals. The time difference ΔT used as an index for controlling the decoding range in the above-described first to third embodiments is obtained by subtracting the time when a certain frame n of interest is decoded from the time nT, and this value is an upper limit. If the value is larger than the value Uq, the decoding range is widened to increase the decoding quality. Conversely, if the value is smaller than the lower limit Lq, the decoding range is narrowed to lower the decoding quality and shorten the decoding processing time per frame. However, the video decoding apparatus according to the fourth embodiment performs similar control by focusing on the number of decoded frame data stored in the buffer 1601.

  In FIG. 18, when the time from the start of decoding to the end of decoding of frame n is d, (n-1) T <d <nT, that is, 0 <ΔT <T, The number of decoded frames stored in the buffer 1601 is m. When ΔT> T, at the end of frame n decoding, m + 1 or more decoded frame data is stored in the buffer 1601. Conversely, when nT <d <(n + 1) T, that is, when −T <ΔT <0, the number of decoded frames stored in the buffer 1601 at the end of frame n decoding is m−1. If ΔT <−T, m−2 or less. If the number of decoded frames stored in the buffer 1601 at the end of the decoding of the frame n is m + 1 or more, the decoding range is widened to improve the decoding quality, and conversely, if m-2 or less, the decoding quality is reduced. If control is performed to shorten the decoding processing time per frame, the same operation as controlling the decoding range based on ΔT with Uq = T and Lq = −T can be performed. Note that storage of decoded frame data in the buffer 1601 by the inverse discrete wavelet transform unit 104 and reading of decoded frame data by the display interface 1602 do not occur at the same time.

  The buffer state monitoring unit 1701 obtains the number p of decoded frames stored in the buffer 1601 at each decoding end time of each frame, and outputs the number to the non-decoded bit plane determining unit 1702.

  The non-decoding bit plane determining unit 1702 determines a non-decoding bit plane for each subband based on the number of stored frames p output from the buffer status monitoring unit 1701. The non-decoding bit plane determining unit 1702 includes therein a variable Q (hereinafter, referred to as a “Q factor”) serving as an index value for determining the number of non-decoding bit planes, and a non-decoding bit of each subband in each Q factor. A table indicating the number of planes and a parameter m for determining a delay time from the start of decoding to the start of display are held. FIG. 8 shows an example of a table indicating the correspondence between the Q factor and the number of non-decoded bit planes of each subband.

  FIG. 19 is a flowchart showing a flow of a decoding process of encoded video data by the video decoding device 400. As shown in FIG. 19, first, the Q factor and the time difference ΔT are reset to 0 at the start of decoding of the encoded video data, that is, before the decoding of the encoded data of frame 1 is started (step S1901).

  Next, the non-decoding bit plane determining unit 1702 reads the number of non-decoding bit planes of each subband from the table based on the Q factor, and sets the number in the bit plane decoding unit 102 (step S1902).

  Subsequently, one frame is decoded by the processing of the inverse discrete wavelet transform unit 104 from the code string reading unit 101, and the frame data is stored in the buffer 1601 (step S1903).

  The buffer monitoring unit 1701 obtains the number p of frame data stored in the buffer 1601 and passes it to the non-decoded bit plane determining unit 1702 (step S1904).

  It is determined whether or not the time mT has elapsed from the start of the decoding process, that is, whether or not the display of the decoded frame data has been started (step S1905). Then, the process returns to the next frame. If the display has been started (YES), the Q factor is updated according to the value of p (step S1906). If p is greater than m (that is, if p> m), 1 is subtracted from Q to reduce the value. The case where p> m is satisfied when the total sum of the decoding time actually taken with respect to the total sum of the target time is small. cut back. Conversely, if p is smaller than m-1 (that is, if p <m-1), 1 is added to Q to increase the value. The case where p <m-1 is satisfied is when the total sum of the decoding time actually taken with respect to the total sum of the target time is large, and by increasing the value to shorten the decoding time of one frame, the non-decoded bits are reduced. Increase the number of planes. However, the range of the value of Q is from 0 to 9, and is 0 when the value is smaller than 0 and 9 when the value is larger than 9 by the above-described update processing. If p = m, the sum of the decoding times actually taken with respect to the sum of the target times is within a good range, so the value of Q is left unchanged.

  In the next step S1907, it is determined whether or not the decoded frame is the last frame. If it is not the last frame (NO), the process returns to step S702 to decode the next frame, and the last frame is decoded. Is satisfied (YES), the decoding process of the encoded video data ends.

  Even when the decoding process is completed, the display interface 1602 continues extracting frame data stored in the buffer 1601 for the time mT.

  As described above, the moving picture decoding apparatus 400 according to the fourth embodiment employs a buffer as a technique equivalent to adjusting the decoding range from the cumulative value of the difference between the time required for decoding one frame and the target decoding time. The decoding range was adjusted according to the number of decoding frames held in. With such a configuration, even when it is difficult to measure the processing time of each frame, by changing the number of non-decoded bitplanes according to the number of decoded frames held in the buffer, the reproduced image can be visually recognized. Can be controlled as much as possible, and the decoding processing time can be controlled.

<Other embodiments>
The present invention is not limited to the embodiments described above. For example, in the above-described first to third embodiments, bit-plane encoding is performed in units of sub-bands. However, sub-bands may be divided into blocks, and bit-plane encoding may be performed for each block. Further, one bit plane may be encoded by a plurality of passes.

  Further, although an example using MQ-Coder as a method of binary arithmetic coding has been described, the present invention is not limited to the above-described embodiment, and for example, an arithmetic coding method other than MQ-Coder, such as QM-Coder, may be used. It may be applied, or any other binary encoding method may be applied as long as it is a method suitable for encoding a multi-context information source.

  Further, the filter for subband decomposition is not limited to the above-described embodiment, and another filter such as a real number type 9 × 7 filter may be used. Further, the number of times of application is not limited to the above embodiment. In the above-described embodiment, the same number of one-dimensional discrete wavelet transforms are performed in the horizontal direction and the vertical direction.

  Further, the structure of the encoded video data is not limited to the above-described embodiment, and the order of the code string, the storage format of the additional information, and the like may be changed. For example, the present invention is suitable for the case where JPEG2000 defined in ISO / IEC15444-1 is used as a frame data encoding method, and encoded data described in the JPEG2000 standard or Motion defined in Part 3 of the standard. It may be JPEG2000 encoded data.

  Also, the measurement of the decoding processing time is not limited to the above-described embodiment. For example, processing such as wavelet transform is estimated to be a substantially constant processing time, and only the time required for bit plane decoding is measured. Alternatively, the processing time may be measured in units of a plurality of frames, and the non-decoding part may be controlled.

  Further, even if the present invention is applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, a printer, and the like), an apparatus including one device (for example, a copier, a facsimile device, and the like) May be applied.

  Further, an object of the present invention is to supply a storage medium (or a recording medium) in which a program code of software for realizing the functions of the above-described embodiments is recorded to a system or an apparatus, and to provide a computer (or a CPU or Needless to say, the present invention can also be achieved by an MPU) reading and executing a program code stored in a storage medium. In this case, the program code itself read from the storage medium realizes the function of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. When the computer executes the readout program code, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a part or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing. Here, examples of the storage medium for storing the program code include a flexible disk, a hard disk, a ROM, a RAM, a magnetic tape, a nonvolatile memory card, a CD-ROM, a CD-R, a DVD, an optical disk, a magneto-optical disk, and an MO. Can be considered.

  Further, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is executed based on the instruction of the program code. It goes without saying that the CPU included in the expansion card or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

  When the present invention is applied to the storage medium, the storage medium stores program codes corresponding to the flowcharts described above.

FIG. 1 is a block diagram illustrating a configuration of a video encoding device according to an embodiment of the present invention. FIG. 3 is a diagram for explaining subbands of an encoding target image processed by two-dimensional discrete wavelet transform. FIG. 11 is a diagram illustrating seven subbands obtained by two two-dimensional discrete wavelet transforms. 35 is a flowchart for describing a processing procedure of encoding a subband Sb in a bit plane encoding unit. FIG. 3 is a diagram illustrating a detailed structure of a code string corresponding to one frame of encoded moving image data generated by the code string forming unit. FIG. 3 is a diagram illustrating an example of a code string of each frame stored in a secondary storage device. It is a block diagram showing the composition of the video decoding device concerning a 1st, 3rd embodiment of the present invention. FIG. 7 is a diagram illustrating a relationship between a Q factor and a non-decoding bit plane ND (Sb) of each subband or a Q factor and the number of non-decoding paths NDP (Sb) of each subband according to the first embodiment of the present invention. is there. It is a flowchart which shows the flow of a process of the video decoding device which concerns on the 1st, 3rd embodiment of this invention. It is a block diagram showing the composition of the video decoding device concerning a 2nd embodiment of the present invention. FIG. 14 is a diagram illustrating an example of a table held in a non-decoding bit plane determination unit according to the second embodiment of the present invention. It is a flow chart which shows a flow of processing of a video decoding device concerning a 2nd embodiment of the present invention. It is a flowchart which shows the flow of update processing of ND (Sb) table and SI in step S1107. FIG. 13 is a diagram illustrating a correspondence between a subband index SI and a subband according to the second embodiment of the present invention. FIG. 15 is a diagram illustrating a structure of encoded video data for one frame to be decoded by the video decoding device according to the third embodiment of the present invention. FIG. 3 is a block diagram illustrating a specific configuration example of a moving image data output unit. It is a block diagram showing the composition of the video decoding device concerning a 4th embodiment of the present invention. 15 is a time chart illustrating a time-series operation of the video decoding device according to the fourth embodiment of the present invention. It is a flow chart which shows a flow of processing of a video decoding device concerning a 4th embodiment of the present invention.

Explanation of reference numerals

REFERENCE SIGNS LIST 100 moving image decoding device 101 code string reading unit 102 bit plane decoding unit 104 inverse discrete wavelet transform unit 105 moving image data output unit 106 decoding processing time measuring unit 107 non-decoding bit plane determining unit 200 moving image decoding device 201 moving image data input Unit 202 discrete wavelet transform unit 203 coefficient quantization unit 204 bit plane encoding unit 205 code string forming unit 206 secondary storage device 207 signal line 300 video decoding device 400 video decoding device 901 coefficient inverse quantization unit 902 inverse discrete wavelet Conversion section 903 Non-decoding bit plane determining section 904 Code string reading section 1601 Buffer 1602 Display interface 1603 Display 1701 Buffer state monitoring section 1702 Non-decoding bit plane determining section

Claims (19)

Each frame of the moving image data is decomposed into a plurality of sub-bands, and the moving image generated by encoding the coefficients of the sub-bands from a higher-order bit to a lower-order bit for each predetermined unit in bit plane or sub-bit plane units A moving image decoding device that decodes encoded data,
Decoding processing time information obtaining means for obtaining information for determining the magnitude of the difference between the time allocated to the decoding processing of the video encoded data of the predetermined unit and the time required for the actual decoding processing,
Based on the information obtained by the decoding processing time information obtaining means, a non-decoding bit plane, or a non-decoding bit plane determining means for determining a sub-bit plane,
Bit plane determined by the non-decoding bit plane determining means, or bit plane decoding means for restoring sub-band coefficients in the predetermined unit from encoded data other than sub-bit planes,
A subband synthesizing unit that synthesizes coefficients of the plurality of subbands obtained by the bit plane decoding unit and generates frame data.   The decoding processing time information obtaining means obtains the decoding processing time required for the decoding processing of the encoded video data, and the non-decoding bit plane determining means obtains the decoding processing time based on the decoding processing time obtained from the decoding processing time information obtaining means. The moving picture decoding apparatus according to claim 1, wherein a bit plane or a sub bit plane not to be decoded is determined. Further provided is a decoded frame data storage means for storing the decoded frame data,
The decoding processing time information obtaining means obtains the number of frames stored in the decoded frame data storage means, and the non-decoding bit plane determining means does not perform decoding based on the number of frames obtained from the decoding processing time information obtaining means. The moving picture decoding apparatus according to claim 1, wherein a bit plane or a sub-bit plane is determined.   The non-decoding bit plane determining unit holds a parameter indicating image quality, adjusts the parameter based on information obtained by the decoding processing time information obtaining unit, and sets a non-decoding bit plane or sub bit of each subband by the parameter. The moving picture decoding apparatus according to claim 1, wherein a plane is determined.   The non-decoding bit plane determining means holds a table for storing the number of bit planes not to be decoded for each sub-band, or the number of sub-bit planes, and is stored in the table in accordance with information obtained by the decoding processing time information obtaining means. 4. The moving picture decoding apparatus according to claim 1, wherein the number of bit planes or sub bit planes not decoded is increased or decreased.   The non-decoding bit plane determining means calculates a difference between a time allocated to the predetermined unit of moving image encoded data decoding processing and a decoding processing time obtained by the decoding processing information obtaining means, and calculates the calculated difference. 3. The moving picture decoding apparatus according to claim 2, wherein a bit plane or a sub-bit plane not to be decoded is determined based on a value obtained by accumulating.   The sub-band decomposition for generating the moving image encoded data is performed by a two-dimensional discrete wavelet transform, and the sub-band synthesizing unit synthesizes the frame data using a two-dimensional inverse discrete wavelet transform. The moving picture decoding device according to claim 1.   The moving picture decoding apparatus according to any one of claims 1 to 7, wherein the predetermined unit is a frame or a block obtained by dividing the frame into a plurality. Each frame of the moving image data is decomposed into a plurality of sub-bands, and the moving image generated by encoding the coefficients of the sub-bands from a higher-order bit to a lower-order bit for each predetermined unit in bit plane or sub-bit plane units A moving image decoding method for decoding encoded data,
A decoding processing time information obtaining step of obtaining information for determining the magnitude of a difference between the time allocated to the decoding processing of the moving image encoded data of the predetermined unit and the time required for the actual decoding processing,
Based on the information obtained by the decoding processing time information obtaining step, a non-decoding bit plane, or a non-decoding bit plane determining step of determining a sub-bit plane,
A bit plane determined by the non-decoding bit plane determining step, or a bit plane decoding step of restoring subband coefficients in the predetermined unit from encoded data other than the sub bit plane,
A subband synthesizing step of synthesizing coefficients of the plurality of subbands obtained in the bitplane decoding step to generate frame data.   In the decoding processing time information obtaining step, a decoding processing time required for decoding the moving picture encoded data is obtained, and in the non-decoding bit plane determining step, based on the decoding processing time obtained in the decoding processing time information obtaining step. The moving picture decoding method according to claim 9, wherein a bit plane or a sub-bit plane not to be decoded is determined. Further comprising a decoded frame data storing step for storing the decoded frame data,
In the decoding processing time information obtaining step, the number of frames stored in the decoded frame data storing step is obtained, and in the non-decoding bit plane determining step, decoding is not performed based on the number of frames obtained in the decoding processing time information obtaining step. The moving picture decoding method according to claim 9, wherein a bit plane or a sub-bit plane is determined.   In the non-decoding bit plane determining step, a parameter indicating image quality is managed, and the parameter is adjusted based on the information obtained in the decoding processing time information obtaining step. The moving picture decoding method according to claim 9, wherein a bit plane is determined.   In the non-decoding bit plane determining step, a table for storing the number of bit planes or sub bit planes that are not decoded for each subband is managed, and stored in the table according to the information obtained in the decoding processing time information obtaining step. 12. The moving picture decoding method according to claim 9, wherein the number of bit planes or sub-bit planes not decoded is increased or decreased.   In the non-decoding bit plane determining step, the difference between the time allocated to the decoding processing of the predetermined unit of moving image encoded data and the decoding processing time obtained in the decoding processing time information obtaining step is calculated and calculated. The moving picture decoding method according to claim 10, wherein a bit plane or a sub-bit plane not to be decoded is determined based on a value obtained by accumulating the differences.   The sub-band decomposition for generating the moving image encoded data is performed by two-dimensional discrete wavelet transform, and the sub-band synthesizing step synthesizes frame data using two-dimensional inverse discrete wavelet transform. The moving picture decoding method according to claim 9.   16. The video decoding method according to claim 9, wherein the predetermined unit is a frame or a block obtained by dividing the frame into a plurality.   9. A program executable by an information processing apparatus, the information processing apparatus executing the program functioning as the moving picture decoding apparatus according to claim 1.   A program executable by an information processing apparatus, comprising a program code for implementing the moving picture decoding method according to claim 9.   A storage medium readable by an information processing apparatus, storing the program according to claim 17.

Images