JP2004242286A - Motion picture encoding method and motion picture decoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a motion picture encoding method and motion picture decoding method or the like with which a picture to be decoded for variable speed reproduction can be easily specified and encoding and decoding suited to variable speed reproduction can be performed. <P>SOLUTION: A motion picture encoding apparatus 1 comprises a map creation part 113 for creating information Map for variable speed reproduction that is information required for variable speed reproduction in accordance with a picture type Ptype, a variable length encoding part 112 for encoding and disposing the information Map for variable speed reproduction in an encoded stream Str, a detection part 114 for detecting whether or not to encode a picture parameter set PPS that a picture of an encoding targets refers to, and a common information adding part 115 for adding the picture parameter set PPS that is detected to be encoded by the detection part 114, to the encoding target picture. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

  The present invention relates to a moving image encoding method for encoding a moving image so that the moving image can be reproduced at a variable speed, a moving image decoding method for decoding a stream encoded as such, and a stream thereof.

  In recent years, the multimedia era, in which audio, images, and other pixel values are integrated, has been approached, and the traditional information media, that is, means for transmitting information such as newspapers, magazines, televisions, radios, and telephones to humans, has It has been taken up as an object. Generally, multimedia means not only characters, but also figures, sounds, and especially images, etc., that are simultaneously associated with each other. Is an essential condition.

  However, when the amount of information of each information medium is estimated as a digital information amount, the amount of information per character is 1-2 bytes in the case of characters, whereas 64 Kbits per second in the case of voice (telephone quality). In addition, for a moving image, an information amount of 100 Mbits per second (current television reception quality) or more is required, and it is not realistic to handle the vast amount of information in the above-mentioned information medium as it is in a digital format. For example, videophones have already been put into practical use by the Integrated Services Digital Network (ISDN), which has a transmission speed of 64 Kbit / s to 1.5 Mbits / s. It is impossible.

Therefore, it is necessary to use information compression technology. For example, in the case of videophones, videos based on the H.261 and H.263 standards recommended by the ITU-T (International Telecommunication Union Telecommunication Standardization Sector) Compression technology is used. Further, according to the information compression technology of the MPEG-1 standard, it is possible to store image information together with audio information in a normal music CD (compact disc).
Here, MPEG (Moving Picture Experts Group) is an international standard for moving image signal compression standardized by ISO / IEC (International Organization for Standardization), and MPEG-1 is used for moving image signals. It is a standard that compresses information of television signals to 5 Mbps, that is, to about 1/100. In addition, since the target quality is set to a medium quality that can be realized at a transmission speed of about 1.5 Mbps mainly in the MPEG-1 standard, MPEG-2 standardized to meet the demand for higher image quality. Then, the TV broadcast quality of the moving image signal is realized at 2 to 15 Mbps.

  Furthermore, at present, the working group (ISO / IEC JTC1 / SC29 / WG11), which has been working on the standardization of MPEG-1 and MPEG-2, has achieved compression ratios exceeding those of MPEG-1 and MPEG-2, and furthermore, on an object-by-object basis. MPEG-4, which enables encoding, decoding, and operation, and realizes new functions required in the multimedia age, has been standardized. MPEG-4 was initially aimed at standardizing a low bit rate coding method, but is now expanded to more general-purpose coding including high bit rates including interlaced images. In addition, ISO / IEC and ITU-T are currently working together to standardize MPEG-4 AVC and ITU H.264 as next-generation image coding systems with higher compression ratios. As of May 2002, a so-called committee draft (CD) has been issued as a next-generation image coding method (for example, see Non-Patent Document 1).

  Generally, in coding of a moving image, the amount of information is compressed by reducing redundancy in the time direction and the space direction. Therefore, in inter-picture predictive coding for the purpose of reducing temporal redundancy, motion detection and a predicted image are created in block units with reference to a forward or backward picture, and the obtained predicted image and coding are performed. Encoding is performed on the difference value from the target picture. Here, a picture is a term representing one screen, and means a frame in a progressive image and a frame or a field in an interlaced image. Here, an interlaced image is an image in which one frame is composed of two fields at different times. In encoding and decoding of an interlaced image, one frame may be processed as a frame, processed as two fields, or processed as a frame structure or a field structure for each block in the frame. it can.

  A picture that does not have a reference picture and performs intra-picture prediction coding is called an I picture. A picture that performs inter-picture predictive encoding by referring to only one picture is called a P picture. A picture that can perform inter-picture predictive encoding by simultaneously referring to two pictures is called a B picture. The B picture can refer to two pictures as an arbitrary combination of display time from the front or the rear. The reference picture (reference picture) can be specified for each block, which is a basic unit of encoding and decoding, but the reference picture described earlier in the encoded bit stream is replaced with the first reference picture. , The one described later is distinguished as the second reference picture. However, as a condition for encoding and decoding these pictures, the picture to be referred to must be already encoded and decoded.

  Motion-compensated inter-picture prediction coding is used for coding a P picture or a B picture. The motion compensated inter-picture predictive coding is a coding method in which motion compensation is applied to the inter-picture predictive coding. Motion compensation refers to detecting a motion amount (hereinafter, referred to as a motion vector) of each part in a picture and performing prediction in consideration of the motion amount, instead of simply predicting from a pixel value of a reference picture. Is a method for improving the prediction accuracy and reducing the amount of data. For example, a data amount is reduced by detecting a motion vector of a current picture to be coded and coding a prediction residual between a predicted value shifted by the motion vector and the current picture to be coded. In the case of this method, since information of a motion vector is required at the time of decoding, the motion vector is also encoded and recorded or transmitted.

  The motion vector is detected in macroblock or block units. Specifically, the macroblock or block on the encoding target picture side is fixed, and the macroblock or block on the reference picture side is moved within the search range. , The motion vector is detected by finding the position of the reference block most similar to the reference macroblock or block.

  FIG. 19 is a diagram showing the configuration of a conventional MPEG2 stream, in which (a) shows the flow of pictures and (b) shows the hierarchical structure of the stream. As shown in FIGS. 19A and 19B, the MPEG2 stream has the following hierarchical structure. The stream (Stream) is composed of a plurality of Group Of Pictures (GOPs), and by using this as a basic unit of the encoding process, editing of moving images and random access are possible. . The group of pictures is composed of a plurality of pictures, and each picture includes an I picture, a P picture, or a B picture. The stream, GOP, and picture further include a synchronization signal (sync) indicating a break of each unit and a header (header) which is data common to the unit.

  FIG. 20 is a diagram showing a hierarchical structure of another conventional stream. This stream supports JVT (H.264 / MPEG-4 AVC), which is currently being jointly standardized by ITU-T and ISO / IEC. In JVT, there is no concept of a header, and common data is arranged at the head of a stream under the name of a parameter set PS. Also, if there is no concept equivalent to a GOP, but data is divided into special picture units that can be decoded without depending on other pictures, a random accessible unit equivalent to the GOP can be configured. Let's call it RAU.

  The parameter set PS includes a picture parameter set PPS, which is data corresponding to the header of each picture, and a sequence parameter set SPS, which corresponds to a GOP of MPEG-2 or a header in sequence units. Each picture is provided with an identifier indicating which of the plurality of candidates for the picture parameter set PPS and the sequence parameter set SPS is to be referred to. That is, the picture parameter set PPS and the sequence parameter set SPS encode a plurality of sets only once, and in each picture, which identifier is used to indicate which of the sets is to be referred to, each picture can be encoded as in MPEG-2. The compression ratio is improved by eliminating the need to encode the header (parameter set) having the same value many times for each time.

The picture number PN is an identification number for identifying a picture. The sequence parameter set SPS includes the maximum number of pictures that can be referred to, the image size, and the like. The picture parameter set PPS includes the type of variable-length coding (switching between Huffman coding and arithmetic coding) and the quantization step. , The number of reference pictures, and the like.
FIG. 21 is a block diagram showing a configuration of a moving picture coding apparatus that realizes a conventional moving picture coding method.

  The video encoding device 3 is a device that outputs an encoded stream Str obtained by compression-encoding an input image signal Vin and converting it into a bit stream such as variable-length encoding. The motion detection unit 101 and the motion compensation unit 102 , Subtractor 103, orthogonal transformer 104, quantizer 105, inverse quantizer 106, inverse orthogonal transformer 107, adder 108, picture memory 109, switch 110, predictive structure determiner 111, and variable length encoder 301 is provided.

  The image signal Vin is input to the subtraction unit 103 and the motion detection unit 101. The subtraction unit 103 calculates a difference value between the input image signal Vin and the predicted image, and outputs the difference value to the orthogonal transformation unit 104. The orthogonal transform unit 104 converts the difference value into a frequency coefficient, and outputs the frequency coefficient to the quantization unit 105. The quantization unit 105 quantizes the input frequency coefficient and outputs a quantized value Qcoef to the variable length coding unit 301.

  The inverse quantization unit 106 inversely quantizes the quantized value Qcoef to restore the frequency coefficient, and outputs the frequency coefficient to the inverse orthogonal transform unit 107. The inverse orthogonal transform unit 107 performs an inverse frequency transform from the frequency coefficient to the pixel difference value, and outputs the result to the adding unit 108. The addition unit 108 adds the pixel difference value and the predicted image output from the motion compensation unit 102 to obtain a decoded image. The switch 110 is turned on when the storage of the decoded image is instructed, and the decoded image is stored in the picture memory 109.

  On the other hand, the motion detection unit 101 to which the image signal Vin is input in units of macroblocks sets the decoded image stored in the picture memory 109 as a search target, detects an image area closest to the input image signal, and determines the position. The pointing motion vector MV is determined. Motion vector detection is performed for each block obtained by further dividing a macroblock. At this time, since a plurality of pictures can be used as reference pictures, an identification number (reference index Index) for specifying a picture to be referenced is required for each block. By using the reference index Index, it is possible to specify a reference picture by associating it with the picture number of each picture in the picture memory 109.

The motion compensation unit 102 extracts an optimal image area for a predicted image from the decoded image stored in the picture memory 109, using the motion vector MV detected by the above process and the reference index Index.
If the target picture is a random access unit start picture RAUin indicating the start position of the random access unit RAU, the prediction structure determination unit 111 codes the target picture as a special picture that can be randomly accessed (intra-screen coding). To the motion detection unit 101 and the motion compensation unit 102 according to the picture type Ptype. Further, the prediction structure determination unit 111 outputs the picture type Ptype to the variable length coding unit 301.

The variable-length coding unit 301 performs variable-length coding on the quantization value Qcoef, the reference index Index, the picture type Ptype, and the motion vector MV to obtain a coded stream Str.
FIG. 22 is a block diagram showing a configuration of a moving picture decoding apparatus that realizes a conventional moving picture decoding method. In the figure, devices that perform the same operations as those of the moving picture coding apparatus that realizes the conventional moving picture coding method of FIG. 21 are denoted by the same reference numerals, and description thereof is omitted.

The variable length decoding unit 401 decodes the coded stream Str, and outputs a quantized value Qcoef, a reference index Index, a picture type Ptype, and a motion vector MV. The quantization value Qcoef, the reference index Index, and the motion vector MV are input to the picture memory 208, the motion compensation unit 204, and the inverse quantization unit 205 to perform decoding processing. It is the same as the chemical conversion device.
Text of Committee Draft of Joint Video Specification (ITU-T Rec.H.264 | ISO / IEC 14496-10 AVC) 0.4 'Overview of the syntax' 0.4.1 'Temporal processing' 8.2.2 'Parameter set decoding'

  However, the random access unit RAU can decode only the picture of the unit, but the conventional JVT encoding method and stream can store information for variable speed playback that is important in storage devices such as VTRs and disk recorders. I can't get it. This is because JVT introduced a very flexible inter-picture prediction structure in order to greatly improve the coding efficiency (compression rate).

  FIG. 23 is a schematic diagram showing an example of a picture reference relationship. FIG. 23A shows a prediction structure between pictures used in MPEG-2. In the figure, the hatched pictures are pictures referred to by other pictures. In MPEG-2, P pictures (P4, P7) can be predictively coded with reference to only one I picture or P picture immediately before the display time. The B pictures (B1, B2, B3, B5, B6) can be predictively coded with reference to one I picture or one P picture immediately before and one immediately after the display time. Furthermore, the order in which the pictures are arranged in the stream is also determined. The I picture and the P picture are arranged in the order of the display time, and the B picture is arranged immediately after the I picture or the P picture displayed immediately after. Therefore, decoding is performed in three ways: 1) decoding all pictures, 2) decoding only I and P picture streams and displaying only I and P pictures, and 3) decoding and displaying only I picture streams. Therefore, three modes, i.e., 1) normal reproduction, 2) medium speed reproduction, and 3) high speed reproduction, can be easily realized.

  In JVT, prediction that refers to a B picture from a B picture is also possible. FIG. 23B shows an example of JVT prediction, in which B pictures (B1, B3) refer to B pictures (B2). In this example, 1) decoding all pictures, 2) decoding and displaying only the referenced streams of I-pictures, P-pictures, and B-pictures, 3) decoding only the I-picture and P-picture streams, and Only the P picture is displayed, and 4) only the I picture stream is decoded and displayed.

  However, in the JVT, it is also possible to refer to a B picture from a P picture, and as shown in FIG. 24, a P picture (P7) can also refer to a B picture (B2). In this case, since the P picture (P7) cannot be decoded unless the B picture (B2) has been decoded, 1) all pictures are decoded, and 2) I pictures, P pictures, and B pictures are referred to. 3) decoding and displaying only the I-picture stream, and 3) decoding and displaying only the I-picture stream.

  As described above, since a very flexible prediction structure is allowed in JVT, it is unknown what kind of variable speed playback is possible unless the actual prediction structure is known. Therefore, only two ways of 1) decoding all pictures and 2) decoding and displaying only a stream of I pictures can be realized irrespective of the prediction structure from the examples of FIGS. This means that there are too few speed options that can be achieved compared to the variable speed playback that can be achieved with MPEG-2.

  The present invention has been made in view of the above circumstances, and can easily specify a picture that needs to be decoded for variable-speed playback, and can perform encoding and decoding suitable for variable-speed playback. It is an object of the present invention to provide a moving image encoding method, a moving image decoding method, and the like that can be performed.

  In order to achieve the above object, a moving picture coding method according to the present invention is a moving picture coding method that generates a coded stream by coding a moving picture signal in picture units, And a coding step of coding the variable-speed playback information to add the variable-speed playback information to the coded stream.

This makes it possible to specify a desired picture to be subjected to variable-speed playback based on the information for variable-speed playback at the time of variable-speed playback, and to decode only the specified picture to decode unnecessary pictures. And the variable speed reproduction can be easily performed.
Here, in the information creating step, the decoding target picture in the random access unit is a random access unit composed of a plurality of pictures, with reference to only the picture in the random access unit. The variable speed reproduction information may be created for each possible random access unit.

As a result, even if the structure of the reference relation of pictures differs in random access unit units, it is possible to specify a desired picture to be subjected to variable speed reproduction based on variable speed reproduction information during variable speed reproduction.
Further, the moving picture coding method further includes a detecting step of detecting the necessity of coding of the common information referred to by the picture to be subjected to the variable speed reproduction, and a detecting step of coding the common information by the detecting step. A common information adding step of adding the common information to the picture whose necessity is detected.

Thus, for example, even if the common information is already added to a picture played back at a speed lower than N times, if it is needed for a picture played back at N times speed, the common information is also applied to a picture played back at N times speed. Since information can always be added, it is possible to avoid a situation where there is no common information to be referred to at the time of variable speed reproduction.
Further, the moving picture decoding method according to the present invention is a moving picture decoding method for decoding an encoded stream in units of pictures, and includes information for variable speed reproduction for specifying a picture to be subjected to variable speed reproduction. And a decoding step of decoding the variable-speed playback information, and identifying and decoding a picture to be subjected to variable-speed playback based on the variable-speed playback information. It is characterized by.

This makes it possible to specify a desired picture to be subjected to variable-speed playback based on the information for variable-speed playback at the time of variable-speed playback, and to decode only the specified picture to decode unnecessary pictures. And the variable speed reproduction can be easily performed.
Here, in the information extracting step, a decoding target picture in the random access unit may be decoded with reference to only a picture in the random access unit, which is a random access unit including a plurality of pictures. The variable speed reproduction information may be extracted for each possible random access unit.

As a result, even if the structure of the reference relation of pictures differs in random access unit units, it is possible to specify a desired picture to be subjected to variable speed reproduction based on variable speed reproduction information during variable speed reproduction.
In addition, the moving picture decoding method may further include storing, in variable-speed playback, a picture referred to by another picture in a picture memory, even if the picture is not a target of the variable-speed playback. A memory control step for controlling may be included.

This makes it possible to prevent a picture designated as a reference picture of a current picture to be decoded from being different from a reference picture at the time of encoding even when a picture to be relatively referenced is designated.
It should be noted that the present invention can be realized not only as such a moving picture coding method and a moving picture decoding method, but also includes characteristic steps included in such a moving picture coding method and a moving picture decoding method. The present invention can also be realized as a moving image encoding device and a moving image decoding device provided as means. Further, these steps may be realized as a program for causing a computer to execute the steps, or as an encoded stream encoded by the moving image encoding method. Needless to say, such a program and an encoded stream can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

  As described above, according to the moving picture coding method and the moving picture decoding method according to the present invention, pictures that need to be decoded for variable-speed playback in the random access unit RAU can be easily specified. Encoding and decoding suitable for variable speed reproduction can be easily realized, and its practical value is high.

Variable speed playback becomes difficult because the JVT prediction structure is too flexible, but before decoding the random access unit RAU, we know what prediction structure is used in the picture of the random access unit RAU If this is possible, variable speed reproduction of two or more speeds can be realized as shown in the examples of FIGS.
Hereinafter, embodiments of the present invention will be described with reference to FIGS.

  Note that the random access unit RAU used in the description of the embodiment does not necessarily need to be a special unit of the JVT, and the parameter set PS is arranged for each random access unit RAU. It may be a set of pictures that start.

(Embodiment 1)
FIG. 1 is a block diagram showing the configuration of an embodiment of a moving picture coding apparatus using the moving picture coding method according to the present invention. Note that the same reference numerals are given to devices that perform the same operations as the respective units of the conventional video encoding device 3 shown in FIG. 21 and description thereof is omitted.

  The moving image encoding device 1 is a device that outputs an encoded stream Str obtained by compression-encoding an input image signal Vin and converting it into a bit stream such as variable-length encoding. The motion detecting unit 101 and the motion compensating unit 102 , Subtractor 103, orthogonal transformer 104, quantizer 105, inverse quantizer 106, inverse orthogonal transformer 107, adder 108, picture memory 109, switch 110, predictive structure determiner 111, variable length encoder 112 , A map creating unit 113, a detecting unit 114, and a common information adding unit 115.

  The map creator 113 generates a variable-speed reproduction information Map (for example, a variable-speed reproduction map RAM, a variable-speed reproduction map table RAMTBL, and a variable-speed reproduction map identifier, which will be described later), which are information necessary for variable-speed reproduction corresponding to the picture type Ptype. RAMID) and outputs it to the variable-length encoding unit 112. The variable-length encoding unit 112 encodes and arranges the variable-speed reproduction information Map in the encoded stream Str.

  When the picture to be coded is a picture to be reproduced (decoded) at N times speed or higher, the detection unit 114 determines that the picture parameter set PPS, which is the common information referred to by the current picture, is a picture to be reproduced at N times speed or higher. The necessity of encoding the picture parameter set PPS is detected by determining whether or not the encoding is already performed. The common information adding unit 115 adds the picture parameter set PPS to the picture for which the necessity of encoding has been detected by the detecting unit 114.

FIG. 2 is a configuration diagram of a stream according to the first embodiment. The difference from the conventional stream configuration diagram in FIG. 20 is that a variable speed reproduction map RAM is arranged in the random access unit RAU.
FIG. 2A shows an example of the structure of the random access unit RAU. In this structure example, a variable speed reproduction map RAM is arranged before a picture in the random access unit RAU, and information on which picture stream can be decoded and reproduced at a desired variable speed is stored in the variable speed reproduction map RAM. It has been described. The moving picture decoding apparatus decodes only the pictures necessary for the desired variable-speed playback in accordance with the information described in the variable-speed playback map RAM, so that variable-speed playback can be easily performed without decoding unnecessary pictures. Will be possible.

  FIG. 2B shows an example of the variable speed reproduction map RAM. For each picture number PN that is an identifier of a picture in the random access unit RAU, the variable speed playback map RAM describes how many times the playback speed is required to decode the picture. That is, information (Speed) indicating how many times the picture is reproduced at a required speed and decoded is described at a position after each picture number PN. By doing so, it is possible to easily know which picture needs to be decoded at a desired reproduction speed. Note that the picture number PN may be arranged collectively at the head of the random access unit RAU, and thereafter information (Speed) indicating how many times speed reproduction is required for decoding may be arranged collectively.

FIG. 2C shows another example of the variable speed reproduction map RAM. The number of times at which each picture in the random access unit RAU is required for reproduction is described in order. As a result, it is not always necessary to arrange the picture number PN in the variable speed reproduction map RAM, and it is possible to save an area necessary for arranging the picture number PN.
FIG. 2D shows another example of the variable speed reproduction map RAM. First, information (Speed) indicating how many times speed reproduction is required for decoding is arranged, and then a picture number PN of a picture required for reproduction at that speed is described.

As the information (Speed) indicating how many times the reproduction speed requires decoding, for example, the speed is set to N when the picture M is described as required at N times speed. At this time, when the reproduction speed is K times, if N <K, the picture M does not need to be decoded, but if N> = K, the picture M needs to be decoded.
FIG. 3 is a schematic diagram showing an example of a picture reference relationship. Here, the hatched pictures are pictures referred to by other pictures.

In the example shown in FIG. 3A, the pictures I0, P3, and P6 are reproduced at the triple speed. In the example shown in FIG. 3B, the pictures I0 and P4 are reproduced at 4 × speed, and the pictures I0, B2, P4 and B6 are reproduced at 2 × speed. In the example shown in FIG. 3C, the pictures I0, P3, and P6 are reproduced at 3 × speed, and the pictures I0, P1, P3, P4, P6 and P7 are reproduced at 1.5 × speed.
Accordingly, in the example shown in FIG. 3A, the information (Speed) indicating how many times the reproduction speed is required for decoding is "3" for the pictures I0, P3, and P6, and "1" for the other pictures. In the example shown in b), the pictures I0 and P4 are "4", the pictures B2 and B6 are "2", and the other pictures are "1". In the example shown in FIG. 3C, the pictures I0, P3 and P6 are "4". 3 ", pictures P1, P4, and P7 are" 1.5 ", and other pictures are" 1 ".

Next, the operation of the moving picture coding apparatus 1 configured as described above will be described. FIG. 4 is a flowchart showing the operation when creating the variable speed reproduction map RAM.
The map creating unit 113 determines whether the picture to be encoded is a random access point, that is, the first picture of the random access unit RAU (step S10). If the result of this determination is that the picture is a random access point (YES in step S10), the map creation unit 113 creates a variable speed reproduction map RAM and outputs it to the variable length encoding unit 112 (step S11). Next, the variable length encoding unit 112 encodes the variable speed reproduction map RAM (step S12). Further, the variable-length encoding unit 112 encodes the current picture (Step S13).

On the other hand, if the result of the determination is that the picture is not a picture of a random access point (NO in step S10), variable-length coding section 112 codes the current picture (step S13).
Next, it is determined whether or not there is an uncoded picture (step S14). If there is an uncoded picture, the above operation (steps S10 to S14) is repeated, and if there is no uncoded picture, the process ends.

  By the way, when decoding the encoded stream Str, decoding is performed with reference to the picture parameter set PPS as described above, and when the picture parameter set PPS is changed, the new picture parameter set PPS is , Before the data of the picture to be referred to. However, when performing variable-speed playback, when decoding a picture that should refer to the changed new picture parameter set PPS, a situation occurs in which there is no picture parameter set PPS. For example, in the example shown in FIG. 3B, the pictures are coded in the order of I0, P4, B2, B1, B3, P8, B6, B5, B7. At this time, it is assumed that the picture parameter set PPS is changed in the picture B2, and the new picture parameter set PPS is added to the data of the picture B2 that refers to the changed new picture parameter set PPS. In this case, there is no problem in normal and double speed playback, but for example, in quadruple speed playback, pictures I0, P4, and P8 are played, and picture B2 is not decoded. Is not decrypted. Therefore, there is a problem that the picture P8 cannot be decoded because the picture parameter set PPS to be referred to by the picture P8 has not been decoded.

  Therefore, in the present embodiment, if the picture parameter set PPS is required for a picture reproduced at N times speed, the picture parameter set PPS is reproduced at N times speed even if it is already added to the picture reproduced at less than N times speed. Is always added to the picture to be copied. That is, in the above example, the new picture parameter set PPS changed in the picture B2 is also added to the data of the picture P8.

FIG. 5 is a flowchart showing the operation when the picture parameter set PPS is added.
The detecting unit 114 determines whether or not the picture to be coded is reproduced (decoded) at N times speed or higher (step S20). If the result of this determination is that the picture is to be played back at N times speed or higher (YES in step S20), the picture parameter set PPS referred to by the target picture has already been coded at the time of coding the picture played back at N times speed or higher. It is determined whether or not it has been performed (step S21). Here, if the picture to be reproduced at the N-times speed or higher has not been encoded yet (NO in step S21), the common information adding unit 115 adds the picture parameter set PPS to the target picture. , To the variable length coding unit 112 (step S22). Next, the variable length coding unit 112 codes the picture parameter set PPS (Step S23). Further, the variable-length encoding unit 112 encodes the current picture (Step S24).

On the other hand, as a result of the above determination, if the picture is not reproduced at the N-fold speed or higher (NO in step S20), and the picture parameter set PPS referred to by the target picture is already encoded at the time of encoding the picture reproduced at the N-fold speed or higher. If the current picture is encoded (YES in step S21), the variable-length encoding unit 112 encodes the current picture (step S24).
Next, it is determined whether or not there is an uncoded picture (step S25). If there is an uncoded picture, the above operation (steps S20 to S25) is repeated, and if there is no uncoded picture, the process ends.

FIG. 6 is a flowchart showing the operation when encoding the current picture.
The detecting unit 114 determines whether or not the picture to be coded is reproduced (decoded) at N-times speed or higher (step S30). If the result of this determination is that the picture is to be played back at N times speed or higher (YES in step S30), the current picture is coded with reference to the picture played back at N times speed or higher (step S31). On the other hand, if the picture is not a picture reproduced at the N-fold speed or higher (NO in step S30), the target picture is encoded with reference to an arbitrary picture (step S32).

  Next, it is determined whether or not the current picture is referred to when encoding another picture (step S33). If the result of this determination is that the picture is referenced by another picture (YES in step S33), the current picture is stored in the picture memory 109 (step S34). On the other hand, if the picture is not a picture referred to by another picture (NO in step S33), the current picture is not stored in the picture memory 109.

Next, it is determined whether or not there is an uncoded picture (step S35). If there is an uncoded picture, the above operation (steps S30 to S35) is repeated, and if there is no uncoded picture, the process ends.
As described above, the variable-speed reproduction map RAM is arranged before the pictures in the random access unit RAU, and the variable-speed reproduction map RAM stores information on which picture stream can be decoded and reproduced at a desired variable speed. It has been described. Further, even if the picture parameter set PPS is already added to a picture played back at a speed lower than N times, if the picture played back at N times speed is required, the picture played back at N times speed is always used. Has been added. Thus, the moving picture decoding apparatus can easily decode unnecessary pictures by decoding only pictures necessary for the desired variable-speed playback in accordance with the information described in the variable-speed playback map RAM. Variable-speed reproduction becomes possible.

  In the present embodiment, when creating a variable-speed playback map RAM, a picture that is not referred to from other pictures does not affect decoding of other pictures, and thus is referred to affect other pictures. A variable speed reproduction map RAM corresponding only to a picture may be created. A function corresponding to the variable speed reproduction map RAM, for example, instead of arranging information of each picture, operation information for each category such as a picture, an I picture, a P picture, and a B picture which is not referred to by others may be arranged. .

Further, identification information indicating that all of the streams are composed of such random access units RAU that can easily perform variable speed reproduction may be added.
Also, in the present embodiment, for example, if the speed of a picture M (information indicating how many times the speed is to be reproduced and decoding is necessary) is N and the reproduction speed is K times, if N <K, the picture M is decoded. Is not necessary, but it has been described that the decoding of the picture M is necessary if N> = K. However, even if N <K, if the difference between N and K is small, the image quality of the variable speed playback is reduced. For improvement, the picture M may be decoded.

  In addition, the information (Speed) indicating how many times the reproduction speed is required for decoding may not be a value indicating the actual reproduction speed, but may be a value indicating the degree of the reproduction speed. For example, "1" is a picture required only for normal-speed playback, "2" is a picture required for next-fast playback, and "3" is a picture required for faster-speed playback. What should I do?

(Embodiment 2)
In the first embodiment, an example in which the variable speed reproduction map RAM is arranged in the random access unit RAU has been described. However, if the variable speed reproduction map RAM of each random access unit RAU has the same contents, the variable speed reproduction map RAM is There is no need to allocate RAM.

FIG. 7 is a configuration diagram of a stream according to the second embodiment.
In the present embodiment, for example, a variable speed reproduction map table RAMTBL including a plurality of variable speed reproduction map RAMs based on the picture reference relationship as shown in FIG. 3 is created as shown in FIG. 7B. Then, in each random access unit RAU, as shown in FIG. 7A, a variable speed reproduction map identifier RAMID indicating which variable speed reproduction map RAM of the variable speed reproduction map table RAMTBL corresponds is arranged.

The variable speed reproduction map table RAMTBL may be arranged at the head of the stream, may be encoded as additional information with another stream, or may be provided in a device with a predetermined value determined in advance.
FIG. 8 is a flowchart showing the operation when creating the variable speed reproduction map identifier RAMID. Here, instead of creating and encoding the variable speed reproduction map RAM in the first embodiment (FIG. 4, steps S11 to S12), the variable speed reproduction map identifier RAMID is created and encoded (step S41). To S42).

By doing as described above, the same thing as the first embodiment can be realized by the variable speed reproduction map identifier RAMID.
When creating a variable speed playback map RAM included in the variable speed playback map table RAMTBL, a picture that is not referred to from other pictures does not affect decoding of other pictures, and therefore, a reference that affects other pictures. A variable speed reproduction map RAM corresponding to only the picture to be reproduced may be created. A function corresponding to the variable speed reproduction map RAM, for example, instead of arranging information of each picture, operation information for each category such as a picture, an I picture, a P picture, and a B picture which is not referred to by others may be arranged. .

Further, identification information indicating that all of the streams are composed of such random access units RAU that can easily perform variable speed reproduction may be added.
Further, in the first and second embodiments, the variable-speed reproduction map RAM and the variable-speed reproduction map identifier RAMID are assigned to each random access unit RAU. However, the present invention is not limited to this. For example, if the entire stream has the same structure, the variable-speed reproduction map RAM and the variable-speed reproduction map identifier RAMID may be assigned to each stream.

  Further, in the first and second embodiments, only the case where at least one variable speed reproduction map RAM is encoded has been described. However, a case where the variable speed reproduction map RAM is fixed by operation may be considered. The encoding of the reproduction map RAM becomes unnecessary. In order to enable such an operation, it is necessary to detect in advance the necessity of encoding the common information referred to by the picture to be subjected to variable-speed playback, and to determine the necessity of encoding the common information in the detected picture. In this case, common information may be added.

(Embodiment 3)
FIG. 9 is a block diagram showing the configuration of a video decoding device using the video decoding method according to the present invention. Note that the same reference numerals are given to devices that perform the same operations as the respective units of the conventional video decoding device 4 shown in FIG. 22, and description thereof is omitted.
The video decoding device 2 is a device that decodes the encoded stream Str encoded by the video encoding device 1 as described above, and includes a stream extraction unit 201, a variable-length decoding unit 202, and an extracted picture selection. A section 203, a motion compensation section 204, an inverse quantization section 205, an inverse orthogonal transform section 206, an addition section 207, a picture memory 208, and a memory control section 209.

  The extracted picture selection unit 203 converts the picture that needs to be decoded to be played back at the playback speed indicated by the playback speed information PlaySpeed input from the outside for the variable speed playback decoded by the variable length decoding unit 202. It is determined based on the information Map and notified to the stream extraction unit 201. The stream extracting unit 201 extracts only a stream corresponding to a picture determined to be required for decoding by the extracted picture selecting unit 203, and transmits the extracted stream to the variable length decoding unit 202. The memory control unit 209 performs control assuming that the picture referred to by another picture is stored in the picture memory 208 at the time of variable-speed playback, even if the picture is not decoded.

Next, the operation of the video decoding device 2 configured as described above will be described. FIGS. 10A and 10B are flowcharts showing the operation of the video decoding device 2, where FIG. 10A corresponds to the stream configuration shown in the first embodiment, and FIG. 10B corresponds to the stream configuration shown in the second embodiment. I do.
When decoding the encoded stream Str encoded with the stream configuration shown in Embodiment 1, the stream extraction unit 201 determines that the picture to be decoded is a random access point, that is, the first picture of the random access unit RAU. It is determined whether or not (step S50). If the result of this determination is that the picture is at a random access point (YES in step S50), the stream extraction unit 201 extracts the variable speed reproduction map RAM and outputs it to the variable length decoding unit 202 (step S51). Then, the variable-length decoding unit 202 decodes the variable-speed reproduction map RAM and outputs it to the extracted picture selection unit 203 (Step S52).

  Next, the extracted picture selection unit 203 determines whether the variable-length decoding unit 202 has decoded the picture that needs to be decoded in order to play at the playback speed indicated by the externally input playback speed information PlaySpeed. It is determined based on the variable speed reproduction map RAM, and is notified to the stream extraction unit 201 (step S53). The stream extracting unit 201 determines whether or not the current picture is a picture for which decoding has been determined to be unnecessary by the extracted picture selecting unit 203 (step S54). Here, if the current picture is not a picture determined not to need to be decoded (NO in step S54), only the stream corresponding to the current picture is extracted and output to the variable length decoding unit 202. The variable length decoding unit 202 decodes the stream corresponding to the input picture (Step S55).

Next, it is determined whether or not there is an undecoded picture (step S56). If there is an undecoded picture, the above operation (steps S50 to S56) is repeated, and if there is no undecoded picture, the process ends.
When decoding the coded stream Str encoded by the stream configuration shown in the second embodiment, as shown in FIG. 10B, the decoding target picture is a picture of a random access point. If it is (YES in step S50), the stream extraction unit 201 extracts the variable speed reproduction map identifier RAMID and outputs it to the variable length decoding unit 202 (step S61). Then, the variable-length decoding unit 202 decodes the variable-speed reproduction map identifier RAMID and outputs it to the extracted picture selection unit 203 (Step S62).

  Next, the extracted picture selection unit 203 determines whether the variable-length decoding unit 202 has decoded the picture that needs to be decoded in order to play at the playback speed indicated by the externally input playback speed information PlaySpeed. It is determined based on the variable speed reproduction map identifier RAMID and the variable speed reproduction map table RAMTBL, and is notified to the stream extraction unit 201 (step S63). Subsequent operations are the same as the case of decoding the coded stream Str coded with the stream configuration shown in the first embodiment. Here, it is assumed that the variable speed reproduction map table RAMTBL is decoded in advance and is included in the extracted picture selection unit 203.

  By the way, when decoding the current picture, since the picture to be referred to is specified using the relative reference index Index, the picture specified as the reference picture when performing variable-speed playback is It may be different from the reference picture. For example, in the stream example shown in FIG. 3B, when all pictures are reproduced at the normal speed, all pictures I0, P4, B2 referred to as shown in FIG. Are stored in the picture memory 208. On the other hand, in the case of quadruple speed reproduction, pictures I0 and P4 which are reproduced (decoded) at quadruple speed reproduction and are referred to are stored in the picture memory 208 as shown in FIG. . For this reason, if the reference index Index (designating the two-preceding picture in the picture memory 208) that designates the picture P4 referenced by the picture P8 is also used in the case of quadruple-speed playback, as shown in FIG. , The picture I0 is specified, and a problem occurs.

Therefore, in the present embodiment, even for a picture that is not reproduced when performing variable-speed reproduction, a picture referred to by another picture is always treated as being stored in the picture memory 208. That is, in the above example, as shown in FIG. 11C, control is performed assuming that data is stored after the picture P4.
FIG. 12 is a flowchart showing the operation when decoding the current picture. Note that the operation shown in this flowchart is a process of determining whether or not decoding is unnecessary in the flowchart of FIG. 10, a process of decoding the current picture, and a process of determining whether or not there is an undecoded picture (step S53 to S56).

  The stream extracting unit 201 determines whether or not the current picture is a picture determined to be required to be decoded at N × speed (Step S90). As a result of this determination, if the current picture is a picture that is determined to need to be decoded (YES in step S90), the variable-length decoding unit 202 decodes the current picture with reference to a picture at N-times speed or higher ( Step S91). Next, the memory control unit 209 determines whether or not there is a picture that has not been decoded at N times speed but is referred to by another picture (step S92).

  As a result of this determination, if such a picture exists (YES in step S92), the memory control unit 209 stores the picture that has not been decoded at N times speed but is referred to by another picture in the picture memory 208. (Step S93). Next, the memory control unit 209 stores the decoded target picture in the picture memory 208 (Step S94).

On the other hand, if no such picture exists (NO in step S92), the memory control unit 209 simply stores the decoded target picture in the picture memory 208 (step S94).
Next, it is determined whether or not there is an undecoded picture (step S95). If there is an undecoded picture, the above operation (steps S90 to S95) is repeated, and if there is no undecoded picture, the process ends. Also, if the current picture is N times faster and does not need to be decoded (NO in step S90), it is similarly determined whether there is an undecoded picture (step S95). (Steps S90 to S95) are repeated, and if there is no undecoded picture, the process ends.

  As described above, the coded stream Str encoded as shown in the first and second embodiments is decoded from only the pictures necessary for the desired variable speed reproduction based on the variable speed reproduction information Map. By doing so, variable-speed playback can be easily performed without omitting unnecessary decoding of pictures.

(Embodiment 4)
In the present embodiment, a case will be described in which the operation of moving image decoding apparatus 2 shown in Embodiment 3 is partially different.

FIGS. 13A and 13B are flowcharts showing the operation of the video decoding device 2. FIG. 13A corresponds to the stream configuration shown in the first embodiment, and FIG. 13B corresponds to the stream configuration shown in the second embodiment. I do. 13, the same processes as those in the flowchart shown in FIG. 10 are denoted by the same step numbers, and description thereof will be omitted.
In the third embodiment, as shown in FIG. 10A, the stream extraction unit 201 determines whether the picture to be decoded is the first picture of the random access unit RAU (step S50). In the present embodiment, it is simply determined whether or not the variable speed reproduction map RAM is arranged as shown in FIG. 13A (step S70). As a result, if the variable speed reproduction map RAM is arranged (YES in step S70), the stream extraction unit 201 extracts the variable speed reproduction map RAM and outputs it to the variable length decoding unit 202 (step S51).

  Similarly, in FIG. 10B, the stream extraction unit 201 determines whether the picture to be decoded is the first picture of the random access unit RAU (step S50). Then, as shown in FIG. 13B, it is determined whether or not the variable speed reproduction map identifier RAMID is simply arranged (step S80). As a result, if the variable speed reproduction map identifier RAMID is arranged (YES in step S80), the stream extraction unit 201 extracts the variable speed reproduction map identifier RAMID and outputs it to the variable length decoding unit 202 (step S61). ).

  As described above, whether or not the variable-speed reproduction map RAM or the variable-speed reproduction map identifier RAMID is arranged can also determine whether the coded stream encoded as described in the first and second embodiments is used. By decoding only the pictures necessary for the desired variable-speed playback based on the variable-speed playback information Map, the variable-speed playback can be easily performed without decoding unnecessary pictures.

(Embodiment 5)
Furthermore, by recording a program for realizing the moving picture encoding method and the moving picture decoding method described in each of the above-described embodiments on a recording medium such as a flexible disk, the above-described embodiments can be used. The illustrated processing can be easily performed in an independent computer system.

FIG. 14 is an explanatory diagram of a case where the moving picture encoding method and the moving picture decoding method of each of the above embodiments are implemented by a computer system using a program recorded on a recording medium such as a flexible disk.
FIG. 14B shows the appearance, cross-sectional structure, and flexible disk of the flexible disk viewed from the front, and FIG. 14A shows an example of the physical format of the flexible disk which is a recording medium body. The flexible disk FD is built in the case F, and a plurality of tracks Tr are formed concentrically from the outer circumference toward the inner circumference on the surface of the disk, and each track is divided into 16 sectors Se in an angular direction. ing. Therefore, in the flexible disk storing the program, the program is recorded in an area allocated on the flexible disk FD.

  FIG. 14C shows a configuration for recording and reproducing the program on the flexible disk FD. When the above-mentioned program for realizing the moving picture encoding method and the moving picture decoding method is recorded on the flexible disk FD, the program is written from the computer system Cs via the flexible disk drive. When the moving picture coding method and the moving picture decoding method are implemented in a computer system by a program for realizing the moving picture coding method and the moving picture decoding method in a flexible disk, the program is executed by a flexible disk drive. Read from flexible disk and transfer to computer system.

  In the above description, the description has been made using a flexible disk as a recording medium. However, the same description can be made using an optical disk. Further, the recording medium is not limited to this, and the present invention can be similarly implemented as long as the program can be recorded, such as an IC card or a ROM cassette.

(Embodiment 6)
Further, here, application examples of the moving picture coding method and the moving picture decoding method described in the above embodiment and a system using the same will be described.

FIG. 15 is a block diagram illustrating an overall configuration of a content supply system ex100 that realizes a content distribution service. A communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed wireless stations, are installed in each cell.
The content supply system ex100 includes, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, and a camera on the Internet ex101 via the Internet service provider ex102 and the telephone network ex104, and the base stations ex107 to ex110. Each device such as a mobile phone ex115 with a tag is connected.

However, the content supply system ex100 is not limited to the combination as shown in FIG. 15, and may be connected in any combination. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.
The camera ex113 is a device capable of shooting moving images, such as a digital video camera. In addition, a mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global System for Mobile Communications) system. Or PHS (Personal Handyphone System) or the like, and either may be used.

  The streaming server ex103 is connected from the camera ex113 to the base station ex109 and the telephone network ex104, and enables live distribution and the like based on the encoded data transmitted by the user using the camera ex113. The encoding process of the captured data may be performed by the camera ex113, or may be performed by a server or the like that performs the data transmission process. Also, moving image data captured by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device such as a digital camera capable of shooting still images and moving images. In this case, encoding of the moving image data may be performed by the camera ex116 or the computer ex111. The encoding process is performed by the LSI ex117 included in the computer ex111 and the camera ex116. The image encoding / decoding software may be incorporated in any storage medium (CD-ROM, flexible disk, hard disk, or the like) that is a recording medium readable by the computer ex111 or the like. Further, the moving image data may be transmitted by a mobile phone with camera ex115. The moving image data at this time is data that has been encoded by the LSI included in the mobile phone ex115.

  In the content supply system ex100, the content (for example, a video of a live music shot) captured by the user with the camera ex113, the camera ex116, or the like is encoded and transmitted to the streaming server ex103 in the same manner as in the above embodiment. On the other hand, the streaming server ex103 stream-distributes the content data to the requesting client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, and the like, which are capable of decoding the encoded data. In this way, the content providing system ex100 can receive and reproduce the encoded data on the client, and further, can receive, decode, and reproduce the encoded data on the client in real time, thereby realizing personal broadcasting. It is a system that becomes possible.

The encoding and decoding of each device constituting this system may be performed using the video encoding device or the video decoding device described in each of the above embodiments.
A mobile phone will be described as an example.
FIG. 16 is a diagram illustrating the mobile phone ex115 using the moving picture coding method and the moving picture decoding method described in the above embodiment. The mobile phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a video image of a CCD camera or the like, a camera unit ex203 capable of taking a still image, a video image captured by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display for displaying data obtained by decoding a received video or the like, a main unit including a group of operation keys ex204, an audio output unit ex208 such as a speaker for outputting audio, and audio input. Input unit ex205 such as a microphone for storing encoded or decoded data, such as data of captured moving images or still images, received mail data, moving image data or still image data, etc. Recording medium ex207, and a slot ex20 for allowing the recording medium ex207 to be attached to the mobile phone ex115. The it has. The recording medium ex207 stores a flash memory element which is a kind of EEPROM (Electrically Erasable and Programmable Read Only Memory) which is a nonvolatile memory which can be electrically rewritten and erased, in a plastic case such as an SD card.

  Further, the mobile phone ex115 will be described with reference to FIG. The mobile phone ex115 controls a power supply circuit unit ex310, an operation input control unit ex304, an image encoding unit, and a main control unit ex311 that integrally controls each unit of a main unit including a display unit ex202 and operation keys ex204. Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / reproducing unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via a synchronous bus ex313. Connected to each other.

The power supply circuit unit ex310 activates the camera-equipped digital mobile phone ex115 in an operable state by supplying power to each unit from the battery pack when the call end and the power key are turned on by a user operation. .
The mobile phone ex115 converts an audio signal collected by the audio input unit ex205 into digital audio data by the audio processing unit ex305 in the audio communication mode based on the control of the main control unit ex311 including a CPU, a ROM, a RAM, and the like. This is spread-spectrum processed by a modulation / demodulation circuit unit ex306, subjected to digital-analog conversion processing and frequency conversion processing by a transmission / reception circuit unit ex301, and then transmitted via an antenna ex201. The mobile phone ex115 amplifies received data received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-to-digital conversion processing, performs spectrum despreading processing in the modulation / demodulation circuit unit ex306, and performs analog voice decoding in the voice processing unit ex305. After the data is converted, the data is output via the audio output unit ex208.

  Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital / analog conversion processing and frequency conversion processing on the transmission / reception circuit unit ex301, and transmits the text data to the base station ex110 via the antenna ex201.

  When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When the image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

  The image encoding unit ex312 includes the moving image encoding device described in the present invention, and encodes the image data supplied from the camera unit ex203 using the moving image encoding device described in the above embodiment. The image data is converted into encoded image data by performing compression encoding according to a demultiplexing method, and is transmitted to the demultiplexing unit ex308. Further, at this time, the mobile phone ex115 simultaneously transmits the voice collected by the voice input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 as digital voice data via the voice processing unit ex305.

  The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and modulates and outputs the resulting multiplexed data. The signal is subjected to spread spectrum processing in ex306 and subjected to digital / analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmitted via the antenna ex201.

When data of a moving image file linked to a homepage or the like is received in the data communication mode, the data received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexed data is obtained. The demultiplexed data is sent to the demultiplexing unit ex308.
To decode the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data into a bit stream of image data and a bit stream of audio data, and performs synchronization. The coded image data is supplied to the image decoding unit ex309 via the bus ex313, and the audio data is supplied to the audio processing unit ex305.

  Next, the image decoding unit ex309 has a configuration including the moving image decoding device described in the present invention, and converts a bit stream of image data into a decoding method corresponding to the encoding method described in the above embodiment. By decoding, reproduced moving image data is generated and supplied to the display unit ex202 via the LCD control unit ex302, whereby, for example, moving image data included in a moving image file linked to a homepage is displayed. At this time, at the same time, the audio processing unit ex305 converts the audio data into analog audio data, and then supplies the analog audio data to the audio output unit ex208, whereby, for example, the audio data included in the moving image file linked to the homepage is reproduced. You.

  It should be noted that the present invention is not limited to the example of the system described above, and digital broadcasting using satellites and terrestrial waves has recently become a topic. As shown in FIG. Any of the video decoding devices can be incorporated. Specifically, at the broadcasting station ex409, the bit stream of the video information is transmitted to the communication or the broadcasting satellite ex410 via radio waves. The broadcast satellite ex410 receiving this transmits a broadcast radio wave, receives this radio wave with a home antenna ex406 having a satellite broadcast reception facility, and outputs the radio wave to a television (receiver) ex401 or a set-top box (STB) ex407. The device decodes the bit stream and reproduces it. In addition, the moving picture decoding apparatus described in the above embodiment can be mounted on a reproducing apparatus ex403 that reads and decodes a bit stream recorded on a storage medium ex402 such as a CD or DVD that is a recording medium. . In this case, the reproduced video signal is displayed on the monitor ex404. Further, a configuration is also conceivable in which a moving image decoding apparatus is mounted in a set-top box ex407 connected to a cable ex405 for cable television or an antenna ex406 for satellite / terrestrial broadcasting, and this is reproduced on a monitor ex408 of the television. At this time, the moving picture decoding device may be incorporated in the television instead of the set-top box. In addition, a car ex412 having an antenna ex411 can receive a signal from the satellite ex410 or a base station ex107 or the like, and can reproduce a moving image on a display device such as a car navigation ex413 included in the car ex412.

  Furthermore, an image signal can be encoded by the moving image encoding device described in the above embodiment and recorded on a recording medium. As a specific example, there is a recorder ex420 such as a DVD recorder for recording an image signal on a DVD disc ex421 or a disc recorder for recording on a hard disk. Furthermore, it can be recorded on the SD card ex422. If the recorder ex420 includes the moving picture decoding device described in the above embodiment, the video signal recorded on the DVD disc ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.

The configuration of the car navigation system ex413 may be, for example, a configuration excluding the camera unit ex203, the camera interface unit ex303, and the image encoding unit ex312 from the configuration illustrated in FIG. 17, and the same applies to the computer ex111 and the television (receiver). ) Ex401 and the like are also conceivable.
In addition, the terminal such as the above-mentioned mobile phone ex114 has three mounting formats, in addition to a transmitting / receiving terminal having both an encoder and a decoder, a transmitting terminal having only an encoder and a receiving terminal having only a decoder. Can be considered.

As described above, the moving picture coding method or the moving picture decoding method described in the above embodiment can be used for any of the devices and systems described above. The effect can be obtained.
Further, the present invention is not limited to the above embodiment, and various changes or modifications can be made without departing from the scope of the present invention.

  As described above, according to the moving picture coding method and the moving picture decoding method according to the present invention, for example, a mobile phone, a DVD device, a personal computer or the like codes each picture constituting a moving picture to generate a code sequence. Or a method for decoding the generated code string.

FIG. 1 is a block diagram illustrating a configuration of a video encoding device that implements the video encoding method of the present invention. It is a block diagram of a stream of the present invention (Embodiment 1), (a) a structural example of a random access unit RAU, (b) an example of a variable speed reproduction map RAM, (c) another example of a variable speed reproduction map RAM , (D) is another example of a variable speed reproduction map RAM. It is a schematic diagram which shows the reference relationship of a picture (Embodiment 1), (a) An example of a reference relationship of a picture, (b) Another example of a reference relationship of a picture, (c) Another example of a reference relationship of a picture It is. 5 is a flowchart showing an operation when creating a variable speed reproduction map RAM. 9 is a flowchart illustrating an operation when a picture parameter set PPS is added. 9 is a flowchart illustrating an operation when encoding a target picture. It is a block diagram (Embodiment 2) of a stream of the present invention, (a) is an example of the structure of the random access unit RAU, (b) is an example of the variable speed reproduction map table RAMTBL. 9 is a flowchart showing an operation when creating a variable speed reproduction map identifier RAMID. It is a block diagram showing the composition of the video decoding device which realizes the video decoding method of the present invention. It is a flowchart which shows operation | movement in a moving image decoding apparatus, Comprising: (a) It is a flowchart corresponding to the stream structure shown in Embodiment 1, and (b) the stream structure shown in Embodiment 2. It is a schematic diagram which shows the preservation | save state of a picture memory, It is a schematic diagram which shows the preservation | save state at the time of (a) at the time of normal reproduction | regeneration, (b) at the time of 4 times speed reproduction of the conventional, and (c) at the time of 4 times speed reproduction of this invention. 9 is a flowchart illustrating an operation when decoding a target picture. 10 is a flowchart illustrating another operation of the video decoding device, and corresponds to (a) a stream configuration described in Embodiment 1 and (b) a flowchart corresponding to the stream configuration described in Embodiment 2. It is an explanatory view of a recording medium for storing a program for realizing the moving picture encoding method and the moving picture decoding method of each embodiment by a computer system, and (a) a flexible disk as a recording medium main body. Explanatory diagram showing an example of a physical format, (b) an external view from the front of a flexible disk, a cross-sectional structure, and an explanatory diagram showing a flexible disk, and (c) a configuration for recording and reproducing the program on the flexible disk FD. FIG. FIG. 1 is a block diagram illustrating an overall configuration of a content supply system that realizes a content distribution service. It is a figure which shows an example of a mobile telephone. FIG. 2 is a block diagram illustrating an internal configuration of the mobile phone. FIG. 1 is a block diagram illustrating an overall configuration of a digital broadcasting system. It is a block diagram of a conventional MPEG2 stream, and is a diagram showing (a) a picture flow and (b) a hierarchical structure of a stream. FIG. 11 is a diagram illustrating a hierarchical structure of another conventional stream. FIG. 11 is a block diagram illustrating a configuration of a moving image encoding device that realizes a conventional moving image encoding method. FIG. 11 is a block diagram illustrating a configuration of a video decoding device that implements a conventional video decoding method. FIG. 3 is a schematic diagram illustrating an example of a picture reference relationship, in which (a) is a prediction structure between pictures in MPEG-2, and (b) is an example of a prediction structure in JVT. FIG. 4 is a schematic diagram illustrating an example of a picture reference relationship.

Explanation of reference numerals

REFERENCE SIGNS LIST 1 moving picture coding apparatus 2 moving picture decoding apparatus 101 motion detecting section 102, 204 motion compensating section 103 subtracting section 104 orthogonal transform section 105 quantizing section 106, 205 inverse quantizing section 107, 206 inverse orthogonal transform section 108, 207 Addition unit 109, 208 Picture memory 110 Switch 111 Prediction structure determination unit 112 Variable length coding unit 113 Map creation unit 114 Detection unit 115 Common information addition unit 201 Stream extraction unit 202 Variable length decoding unit 203 Extracted picture selection unit 209 Memory control Department

Claims (25)

A moving picture coding method for coding a moving picture signal on a picture basis to generate a coded stream,
An information creating step of creating variable speed playback information for specifying a picture to be subjected to variable speed playback,
Encoding the variable speed playback information and adding it to the encoded stream. In the information creating step, the random access unit is composed of a plurality of pictures, and the decoding target picture in the random access unit can be decoded with reference to only the pictures in the random access unit. The moving picture encoding method according to claim 1, wherein the variable speed reproduction information is created for each of the random access units. 2. The information creating step according to claim 1, wherein the variable-speed playback information is created by associating a variable-speed playback speed with information specifying a picture to be played back at the speed. Video encoding method. The moving picture encoding method according to claim 3, wherein the variable-speed playback information describes the speed at which decoding of the picture is required for each picture number. The moving picture encoding method according to claim 3, wherein the variable-speed reproduction information describes, in order, the rates at which decoding is required for each picture. The moving picture encoding method according to claim 3, wherein the variable speed reproduction information describes a picture number of a picture necessary for reproduction at the speed for each of the speeds. The moving picture encoding method according to claim 1, wherein, in the information creating step, an index for identifying a corresponding reference relation from a list of predetermined reference relations is created as the variable speed reproduction information. The list of the predetermined reference relationship is a map table having a plurality of sets of map information in which the speed of variable-speed playback and information for specifying a picture to be played at the speed are associated with each other,
The moving picture encoding method according to claim 7, wherein, in the information creating step, an identifier for selecting corresponding map information from the map table is created as the variable speed reproduction information. In the information creating step, the map table is created,
The moving image encoding method according to claim 8, wherein, in the encoding step, the map table is encoded and added to the encoded stream. The video encoding method further includes:
A detecting step of detecting the necessity of encoding the common information referred to by the picture targeted for the variable-speed playback,
2. The moving picture encoding method according to claim 1, further comprising: adding a common information to the picture for which the necessity of encoding the common information is detected in the detecting step. A moving image decoding method for decoding an encoded stream in units of pictures,
An information extraction step of extracting variable speed reproduction information for specifying a picture to be subjected to variable speed reproduction,
Decoding the variable-speed playback information, and identifying and decoding a picture to be subjected to variable-speed playback based on the variable-speed playback information. Method. The information extracting step is a random access unit composed of a plurality of pictures, and is capable of decoding a decoding target picture in the random access unit with reference to only the pictures in the random access unit. The moving picture decoding method according to claim 11, wherein the variable speed reproduction information is extracted for each of the random access units. The variable speed reproduction information is associated with a variable speed reproduction speed and information specifying a picture to be reproduced at the speed,
The moving picture decoding method according to claim 11, wherein in the decoding step, a picture to be reproduced at the specified speed is specified based on the specified speed and the variable speed reproduction information. . The moving picture decoding method according to claim 13, wherein the variable-speed playback information describes the speed at which decoding of the picture is required for each picture number. The moving picture decoding method according to claim 13, wherein the variable speed reproduction information describes the speeds at which decoding is required for each picture in order. The moving picture decoding method according to claim 13, wherein the variable speed reproduction information describes a picture number of a picture necessary for reproduction at the speed for each of the speeds. The variable-speed playback information is an index that specifies a corresponding reference relationship from a predetermined list of reference relationships,
The moving picture decoding method according to claim 11, wherein, in the decoding step, a corresponding reference relation is specified from the predetermined reference relation list based on the index. The list of the predetermined reference relationship is a map table having a plurality of sets of map information in which the speed of variable-speed playback and information for specifying a picture to be played at the speed are associated with each other,
The index is an identifier for selecting corresponding map information from the map table,
The moving picture decoding according to claim 17, wherein, in the decoding step, a picture to be reproduced at the designated speed is specified based on the designated speed, the map table, and the identifier. Method. In the information extracting step, the map table is extracted,
19. The moving picture decoding method according to claim 18, wherein, in the decoding step, the map table is decoded. The video decoding method may further include:
At the time of variable speed playback, a memory control step of controlling a picture referred to by another picture as being stored in a picture memory, even if the picture is not a target of the variable speed playback, is included. The moving picture decoding method according to claim 11. A moving image encoding apparatus that encodes a moving image signal on a picture basis to generate an encoded stream,
Information creating means for creating information for variable speed playback for specifying a picture to be subjected to variable speed playback,
Encoding means for encoding the variable speed reproduction information and adding the encoded information to the encoded stream. A moving picture decoding apparatus for decoding an encoded stream in units of pictures,
Information extracting means for extracting information for variable-speed reproduction for specifying a picture to be subjected to variable-speed reproduction,
Decoding means for decoding the information for variable-speed playback and for specifying and decoding a picture to be subjected to variable-speed playback based on the information for variable-speed playback. apparatus. A program for encoding a moving image signal on a picture basis to generate an encoded stream,
An information creating step of creating variable speed playback information for specifying a picture to be subjected to variable speed playback,
A step of encoding the variable speed reproduction information and adding the encoded information to the encoded stream. A program for decoding an encoded stream in units of pictures,
An information extraction step of extracting variable speed reproduction information for specifying a picture to be subjected to variable speed reproduction,
Decoding the variable-speed playback information, and identifying and decoding a picture to be subjected to variable-speed playback based on the variable-speed playback information. . A coded stream obtained by coding a moving image signal on a picture basis,
A coded stream including variable-speed playback information for specifying a picture to be subjected to variable-speed playback.

Images