KR101154743B1 - Encoder apparatus, encoding method, decoder apparatus, decoding method, recording medium, and playback apparatus - Google Patents

Abstract

The pictures of the progressive scan are displayed so as not to impair their vertical resolution. In the image encoding apparatus 101, the determination result of which picture is interlaced scanning or sequential scanning is set to the scanning flag, information indicating the display system of the picture is set to the display system instruction information, and they are multiplexed. The encoded data is output. In the image decoding device 102, a picture is recognized based on the scan flag of the encoded data and the display method indication information. When the picture is a picture for interlaced scanning, a frame picture is generated and output by field interpolating the picture. When the picture is a picture of sequential scanning, the picture is repeatedly output as many times as the frame picture corresponding to the display system indication information. The present invention can be applied, for example, to the case of encoding / decoding an image based on H.264 / AVC.

Image data, interlaced scanning, sequential scanning, encoding, decoding

Description

Coding apparatus and coding method, decoding apparatus and decoding method, recording medium, and reproducing apparatus {ENCODER APPARATUS, ENCODING METHOD, DECODER APPARATUS, DECODING METHOD, RECORDING MEDIUM, AND PLAYBACK APPARATUS}

The present invention relates to an encoding device and an encoding method, a decoding device and a decoding method, a program and a program recording medium, a data recording medium and a data structure, and a reproduction device. In particular, for example, a picture of sequential scanning is vertically oriented. The present invention relates to an encoding device and an encoding method, a decoding device and a decoding method, a program and a program recording medium, a data recording medium and a data structure, and a reproducing device, which can be displayed so as not to impair the resolution.

In recent years, an image signal is treated as digital data, and the image signal is compressed by orthogonal transformation such as discrete cosine transform and motion compensation, using the redundancy peculiar to the image for the purpose of transferring and accumulating highly efficient information. Image processing apparatuses conforming to standards such as MPEG (Moving Picture Experts Group) to be decoded are widely used by broadcasting stations that deliver information and the like and general homes that receive information.

With the dissemination of such an image processing apparatus, the fidelity of the content which can be handled by the image processing apparatus is aimed at, For example, the content of a movie is a content.

Since a movie is a continuous photograph taken on a film (movie film), it is an electric signal for television broadcasting, for example, to broadcast it from a broadcasting station as a program of a television broadcast, or to sell it as a video package. It must be converted into an in picture signal (video signal). That is, the image on the optically projected film must be electrically acquired to perform the conversion of the image display rate and other processing. This technique is commonly called telecine.

Usually, in the image signal and the movie film, the number of frames (comma) displayed per second, that is, the display rate is different. At present, many picture films display pictures of 24 frames per second, for example, in the NTSC (National Television System Committee) system, pictures of 30 frames are displayed for one second. Therefore, in telecine, such display rate conversion is performed as one of the main processes. This display rate conversion is called 2-3 pulldown.

In addition, the display rate of the NTSC system image signal is 29.97 Hz, and the display rate of the movie film is 23.976 Hz.

Here, the picture of the frame can be divided into a picture called a field, as shown in FIG. For example, pixels (pixel values) of odd lines of a frame are called top fields, and pixels of even lines are called bottom fields, respectively.

The picture includes a picture whose scanning method is interlaced scanning and a picture that is sequentially scanned. In a picture of interlaced scanning and a picture of sequential scanning, the time when a pixel was sampled differs. That is, in the interlaced picture, the sample time is shifted between even and odd lines in one frame, and as a result, the sample time of the top field and the bottom field is different. On the other hand, in sequential scanning, the sample time of the top field and the bottom field is the same.

The picture of the movie film is displayed in units of frames as a picture of sequential scanning. A picture of the NTSC system is a picture of interlaced scanning and displayed in field units. That is, in sequential scanning, scanning of frames is performed sequentially, and as a result, pictures of the frames are displayed. In interlaced scanning, scanning of a frame is performed interlaced, that is, scanning of a frame is alternately performed on the top field and the bottom field constituting the frame, and as a result, the picture and the bottom field of the top field constituting the frame are consequently. Pictures are displayed alternately.

For this reason, in telecine, a frame picture of a 24 영화 movie film is converted into a 30 NT NTSC frame picture composed of a 60 ㎐ field (exactly 59.94 ㎐) field by 2-3 pull down.

That is, in the 2-3 pulldown, first, the frame of the 24 영화 movie film is divided into a top field and a bottom field as shown in FIG. 1. The top field and the bottom field are appropriately allocated to the fields constituting the 30 ms NTSC frame.

Specifically, for example, four frames of a 24 ㎐ movie film are represented by A, B, C, and D in time series, and, for example, a top field obtained by dividing frame A is indicated by A (1). If the bottom field is represented as A (2), in the 2-3 pulldown, as shown in FIG. 2, for example, the top field A (1) and the bottom field A (2) are initially 30 ms. An NTSC frame of this type is constructed, and a second 30 ms NTSC type frame is formed of the top field B (1) and the bottom field B (2). In addition, a third 30 ms NTSC frame is composed of the top field B (1) and the bottom field C (2), and the fourth 30 is made of the top field C (1) and the bottom field D (2). NT NTSC frame is configured. The top field D (1) and the bottom field D (2) constitute a fifth 30 ms NTSC frame.

In the 2-3 pull-down, as described above, the field B (1) constituting the frame B of the 24 영화 movie film is the top field of the second 30 NT NTSC system frame and the top field behind the two fields ( The field D (2) constituting the frame D of the 24 영화 movie film is allocated to both the top field of the 3 rd frame), and the bottom field of the fourth 30 NT NTSC system frame, and the 2 fields. By assigning to both of the following bottom fields (bottom fields of the fifth frame), the frame of a 24 영화 movie film is converted into a 30 NT NTSC frame of 5 frame units in 4 frame units.

Therefore, in 2-3 pulldown, one frame of the movie film is converted into two fields of the NTSC system, and the next frame of the movie film is converted into the next three fields of the NTSC system, One frame is converted into the next two fields of the NTSC system, and in the same manner, each frame of the movie film is alternately converted into two fields and three fields of the NTSC system.

By the way, recently, H.264 or MPEG AVC (Advanced Video Coding) (ITU-T Q6 / 16 or ISO / IEC 14496-10) (hereinafter, H.264 / A standardization of AVC (described as AVC) was performed. In H.264 / AVC, image signals are compressed / decoded by orthogonal transform and motion compensation, as in conventional encoding (decoding) methods such as MPEG2 and MPEG4. In addition, although H.264 / AVC requires a large amount of computation by encoding and decoding, compared to such a conventional coding method, it is known that higher coding efficiency is realized.

FIG. 3 shows a configuration of an example of an image coding apparatus for compression coding a picture of an NTSC system obtained by 2-3 pulldown of a picture of a movie film based on the H.264 / AVC standard.

The NTSC picture (picture signal) obtained by 2-3 pulldown is supplied to the A / D conversion unit 1 in units of fields, and the A / D conversion unit 1 displays the pictures supplied there. / D (Analog / Digital) conversion converts the signal into a digital signal and supplies it to the 2-3 detection unit 2. The 2-3 detection unit 2 detects 2-3 rhymes using the inter-field difference information, which is the difference between two fields, of the picture from the A / D conversion unit 1, and the like.

That is, as described above, in the 2-3 pulldown, each frame of the movie film is alternately converted into two fields and three fields of the NTSC system, so that the sequence of the NTSC system picture is from one frame of sequential scanning of the movie film. The pair of two fields obtained and the pair of three fields obtained from one frame of sequential scanning of a movie film have so-called 2-3 rhythms alternately repeated. In the 2-3 detection unit 2, detection of the pair of two fields and the pair of three fields (detection of 2-3 rhymes) is performed.

Also, the 2-3 detection unit 2 determines whether the picture of each field of the NTSC system is a sequential scan or interlaced picture based on the difference between the fields and the detection result of the 2-3 rhymes. Determine. Here, suppose that the picture supplied to the picture coding apparatus of FIG. 3 is obtained by 2-3 pull down of a picture of an arbitrary movie film. In the 2-3 detection unit 2, two fields obtained from one frame of the movie film. The set of pictures or the set of pictures of three fields is determined to be a picture of progressive scanning.

Then, the 2-3 detection unit 2 constitutes a sequential scanning picture of one frame from a set of two-field pictures or a set of three-field pictures determined to be sequential scanning pictures, that is, in the present case, A picture of a frame of a 24 ㎐ movie film before 2-3 pulldown is constructed and supplied to the screen rearrangement buffer 3.

Further, in the 2-3 detection unit 2, for example, when it is determined that two fields constituting an arbitrary frame are pictures of interlaced scanning, the pictures of the two fields are each pictures of the fields, or the two fields. The picture rearrangement buffer 3 is supplied to the picture rearrangement buffer 3 as a picture of one frame.

The picture (determined) of the interlaced scan 2 field is determined by which picture of the field or frame depends on, for example, which picture of the field or frame is encoded according to the higher coding efficiency. In other words, if the encoding efficiency of the field is higher than the coding efficiency, the interlaced picture is the picture of the field. If the encoding of the field is high encoding efficiency, the picture of the interlaced scanning is the frame of the frame. It becomes a picture.

The screen rearrangement buffer 3 temporarily stores a picture from the 2-3 detection unit 2, rearranges the stored picture in a predetermined encoding order, and executes macroblock units as a picture to be encoded as a target picture. To the adder (4).

When the picture of interest is an intra coded picture to be intra coded, the adder 4 supplies the picture of interest to the orthogonal transform unit 5 as it is.

In addition, when the picture of interest is an inter-encoded picture to be inter-coded, the adder 4 subtracts the predictive image supplied from the motion prediction / compensation (ME / MC) unit 12 from the picture of interest and orthogonally transforms it. It supplies to the part 5.

That is, the motion predictor / compensator 12 detects the motion vector of the picture stored in the screen rearrangement buffer 3 and, from the frame memory 11, the reference picture of the picture of interest, which is already encoded and decoded. The predicted picture of the picture of interest in the optimum prediction mode (prediction mode with the highest encoding efficiency) is generated by reading a picture to be obtained and performing motion compensation on the reference picture based on the motion vector. Then, the motion predictor / compensator 12 supplies the predictive image to the adder 4, and the adder 4 thus processes the predictive image supplied from the motion predictor / compensator 12 in the interest picture. It subtracts from and supplies it to the orthogonal conversion part 5.

The orthogonal transform unit 5 performs, for example, orthogonal transform such as discrete cosine transform or the like on the picture of interest supplied from the adder 4 or the difference image (picture) that is the result of subtracting the predictive picture from the picture of interest. The resulting transform coefficient is supplied to the quantization (Q) unit 6.

The quantization unit 6 quantizes the transform coefficients from the orthogonal transform unit 5 in the quantization step according to the control from the rate control unit 13 to be described later, and reversibly encodes the resulting quantization coefficients (VLC ENC). The part 7 and the inverse quantization (INY Q) part 9 are supplied.

The reversible coding unit 7 performs variable length coding or arithmetic coding on the quantization coefficients from the quantization unit 6 and the motion vectors detected by the motion prediction / compensation unit 12, although not shown. Reversible coding is performed to perform multiplexing, and the resulting encoded data (stream) is supplied to the accumulation buffer 8.

In the reversible coding unit 7, a motion vector or the like is inserted (multiplexed) into a so-called header portion of the coded data.

The accumulation buffer 8 temporarily stores the encoded data from the reversible encoding unit 7 and outputs the encoded data at a predetermined rate. The encoded data output by the accumulation buffer 8 is, for example, recorded on a DVD (Digital Versatile Disc) or the like. Here, for the DVD, for example, see Non-Patent Document 1 "DVD Specifications for Read-Only Disc Part 3; Version 1.1 December 1997.

In addition, the storage amount (accumulation amount) of the encoded data of the accumulation buffer 8 is supplied to the rate control unit 13, and the rate control unit 13 is based on the storage amount of the accumulation buffer 8 and stores the accumulation buffer ( The quantization step of the quantization unit 6 is feedback controlled so that 8) does not overflow or underflow.

On the other hand, the inverse quantization unit 9 inversely quantizes the transform coefficient supplied from the quantization unit 6 in the same quantization step as the quantization step in the quantization unit 6, and inversely orthogonally transforms the resulting transform coefficient. It supplies to the part 10. The inverse orthogonal transform unit 10 performs an inverse orthogonal transform process on the transform coefficients from the inverse quantization unit 9, thereby decoding the original intra coded picture or performing a predicted picture from the original inter coded picture. The subtracted difference image is decoded and supplied to the frame memory 11.

The frame memory 11 stores the decoded intra coded picture. In the frame memory 11, the inter coded picture is obtained by decoding the difference picture from the motion predictor / compensator 12 by adding the predicted picture subtracted from the inter coded picture to obtain the difference picture. Decode and remember.

In the motion predictor / compensator 12, a predictive image is generated by using the picture stored in the frame memory 11 as a reference image as described above.

Next, FIG. 4 shows the structure of an example of the image decoding apparatus which decodes the encoded data obtained by the image encoding apparatus of FIG.

The coded data is supplied to the accumulation buffer 21 and temporarily stored. The reversible decoding (VLC DEC) unit 22 appropriately reads the coded data stored in the storage buffer 21, and properly separates the information contained in the coded data, for example, variable length decoding, arithmetic decoding, or the like. Do it. The reversible decoding device 22 supplies the quantization coefficients obtained by variable length decoding, calculation decoding, and the like to the inverse quantization unit 23, and supplies the motion vectors to the motion prediction / compensation unit ( 27).

The inverse quantization unit 23 inversely quantizes the quantization coefficients from the reversible decoding unit 22 to obtain transform coefficients and supply them to the inverse orthogonal transform unit 24. The inverse orthogonal transform unit 24 performs inverse orthogonal transform processing such as an inverse discrete cosine transform on the transform coefficients from the inverse quantization unit 23, and decoded images of the resulting intra coded pictures or inter coded pictures. The decoding result of the difference image is supplied to the adder 25 in units of macroblocks.

When the output of the inverse orthogonal transform unit 24 is a decoded picture of an intra coded picture, the adder 25 supplies the picture which is the decoded picture to the frame memory 26 and the screen rearrangement buffer 28. In addition, when the output of the inverse orthogonal transform unit 24 is a decoding result of the difference image of the inter-encoded picture, the adder 25 supplies the predictive image supplied from the motion prediction / compensation unit 27 to the decoding result. Is added to decode the inter coded picture, and the picture, which is the resulting decoded picture, is supplied to the frame memory 26 and the screen rearrangement buffer 28.

The frame memory 26 stores the decoded picture (picture) from the adder 25 as a reference picture, and the motion predictor / compensator 27 reversible the reference picture stored in the frame memory 26. By performing motion compensation according to the motion vector supplied from the decoder 22, a predictive image is generated and supplied to the adder 25. FIG.

On the other hand, the screen rearrangement buffer 28 temporarily stores the picture which is the decoded image from the adder 25, and rearranges the order of the pictures in the display order. In this case, since the picture stored in the screen rearrangement buffer 28 is a picture of a 24 영화 movie film, the screen rearrangement buffer 28 selects a picture of the 24 영화 movie film in H.264 /. D / A (Digital / Analog) conversion unit 29, which is a 30-Hz NTSC system picture that is obtained by 2-3 pulldown according to pic_struct described later, which is multiplexed (included) by the encoded data of AVC, Supplies).

In the D / A converter 29, a picture (picture signal) from the screen rearrangement buffer 28 is D / A converted from a digital signal to an analog signal and supplied to a display (not shown) for display.

By the way, the picture coding apparatus of FIG. 3 is supplied as an encoding target with a 30 Hz NTSC system picture obtained by 2-3 pull down of a picture of a 24 GHz movie film.

Currently, the picture supplied to the picture coding apparatus is shown in the second from the top of FIG. 5, which is obtained by, for example, 2-3 pulling down the picture of the 24 ㎐ movie film shown first from the top of FIG. 5. As described above, in the case of a 30 ms NTSC system picture having 2-3 rhymes, the 2-3 detection unit 2 (FIG. 3) basically determines that the pictures are all sequentially scanned pictures.

Here, in Fig. 5 (also in Figs. 13 to 15, 22, and 23 described later), a rectangle indicates a picture of a frame or field. In addition, the rectangle without a horizontal line represents the picture of a frame, and the rectangle with a horizontal line represents the picture of a field.

However, as shown in the third from the top of FIG. 5, the picture of the NTSC system of 30 ms with the 2-3 rhymes maintained in the second from the top of FIG. 5 is cut and edited. When three rhymes become pictures of an irregular sequence and are supplied to the picture coding apparatus, the 2-3 detection unit 2 generates a picture in which duplicate determination is made that the picture is both a picture of interlaced scan and a picture of sequential scan.

That is, in FIG. 5, as shown in the second from the top, the picture of each frame of the 24 영화 movie film shown first from the top has three fields, two fields, and three fields of the 30 SC NTSC system picture. ,… Is being converted to In addition, the third and fourth fields from the beginning of the 30-second NTSC system picture shown second from the top are cut, and the 30-second NTSC system picture shown third from the top is formed.

In this case, both the first and second fields of the 30-Hz NTSC-type picture (the NTSC-type picture having a frame rate of 30 Hz (field rate of 60 Hz)) shown in the third from the top are both movie films. Since it is obtained from the first frame of, it is determined as a picture of sequential scanning. In addition, since the 3rd and 4th fields of the 30 ms of NTSC system picture shown from the top are obtained from the 3rd and 4th frames of a movie film, respectively, it is determined that it is an interlaced picture. In addition, since all the 4th-6th fields of the 30-microsecond NTSC system picture shown from the top are obtained from the 3rd frame of a movie film, it is determined that it is a picture of progressive scanning.

As described above, the fourth field of the 30-second NTSC system picture shown third from the top is repeatedly determined to be an interlaced picture or a sequential scan picture.

Irregularities of 2-3 rhymes that cause such overlapping judgments are reset, except for editing, for example, reset of 2-3 rhymes by switching of film rolls that are performed when 2-3 pictures of a movie film are pulled down. It also occurs due to the misdetection of 2-3 rhymes due to noise included in the picture of the movie film to be -3 pulldown.

By the way, as in H.264 / AVC, a syntax (syntax) designation method for identifying whether the original picture to be encoded is a picture of sequential scanning or interlaced scanning is specified on the decoding side. In the case of using a non-encoding method, since the encoded picture is not accompanied with identification information indicating whether the scanning method of the picture is interlaced scanning or sequential scanning, whether the picture is a picture of interlaced scanning for each picture. There was no way to determine if the picture was a sequential scan. For this reason, even when a picture is a picture of sequential scanning, it may be output as a field to be interlaced on the decoding side, and in this case, there is a problem that the vertical resolution of one field is lost.

This invention is made in view of such a situation, and is made so that the picture of a sequential scan can be displayed so that the vertical resolution may not be impaired.

An encoding device of the present invention is an encoding device for encoding a picture, comprising encoding means for encoding the picture, picture determination means for determining whether the picture is a picture of interlaced scanning or sequential scanning, and the picture. Flag set means for setting the determination result in the determination means to a scanning flag indicating a scanning method of a picture and information indicating the display method of the picture to set display method indication information indicating a display method of a coded picture of interest And information set means, multiplexing means for multiplexing and outputting the encoding result of the picture, the scanning flag and the display mode indication information, wherein the scanning flag is information for calculating a clockTimestamp, and the encoding means is a picture. Is determined to be a duplicate, the result of the determination that the picture is an interlaced scan picture It is characterized by giving priority to one of the determination results called a progressive scan picture.

An encoding method of the present invention includes an encoding method for encoding a picture, the encoding step of encoding the picture, a picture determination step of determining whether the picture is a picture of interlaced scanning or sequential scanning, and the picture. A flag set step of setting the determination result in the determination step to a scanning flag indicating a scanning method of a picture, and information indicating a display method of the picture to set display method indication information indicating a display method of a coded picture of interest. And a multiplexing step of multiplexing and outputting an information set step, an encoding result of the picture, and the scanning flag and display method indication information for each picture, wherein the scanning flag is information for calculating a clockTimestamp, and a picture is duplicated. If it is determined to be, the picture is sequentially with a determination result of an interlaced scan picture. It characterized in that that first one of the determination result that four picture.

In the first program of the present invention, a picture determination step of determining whether a picture is a picture of interlaced scanning or sequential scanning and a result of the determination in the picture determination step are set as a scan flag indicating a picture scanning method. An information set step for setting a flag set step to be performed, information indicating a picture display method as display method indication information indicating a picture display method, a picture encoding result, a scan flag for each picture, and display method indication information It is characterized in that it comprises a multiplexing step of multiplexing the output.

A program recording medium in which a first program of the present invention is recorded is a program recording medium in which a program for causing a computer to perform encoding processing for encoding a picture is encoded, the encoding step of encoding the picture, and the picture, A picture determination step for determining whether the picture is interlaced or sequential scanning, a flag set step for setting a result of the determination in the picture determination step as a scan flag indicating a picture scanning method, and display of the picture An information set step of setting the information indicating the method to display method indication information indicating a display method of a coded picture of interest, the encoding result of the picture, and the scanning flag and display method indication information for each picture, and outputting the multiplexed information; And a multiplexing step, wherein the scan flag yields a clockTimestamp. For information, and is characterized in that the picture is a case where it is determined as a duplicate, and the picture is a recorded program, comprising a first one of the determination result that the determination result and the progressive picture, called interlaced picture.

The data recording medium of the present invention is characterized in that a scanning flag indicating a scanning method for each picture and coded data obtained by multiplexing display method indication information indicating a display method for each picture are recorded.

The data structure of the present invention is characterized in that the scanning flag indicating the scanning method for each picture and the display method indication information for indicating the display method for each picture are multiplexed.

The decoding device according to the present invention is a decoding device for decoding coded data obtained by encoding a picture, the scanning flag being information for calculating the clockTimestamp by indicating the encoding result of the picture and the scanning method for each picture, and the encoding target picture. When the display method indication information indicating the display method is multiplexed and the picture is determined to be redundant, the picture data is decoded by giving priority to one of a determination result of an interlaced scan picture and a determination result of a sequential scan picture. Recognition means for recognizing which picture is interlaced or sequential scanning based on decoding means, the scanning flag and the display mode indication information of the picture, and the picture is recognized as a picture for interlaced scanning In this case, the frame is interpolated to generate and output a frame picture, and the picture is in order. When it is recognized as a picture to be scanned, the frame picture output means for outputting the picture as a frame picture and the frame picture output from the frame picture output means are repeatedly output as many times as the number corresponding to the display mode indication information. It is characterized by including a repeat output means.

The decoding method according to the present invention is a decoding method for decoding coded data obtained by encoding a picture, the scanning flag being information for calculating the clockTimestamp by indicating the encoding result of the picture and the scanning method for each picture, and the encoding target picture. When the display method indication information indicating the display method is multiplexed and the picture is determined to be redundant, the picture data is decoded by giving priority to one of a determination result of an interlaced scan picture and a determination result of a sequential scan picture. A recognition step of recognizing whether the picture is an interlaced scan or a sequential scan based on a decoding step, the scanning flag of the picture, and display mode indication information; and wherein the picture is recognized as a picture for interlaced scanning. In this case, the picture is generated by field interpolation to generate and output a frame picture, and the picture is If it is recognized as a picture of difference scanning, the frame picture output step of outputting the picture as a frame picture and the frame picture output in the frame picture output step are repeatedly output as many times as the number corresponding to the display mode indication information. It characterized in that it comprises a repeating output step.

The second program of the present invention includes a decoding step of decoding a coded result of a picture, a scan flag indicating a scanning method for each picture, and coded data multiplexed with display method instruction information indicating a display method for each picture, and a picture. A recognition step of recognizing which picture is interlaced scanning or sequential scanning based on the scan flag and the display system indication information of the field; and when the picture is recognized as a picture for interlaced scanning, the field is interpolated. In this way, when a frame picture is generated and output, and the picture is recognized as a sequential scanning picture, the frame method output step for outputting the picture as a frame picture and the frame picture output in the frame picture output step are displayed. And a repeating output step of repeatedly outputting the number of times corresponding to the information.

A program recording medium in which a second program of the present invention is recorded is a program recording medium in which a computer is executed to decode a coded data obtained by encoding a picture. When a scan flag that is a scan method for each picture and information for calculating a clockTimestamp and display method instruction information indicating a display method of a coded picture of interest are multiplexed and the picture is determined to be redundant, the picture is referred to as an interlaced scan picture. The picture is subjected to interlaced scanning or progressive scanning based on a decoding step of decoding the coded data prioritized by one of the determination result and the determination result called a sequential scanning picture, and the scanning flag and the display method indication information of the picture. A recognition step of recognizing which picture is the picture; and the picture is interlaced A frame picture output step of generating and outputting a frame picture by field interpolation when the picture is recognized as a private picture, and outputting the picture as a frame picture when the picture is recognized as a picture of sequential scanning; And a repetitive output step of repeatedly outputting the frame picture output in the frame picture output step as many times as the number corresponding to the display mode indication information.

A reproducing apparatus of the present invention is a reproducing apparatus for reproducing coded data obtained by encoding a picture, the scanning flag being information for calculating a clockTimestamp by indicating the encoding result of the picture, a scanning method for each picture, and a coded picture of interest. When the display method indication information indicating the display method is multiplexed and the picture is determined to be redundant, the picture data is decoded by giving priority to one of a determination result of an interlaced scan picture and a determination result of a sequential scan picture. Recognition means for recognizing which picture is interlaced or sequential scanning based on decoding means, the scanning flag and the display mode indication information of the picture, and the picture is recognized as a picture for interlaced scanning In this case, the frame is interpolated to generate and output a frame picture, and the picture is in order. When it is recognized as a picture to be scanned, the frame picture output means for outputting the picture as a frame picture and the frame picture output from the frame picture output means are repeatedly output as many times as the number corresponding to the display mode indication information. It is characterized by including a repeat output means.

In the encoding device and the encoding method, and the first program and the first program recording medium of the present invention, it is determined whether the picture is a picture of interlaced scanning or sequential scanning, and the determination result is a picture scanning method. Is set to a scan flag indicating. In addition, information indicating a picture display method is set as display method indication information indicating a picture display method, and the picture encoding result, scan flag and display method indication information for each picture are multiplexed and output.

In the data recording medium and the data structure of the present invention, a scanning flag indicating a scanning method for each picture and display method instruction information for indicating a display method for each picture are multiplexed.

In the decoding device and the decoding method, and the second program and the second program recording medium of the present invention, a display method instruction for instructing a picture encoding result, a scan flag indicating a scanning method for each picture, and a display method for each picture The encoded data multiplexed with information is decoded. Based on the scanning flag of the picture and the display mode indication information, when the picture is recognized as a picture of interlaced scanning or sequential scanning, and the picture is recognized as a picture for interlaced scanning, the picture is field interpolated. As a result, a frame picture is generated and output. In addition, when a picture is recognized as a picture of sequential scanning, the picture is output as a frame picture. Further, the frame picture is repeatedly output as many times as the number corresponding to the display mode indication information.

In the reproduction device of the present invention, encoded data obtained by multiplexing a result of encoding a picture, a scan flag indicating a scanning method for each picture, and display method indication information indicating a display method for each picture are decoded. Based on the scanning flag of the picture and the display mode indication information, when the picture is recognized as a picture of interlaced scanning or sequential scanning, and the picture is recognized as a picture for interlaced scanning, the picture is field interpolated. As a result, a frame picture is generated and output. In addition, when a picture is recognized as a picture of sequential scanning, the picture is output as a frame picture. Further, the frame picture is repeatedly output as many times as the number corresponding to the display mode indication information.

According to the present invention, it becomes possible to display pictures of sequential scanning so as not to impair the vertical resolution thereof.

BRIEF DESCRIPTION OF THE DRAWINGS The figure explaining the picture of a movie film and the picture of NTSC system.

2 illustrates 2-3 pulldown.

3 is a block diagram showing a configuration of an example of a conventional picture coding apparatus.

4 is a block diagram showing a configuration of an example of a conventional image decoding device.

FIG. 5 is a view for explaining the case where the picture to be encoded is determined to be both a picture of interlaced scanning and a picture of progressive scanning;

6 is a block diagram showing a configuration example of an embodiment of an image processing system to which the present invention is applied.

Fig. 7 is a diagram showing a table of pic_structs defined in H.264 / AVC.

8 is a diagram illustrating a table of ct_type specified in H.264 / AVC.

9 is a block diagram illustrating a configuration example of a picture encoding apparatus 101.

10 is a flowchart for describing a process of the image encoding device 101.

11 is a block diagram illustrating a configuration example of an image decoding device 102.

12 is a flowchart for describing processing of the image decoding device 102.

FIG. 13 is a diagram for explaining processing of the image coding apparatus 101 and the image decoding apparatus 102. FIG.

14 is a diagram for explaining processing of the image coding apparatus 101 and the image decoding apparatus 102. FIG.

FIG. 15 is a diagram for explaining processing of the image encoding device 101 and the image decoding device 102. FIG.

FIG. 16 is a diagram illustrating syntax of user data registered by ITU-T Recommendation T.35 SEI as defined in H.264 / AVC. FIG.

FIG. 17 is a diagram illustrating syntax of user data unregistered SEI specified in H.264 / AVC. FIG.

FIG. 18 is a diagram illustrating syntax of scan_information (). FIG.

Fig. 19 is a block diagram showing an example of the configuration of an image processing system which encodes and decodes a picture of 24 sequential scanning as an encoding target, and displays by 60 sequential scanning;

20 is a block diagram illustrating a configuration example of a picture coding apparatus 201.

21 is a block diagram illustrating a configuration example of an image decoding device 202.

22 is a diagram for explaining the control of the display image output buffer 211 by the output control unit 212. FIG.

23 is a diagram for explaining the control of the display image output buffer 211 by the output control unit 212. FIG.

24 is a block diagram showing a configuration example of an embodiment of a computer to which the present invention is applied.

<Code description>

S1: Coding of the picture of interest

S11: Interlaced or sequential injection?

S12: Scan flag = "sequential scan"

S13: pic_struct = 0, 3, 4, 5, or 6

S14: Multiplexing the scan flag and pic_struct with data of the picture of interest

S15: Scan flag = "interlaced scan"

S16: Field picture or frame picture?

S17: pic_struct = 1 or 2

S18: pic_struct = 3 or 4

S21: Decoding the Attention Picture

S31: Retrieve scan flag and pic_struct of the picture of interest

S32: pic_struct?

S33: Recognize the picture of interest as a picture of sequential scanning

S34: Store the attention picture as it is as a frame picture

S35: Output frame picture a number of times corresponding to pic_struct

S36: size conversion

S37: Recognize the picture of interest as a picture of interlaced scanning

S38: Field interpolate the picture of interest and generate and store a frame picture

S39: frame picture output

S40: Injection flag?

ST1: Repeated output of frame picture twice

ST2: Sequential Scan Conversion by Field Complement from One Field

ST3: Sequential Scan Conversion by Field Complement from One Field

ST4: Repeated output of frame picture 3 times

BEST MODE FOR CARRYING OUT THE INVENTION [

EMBODIMENT OF THE INVENTION Although embodiment of this invention is described below, when the correspondence of the structural requirement described in a claim and the specific example in embodiment of this invention is illustrated, it becomes as follows. This description is for confirming that the specific example which supports the invention described in the Claim is described in embodiment of this invention. Therefore, although described in embodiment of this invention, although it corresponds to a structural requirement and there exists a specific example which is not described here, it does not mean that the specific example does not correspond to the structural requirement. On the contrary, even if a specific example is described here as corresponding to a structural requirement, it does not mean that the specific example does not correspond to a structural requirement other than the structural requirement.

In addition, this description does not mean that all the inventions corresponding to the specific example described in embodiment of invention are described in a claim. In other words, this description is an invention corresponding to an embodiment described in the embodiments of the invention, and the existence of an invention not described in the claims of this application, that is, in the future, divided application or added by correction It does not deny the existence of the invention.

The encoding device of the present invention,

As an encoding device (for example, the image encoding device 101 of FIG. 9) that encodes a picture,

Encoding means (for example, adder 4, orthogonal transform unit 5, quantization unit 6, reversible encoder 7, inverse quantization unit 9, inverse orthogonal transform) for encoding the picture. A unit 10, a frame memory 11, and a motion predictor / compensator 12,

Picture determination means (e.g., the identification unit 111 of FIG. 9 that performs the processing of step S11 of FIG. 10) for determining whether the picture is a picture of interlaced scanning or progressive scanning;

Flag set means (e.g., the identification unit 111 in FIG. 9 which performs the processing of step S12 or S15 in FIG. 10) and sets the result of the determination in the picture determination means to a scanning flag indicating a picture scanning method; ,

Information set means (for example, the identification section of FIG. 9 which performs the processing of step S13, S17 or S18 in FIG. (111)),

Multiplexing means (e.g., the reversible coding unit 7 of FIG. 9) for multiplexing and outputting the result of encoding the picture and the scanning flag and display method indication information for each picture.

And FIG.

The encoding method of the present invention,

An encoding method for encoding a picture,

An encoding step (e.g., step S1 in FIG. 10) for encoding the picture,

A picture determination step (e.g., step S11 of FIG. 10) for determining whether the picture is a picture of an interlaced scan or a progressive scan;

A flag set step (e.g., step S12 or S15 in FIG. 10) for setting the determination result in the picture determination step to a scan flag indicating a picture scanning method;

An information set step (e.g., steps S13, S17, or S18 in FIG. 10) for setting information indicating a display method of the picture to display method indication information indicating a picture display method;

A multiplexing step of multiplexing and outputting the encoding result of the picture and the scanning flag and display method indication information for each picture (for example, step S14 of FIG. 10)

And a control unit.

The program recorded in the program recording medium of the present invention,

An encoding step (e.g., step S1 in FIG. 10) for encoding the picture,

A picture determination step (e.g., step S11 of FIG. 10) for determining whether the picture is a picture of interlaced scanning or progressive scanning;

A flag set step (e.g., step S12 or S15 in FIG. 10) for setting the determination result in the picture determination step to a scan flag indicating a picture scanning method;

An information set step (e.g., steps S13, S17, or S18 in FIG. 10) for setting information indicating a display method of the picture to display method indication information indicating a picture display method;

A multiplexing step of multiplexing and outputting the encoding result of the picture and the scanning flag and display method indication information for each picture (for example, step S14 of FIG. 10)

And a control unit.

The decoding device of the present invention,

As a decoding apparatus (for example, the image decoding apparatus 102 of FIG. 11) which decodes the encoded data which encoded the picture,

Decoding means for decoding the encoded data multiplexed with a result of encoding the picture, a scan flag indicating a scanning method for each picture, and display method instruction information indicating a display method for each picture (for example, FIG. 11) A reversible decoder 22, an inverse quantizer 23, an inverse orthogonal transform unit 24, an adder 25, a frame memory 26, and a motion predictor / compensator 27,

Recognition means for recognizing which picture is interlaced scanning or sequential scanning based on the scanning flag and display mode indication information of the picture (for example, a process of performing steps S33 or S37 in FIG. 12). 11 I / P conversion section 122),

When the picture is recognized as a picture for interlaced scanning, the picture is generated and outputted by field interpolation by field interpolation, and when the picture is recognized as a picture for sequential scanning, the picture is output as a frame picture. Frame picture output means (for example, the I / P converter 122 in FIG. 11 that performs the processing in step S34 or S38 in FIG. 12),

Repetitive output means (e.g., display image output buffer 123 in Fig. 11) for repeatedly outputting the frame picture output from the frame picture output means a number of times corresponding to the display method indication information.

And FIG.

The decoding method of the present invention,

A decoding method for decoding coded data obtained by encoding a picture,

A decoding step of decoding the encoded data multiplexed with a result of encoding the picture, a scan flag indicating a scanning method for each picture, and display method instruction information indicating a display method for each picture (for example, FIG. 12). Step S21),

A recognition step (for example, step S33 or S37 in FIG. 12) for recognizing which picture is interlaced scanning or progressive scanning based on the scanning flag and display mode indication information of the picture;

When the picture is recognized as a picture for interlaced scanning, the picture is generated and outputted by field interpolation by field interpolation, and when the picture is recognized as a picture for sequential scanning, the picture is output as a frame picture. A frame picture output step (for example, step S34 or S38 in FIG. 12);

Repetitive output step (e.g., step S35 in FIG. 12) for repeatedly outputting the frame picture output in the frame picture output step a number of times corresponding to the display mode indication information.

And a control unit.

The program recorded in the program recording medium of the present invention,

A program for causing a computer to execute a decoding process for decoding coded data obtained by encoding a picture,

A decoding step of decoding the encoded data multiplexed with a result of encoding the picture, a scan flag indicating a scanning method for each picture, and display method instruction information indicating a display method for each picture (for example, FIG. 12). Step S21),

A recognition step (for example, step S33 or S37 in FIG. 12) for recognizing which picture is interlaced scanning or progressive scanning based on the scanning flag and display mode indication information of the picture;

When the picture is recognized as a picture for interlaced scanning, the picture is generated and outputted by field interpolation by field interpolation, and when the picture is recognized as a picture for sequential scanning, the picture is output as a frame picture. A frame picture output step (for example, step S34 or S38 in FIG. 12);

Repetitive output step (e.g., step S35 in FIG. 12) for repeatedly outputting the frame picture output in the frame picture output step a number of times corresponding to the display mode indication information.

And a control unit.

The playback apparatus of the present invention,

As a reproducing apparatus (for example, the image decoding apparatus (reproducing apparatus) 102 of FIG. 11) which reproduces the encoded data which encoded the picture,

Decoding means for decoding the encoded data multiplexed with a result of encoding the picture, a scan flag indicating a scanning method for each picture, and display method instruction information indicating a display method for each picture (for example, FIG. 11) A reversible decoder 22, an inverse quantizer 23, an inverse orthogonal transform unit 24, an adder 25, a frame memory 26, and a motion predictor / compensator 27,

Recognition means for recognizing which picture is interlaced scanning or sequential scanning based on the scanning flag and display mode indication information of the picture (for example, a process of performing steps S33 or S37 in FIG. 12). 11 I / P conversion section 122),

When the picture is recognized as a picture for interlaced scanning, the picture is generated and outputted by field interpolation by field interpolation, and when the picture is recognized as a picture for sequential scanning, the picture is output as a frame picture. Frame picture output means (for example, the I / P converter 122 in FIG. 11 that performs the processing in step S34 or S38 in FIG. 12),

Repetitive output means (e.g., display image output buffer 123 in Fig. 11) for repeatedly outputting the frame picture output from the frame picture output means a number of times corresponding to the display method indication information.

And FIG.

EMBODIMENT OF THE INVENTION Hereinafter, embodiment of this invention is described with reference to drawings.

6 shows a structural example of an embodiment of an image processing system to which the present invention is applied.

The image processing system of FIG. 6 is comprised of the image coding apparatus 101, the image decoding apparatus (reproduction apparatus) 102, and the display 103. As shown in FIG.

The picture coding apparatus 101 is supplied with an NTSC picture having a frame rate of 30 Hz, which is obtained by, for example, 2-3 pull-down of a picture of a movie film having a frame rate of 24 Hz. 101 basically encodes a 30 ms NTSC system picture supplied therein according to the H.264 / AVC standard and outputs the resulting encoded data.

In other words, the image encoding apparatus 101 encodes the picture supplied therein in accordance with the H.264 / AVC standard. In addition, the image encoding apparatus 101 determines whether the picture to be encoded is a picture of a scanning system of interlaced scanning or sequential scanning, and sets the determination result to a scan flag indicating a picture scanning system. In addition, the image encoding device 101 obtains information indicating a display method of a picture to be encoded. It is set to display method indication information indicating a picture display method, and the picture encoding result, the scan flag for each picture, and the display method indication information are multiplexed and output as encoded data.

The coded data output by the picture coding apparatus 101 is supplied to and recorded on a recording medium 104 such as an optical disk, a magneto-optical disk, a magnetic disk (including a hard disk), a semiconductor memory, or the like, or It is transmitted via a wired or wireless transmission medium 105 such as the Internet, a telephone line, a wired LAN (Local Area Network), a CATV (Cable Television), or a satellite line, a terrestrial wave or a wireless LAN.

The image decoding device 102 receives and reproduces the encoded data read from the recording medium 104 or transmitted via the transmission medium 105. That is, the image decoding device 102 basically decodes the encoded data based on the standard of H.264 / AVC, for example. In addition, the image decoding device 102 converts the decoded image obtained by decoding the encoded data into an image corresponding to the display 103 at the next stage and supplies it to the display 103.

That is, the display 103 displays an image at a refresh rate of 60 Hz (exactly 59.94 Hz) which is equal to the display rate (field rate) of the NTSC system field by sequential scanning, for example. In the case of a display, the image decoding device 102 converts the decoded image into an image (picture) of a 60-second progressive scan frame and supplies it to the display 103.

Specifically, the image decoding device 102 recognizes which picture is interlaced or sequential scanning based on the scanning flag of the picture which is the decoded image and the display method indication information. When the picture decoding apparatus 102 recognizes that the picture is a picture for interlaced scanning, the picture decoding apparatus 102 generates a picture (frame picture) of the frame by field interpolating the picture. When the picture decoding apparatus 102 recognizes that the picture is a picture of sequential scanning, the picture decoding apparatus 102 sets the picture as a frame picture as it is. Then, the image decoding device 102 repeatedly outputs the frame picture as many times as the number corresponding to the display method instruction information, if necessary, so as to display an image (picture) of the 60-second progressive scan frame (display). To feed.

As a result, the display 103 displays a 60-second progressive scan image (picture).

Next, although the detail of the image coding apparatus 101 and the image decoding apparatus 102 of FIG. 6 is demonstrated, some syntax prescribed | regulated by H.264 / AVC is demonstrated before that.

In H.264 / AVC (ISO / IEC 14496-10), pic_struct and ct_type are described as follows.

pic_struct indicates whether a picture should be displayed as a frame or one or more fields, according to Table D-1. Frame doubling (pic_struct equal to 7) indicates that the frame should be displayed two times consecutively, and frame tripling (pic_struct equal to 8) indicates that the frame should be displayed three times consecutively.

NOTE-Frame doubling can facilitate the display, for example, of 25p video on a 50p display and 29.97p video on a 59.94p display. Using frame doubling and frame tripling in combination on every other frame can facilitate the display of 23.98p video on a 59.94p display.

NumClockTS is determined by pic_struct as specified in Table D-1. There are up to NumClockTS sets of clockTimestamp information for a picture, as specified by clock_timestamp_flag [i] for each set. The sets of clockTimestamp information apply to the field (s) or the frame (s) associated with the picture by pic_struct.

The contents of the clockTimestamp syntax elements indicate a time of origin, capture, or alternative ideal display. This indicated time is computed as

clockTimestamp = ((hH * 60 + mM) * 60 + sS) * time_scale + nFrames * (num_units_in_tick * (1 + nuit_field_based_flag)) + tOffset,

(D-1) in units of clock ticks of a clock with clock frequency equal to time_scale ㎐, relative to some unspecified point in time for which clockTimestamp is equal to 0.Output order and DPB output timing are not affected by the value of clockTimestamp . When two or more frames with pic_struct equal to 0 are consecutive in output order and have equal values of clockTimestamp, the indication is that the frames represent the same content and that the last such frame in output order is the preferred representation. NOTE-clockTimestamp time indications may aid display on devices with refresh rates other than those well-matched to DPB output times.

clock_timestamp_flag [i] equal to 1 indicates that a number of clockTimestamp syntax elements are present and follow immediately. clock_timestamp_flag [i] equal to 0 indicates that the associated clockTimestamp syntax elements are not present. When NumClockTS is greater than 1 and clock_timestamp_flag [i] is equal to 1 for more than one value of i, the value of clockTimestamp shall be non-decreasing with increasing value of i.

ct_type indicates the scan type (interlaced or progressive) of the source material as follows: Two fields of a coded frame may have different values of ct_type. When clockTimestamp is equal for two fields of opposite parity that are consecutive in output order, both with ct_type equal to 0 (progressive) or ct_type equal to 2 (unknown), the two fields are indicated to have come from the same original progressive frame. Two consecutive fields in output order shall have different values of clockTimestamp when the value of ct_type for either field is 1 (interlaced).

Here, Fig. 7 shows a table of pic_struct described in H.264 / AVC, and Fig. 8 shows a table of ct_type.

The table of pic_struct of FIG. 7 is Table D-1 in the above-described H.264 / AVC description, and the table of ct_type of FIG. 8 is a table described in the above description of ct_type of H.264 / AVC. .

According to the above description of H.264 / AVC, pic_struct is information indicating a display method of how to display a picture, and this pic_struct is used as the display method indication information in the image encoding apparatus 101 (FIG. 6). .

According to the table of the pic_struct of FIG. 7, when a picture is indicated to display one frame, 0 is set in the pic_struct of the picture as shown in the leftmost Value column of FIG. Similarly, when a picture is instructed to display the top field or the bottom field, 1 or 2 is set in the pic_struct of the picture, respectively. In addition, when the picture is instructed to display the top field and the bottom field in that order, 3 is set in the pic_struct of the picture, and when the bottom field and the top field are displayed in that order, 4 is set in pic_struct. When a picture is instructed to display the top field, the bottom field, and the repeating top field in that order, 5 is set in the pic_struct of the picture, and the bottom field, the top field, and the repeating bottom field are in that order. When instructing to display, 6 is set in the pic_struct. In addition, when a picture is instructed to display one frame twice or three times, 7 or 8 are set in the pic_struct of the picture, respectively.

Therefore, in H.264 / AVC, when a picture is a field picture, the value of the pic_struct of the picture is limited to either one or two. In addition, when a picture is a frame picture, the value of the pic_struct of the picture is restricted to one of 0, 3 to 8.

On the other hand, according to the above-described H.264 / AVC description, ct-type is information used for calculating clockTimestamp. In addition, ct-type is one of the elements of cloek_timestamp_flag [i] prescribed in H.264 / AVC. clock_timestamp_flag [i] is set to either 0 or 1, and when clock_timestamp_flag [i] is O, the element does not exist in the clock_timestamp-flag [i], and when the clock_timestamp_flag [i] is 1, the elock_timestamp_flag In [i], an element exists.

Here, the argument i of clock_timestamp_flag [i] has an integer value in the range of 0 to NumClockTS. In H.264 / AVC, NumClockTS is determined according to the value that pic_struct takes, as shown in the rightmost column of FIG.

In the picture coding apparatus 101 of FIG. 6, the ct_type in the element of clock_timestamp_flag [0] in which 1 is set is used as a scan flag indicating whether the picture scanning method is interlaced scanning or sequential scanning, rather than calculating clockTimestamp. do.

FIG. 9 shows an example of the configuration of the picture coding apparatus 101 of FIG. 6 that uses pic_struct as the display method indication information and uses ct_type as the scan flag as described above.

In FIG. 9, the same code | symbol is attached | subjected about the part corresponding to the case in the image coding apparatus of FIG. 3, and the description is abbreviate | omitted suitably below. That is, the picture coding apparatus 101 of FIG. 9 denotes the pic_struct and the interlaced scanning / sequential scanning identification unit 111 (hereinafter, referred to as "identification unit" in the drawing), and simply and appropriately, the identification unit 111. Is basically provided similarly to the picture coding apparatus of FIG.

The identification unit 111 is supplied with the detection result of the 2-3 rhymes and the difference information between the fields from the 2-3 detection unit 2. The identification part 111 performs the encoding object output from the screen rearrangement buffer 3 to the adder 4 based on the detection result of the 2-3 rhymes from the 2-3 detection part 2 and the difference information between fields. It is determined in the same manner as in the 2-3 detection unit 2 to determine whether the picture of interest as a picture is a picture of an interlaced scan or a progressive scan. And the identification part 111 sets the determination result to ct_type as a scanning flag, and supplies it to the reversible coding part 7. As shown in FIG.

Here, in the identification part 111, the scanning method of the picture of interest is determined similarly to the 2-3 detection part 2, for example. Therefore, since the determination result of the identification part 111 and the determination result of the 2-3 detection part 2 become the same, the identification part 111 does not determine the scanning method of the picture of interest, but does not determine the 2-3 detection part 2 May be used.

The identification unit 111 also sets a value (the value column in the Value column of FIG. 7) indicating the display system of the picture of interest to the pic_struct.

In H.264 / AVC, as described above, when the picture is a field picture, the value of pic_struct is limited to either 1 or 2, and when the picture is a frame picture, the value of pic_struct is 0, Constrained to any one of 3 to 8.

The identification part 111 sets the value which shows the display system of the picture of interest to pic_struct under such a restriction.

Specifically, when the identification unit 111 determines that the picture of interest is a picture for interlaced scanning, the identification unit 111 indicates that the top picture is to be displayed by the picture of interest and the bottom field according to the display method of the picture of interest. Is set to pic_struct 2 for indicating to display the text, 3 for indicating the top field and the bottom field in that order, or 4 for indicating the bottom field and the top field in that order. That is, in this case, if the picture of interest that is the picture of interlaced scanning is a field picture, one of 1 or 2 is set to pic_struct, and if the picture of interest is a frame picture, one of 3 or 4 is set to pic_struct.

In addition, when the identification unit 111 determines that the target picture is a picture of sequential scanning, O instructing to display a frame by the target picture according to the display method of the target picture, or the top field and the bottom field. 3 indicates to display the bottom field and the top field in that order, or 5 indicates to display the top field, the bottom field, and the top field in that order, or 6 that indicates that the bottom field, the top field, and the bottom field are displayed in that order is set.

As described above, the identification unit 111 sets a value indicating the display system of the picture of interest in the pic_struct of the picture of interest and supplies it to the reversible coding unit 7.

In this case, it is assumed that 7 or 8 (FIG. 7) indicating that two or three frames are respectively displayed by the picture is not set in the pic_struct. However, when fixed_frame_rate_flag specified in H.264 / AVC is 1 and the sequence of the picture to be encoded satisfies the fixed frame rate, 7 or 6 is specified in the pic_struct to specify that the pictures to be sequentially scanned are repeatedly displayed. 8 can be set.

Next, with reference to the flowchart of FIG. 10, the process of the image coding apparatus 101 of FIG. 9 is demonstrated.

In the picture coding apparatus 101, in step S1, the pictures output from the screen rearrangement buffer 3 to the adder 4 are sequentially regarded as the pictures of interest, and the pictures of interest are regarded as the pictures coding apparatus of FIG. In the same manner as in the case of encoding.

Here, as described with reference to FIG. 3, the 2-3 detection unit 2 determines that a pair of two-field pictures or a pair of three-field pictures from the A / D conversion unit 1 are pictures of progressive scanning. In this case, from the group of the picture of the two fields or the group of the picture of the three fields, a picture of sequential scanning is formed and supplied to the screen rearrangement buffer 3. In addition, when the 2-3 detection unit 2 determines that two fields constituting an arbitrary frame are pictures of interlaced scanning, the pictures of the two fields are each field pictures, or, depending on the coding efficiency, It is supplied to the screen rearrangement buffer 3 as a frame picture of one frame composed of the two fields. Therefore, the rate of the picture supplied from the 2-3 detection unit 2 to the screen rearrangement buffer 3 and from the screen rearrangement buffer 3 to the adder 4 is 20 Hz (3 field pictures). In the case where a picture of a sequential scan of one frame is configured).

In the image encoding apparatus 101, in parallel with the encoding process of step S1, in step S11, the detection part 111 detects 2-3 rhythms supplied from the 2-3 detection part 2, and the difference information between fields. Based on this, it is determined whether the picture of interest, which is the picture of the encoding target output from the screen rearrangement buffer 3 to the adder 4, is a picture of a scanning method of interlaced scanning or sequential scanning.

If it is determined in step S11 that the picture of interest is a picture of sequential scanning, the flow advances to step S12, where the identification unit 111 indicates a value indicating "sequential scanning" in ct_type of the picture of interest as the scanning flag (for example, , 0) are set, supplied to the reversible coding unit 7, and the flow proceeds to step S13.

In step S13, the identification part 111 sets the value according to the display method of the picture of interest among 0, 3-6 to the pic_struct of the picture of interest determined as a picture of sequential scanning, and supplies it to the reversible coding part 7. It supplies, and it progresses to step S14.

In step S14, the reversible coding unit 7 multiplexes the coding result of the picture of interest and the ct_type and pic_struct of the picture of interest from the identification unit 111, and stores the encoded data as the multiplexing result as the storage buffer 8. Output through Then, returning from step S14 to S11, the next picture to be encoded is regarded as a picture of interest, and the same processing is performed on the picture of interest.

Here, in step S14, the reversible coding unit 7 sets at least 1 to clock_timestamp_flag [0] of the picture of interest and ct_type of the picture of interest from the identification unit 111 as an element of the clock_timestamp_flag [0]. Multiplex

For clock_timestamp_flag [k] (where k is an integer value of 1 or more) other than clock_timestamp_flag [0], information based on H.264 / AVC may be set, and clock_timestamp_flag [k] = 0 There is no need to set any information.

On the other hand, when it is determined in step S11 that the picture of interest is a picture of interlaced scanning, the flow advances to step S15, where the identification unit 111 indicates a value indicating "interlaced scanning" in ct_type of the picture of interest as the scan flag (example For example, 1) is set, supplied to the reversible coding unit 7, and the flow proceeds to step S16.

In step S16, the identification unit 111 determines whether the picture of interest is a field picture or a frame picture.

In step S16, when it is determined that the picture of interest is a field picture, the flow advances to step S17, and the identification unit 111 determines whether the picture of interest is one or two of pic_struct of the picture of interest that is determined to be a picture of interlaced scanning while being a field picture. Set the value according to the display method of. In addition, in step S17, the identification part 111 supplies the pic_struct of the picture of interest to the reversible coding part 7, and advances to step S14, and the same process as the above-mentioned case is performed hereafter.

In addition, in step S16, when it is determined that the picture of interest is a frame picture, the flow advances to step S18, and the identification unit 111 selects 3 or 4 of pic_struct of the picture of interest that is determined to be a frame picture and interlaced picture. A value according to the display method of the picture of interest is set. In addition, in step S18, the identification part 111 supplies the pic_struct of the picture of interest to the reversible coding part 7, and progresses to step S14 hereafter and the process similar to the above-mentioned case is performed.

As described above, in the image encoding apparatus 101, the determination result of the scanning method of the picture of interest is set to ct_type, and the coding result of the picture of interest and the encoded data multiplexed with ct_type and pic_struct are output.

Next, FIG. 11 shows an example of the configuration of the image decoding device 102 of FIG. 6. 11, the same code | symbol is attached | subjected about the part corresponding to the case in the image decoding apparatus of FIG. 4, and the description is abbreviate | omitted suitably below. That is, in the image decoding apparatus 102 of FIG. 11, the screen rearrangement buffer 121 is provided in place of the screen rearrangement buffer 28, and the screen rearrangement buffer 121 and the D / A. Similarly to the image decoding apparatus of FIG. 4, except that the I / P converter 122, the display image output buffer 123, and the picture frame size converter 124 are newly provided between the converters 29. Consists of.

The screen rearrangement buffer 121 temporarily stores decoded images (pictures) supplied from the adder 25, rearranges them in the display order, and outputs them to the I / P conversion unit 122.

Here, the screen rearrangement buffer 28 in the image decoding apparatus of FIG. 4 uses the NTSC system of 30 ms according to the pic_struct, as described above, the pictures of 24 ms movie film rearranged in the display order. The picture rearrangement buffer 121 simply outputs the picture supplied from the adder 25 to the I / P conversion unit 122 in the display order. The rate of a picture output from the screen rearrangement buffer 121 to the I / P converter 122 is determined by the screen rearrangement buffer 3 from the 2-3 detection unit 2 in the image encoding apparatus 101 of FIG. 9. In addition to the rate of the picture supplied to and supplied from the screen rearrangement buffer 3 to the adder 4, the value is in the range of 20 ms to 30 ms.

As described above, the I / P conversion unit 122 is supplied with the picture (decoded image) from the screen rearrangement buffer 121, and from the reversible decoding unit 22, from the screen rearrangement buffer 121. The ct_type and pic_struct of each picture in are supplied.

The I / P (Interlace / Progressive) converting unit 122 sequentially uses the pictures supplied from the screen rearrangement buffer 121 as the desired pictures, and the ct_type of the pictures of interest supplied from the reversible decoding unit 22 and the like. Based on the pic_struct, it recognizes whether the picture of interest is a picture of interlaced scanning or sequential scanning. In addition, when recognizing the picture of interest as a picture for interlaced scanning, the I / P converter 122 generates a frame picture by field interpolating the picture of interest and outputs it to the display image output buffer 123 for storage. . In addition, when recognizing the picture of interest as a picture of sequential scanning, the I / P converter 122 outputs the picture of interest as a frame picture to the display image output buffer 123 as it is.

In addition, the I / P conversion unit 122 outputs the frame pictures stored in the display image output buffer 123 according to the pic_struct of the picture of interest (the number of times of reading the frame pictures stored in the display image output buffer 123). ).

The display image output buffer 123 stores the frame picture supplied from the I / P converter 122. In addition, the display image output buffer 123 reads the frame picture stored therein only once or repeatedly several times under the control of the I / P conversion section 122, and reads the frame picture into the picture frame size conversion section 124. Supply.

The picture frame size conversion unit 124 converts the picture frame size of the frame picture from the display image output buffer 123 into a picture frame size corresponding to the display screen of the display 103 (FIG. 6), and the D / It supplies to A conversion part 29.

Next, the process of the image decoding device 102 of FIG. 11 will be described with reference to the flowchart of FIG. 12.

In the image decoding apparatus 102, in step S21, the encoded data supplied therein is decoded in the same manner as in the image decoding apparatus of FIG. 4, and the resulting decoded image (picture) is the screen rearrangement buffer 121. It is sequentially stored in. The pictures stored in the screen rearrangement buffer 121 are rearranged in the display order in the screen rearrangement buffer 121, and are sequentially output to the I / P conversion unit 122.

Here, in the decoding process of step S21, the reversible decoding unit 22 separates the ct_type and pic_struct of each picture from the encoded data, in addition to performing the same processing as in the image decoding device of FIG. It supplies to the I / P conversion part 122. FIG.

In addition, the reversible decoding unit 22 acquires ct_type of the element by analyzing clock_timestamp_flag [i] of 1, that is, multiplexed with encoded data, that is, clock_timestamp_flag [0] in the present embodiment. , To the I / P converter 122. In addition, the reversible decoding unit 22 may or may not analyze clock_timestamp_flag [k] (where k is an integer value of 1 or more) other than clock_timestamp_flag [0]. However, the reversible decoding unit 22 ignores the ct_type even when the clock_timestamp_flag [k] other than clock_timestamp_flag [0] is obtained to acquire the ct_type.

In the image decoding device 102, in parallel with the decoding process of step S21, in step S31, the I / P conversion unit 122 uses the picture from the screen rearrangement buffer 121 as the attention picture, and a reversible decoding unit. From ct_type and pic_struct of each picture supplied from (22), ct_type and pic_struct of the picture of interest are obtained, and the flow proceeds to step S32.

In step S32, the I / P conversion unit 122 determines the value of the pic_struct of the picture of interest.

In step S32, the pic_struct of the picture of interest is 0 indicating that the frame is to be displayed by the picture of interest, or 5 indicating that the top field, the bottom field, and the top field are displayed in that order, or the bottom field, If it is determined that 6 indicates that the top field and the bottom field are to be displayed in that order, the flow advances to step S33, and the I / P converter 122 recognizes that the picture of interest is a picture of progressive scanning, and proceeds to step S34. Proceed.

In step S34, the I / P conversion unit 122 supplies the picture of interest, which is a picture of sequential scanning, as it is to the display image output buffer 123 as a frame picture, and proceeds to step S35.

In step S35, the I / P conversion unit 122 controls the display image output buffer 123 to repeatedly read the frame picture stored in the previous step S34 by the number of times corresponding to the pic_struct of the picture of interest. As a result, the frame picture stored in the display image output buffer 123 is repeatedly read as many times as the number of times under the control of the I / P converter 122, and the frame frame size converting unit 124 is read from the display image output buffer 123. ) Is supplied (output).

That is, at present, the pic_struct of the picture of interest is any one of 0, 5, or 6, and if the pic_struct of the picture of interest is O, the frame picture stored in the display image output buffer 123 is read only once and the picture frame size is The converter 124 is supplied. In addition, when the pic_struct of the picture of interest is 5 or 6, in any case, the frame picture stored in the display image output buffer 123 is repeatedly read only three times and supplied to the picture frame size conversion unit 124.

In addition, the encoded data includes the rate information of the field rate of the NTSC system picture coded by the picture coding apparatus 101 of FIG. 9, and the frame from the display image output buffer 123 to the picture frame size converting section 124 is provided. The picture is output at the rate indicated by the rate information. Therefore, here, the frame picture is output from the display image output buffer 123 to the picture frame size converting unit 124 at the NTSC field rate, that is, 60 Hz, which is also the display rate of the display 103. .

After the process of step S35, it progresses to step S36 and the image frame size conversion part 124 converts the image frame size of the frame picture from the display image output buffer 123, and the D / A conversion part 29 Through the display 103 (FIG. 6). Then, returning from step S36 to S31, the next picture supplied from the screen rearrangement buffer 121 to the I / P converter 122 is regarded as the picture of interest, and the same processing is performed in the following with respect to the picture of interest. Is done.

On the other hand, if it is determined in step S32 that the pic_struct of the picture of interest is 1 or 2 indicating that the top picture or the bottom field is indicated by the picture of interest, respectively, the flow advances to step S37, where the I / P converter ( 122 recognizes that the picture of interest is a picture of interlaced scanning, and proceeds to step S38.

In step S38, the I / P conversion unit 122 generates a frame picture by field interpolating a picture of interest that is a picture of interlaced scanning, supplies it to the display image output buffer 123, stores it, and proceeds to step S39. .

In other words, the pic_struct of the picture of interest is either 1 or 2, and therefore the picture of interest, which is a picture of interlaced scanning, is a field picture. The I / P converter 122 generates a frame picture (so-called interlace-to-progressive conversion) by field interpolating a picture of interest, which is a field picture, and supplies it to the display image output buffer 123 for storage. In addition, the method of field interpolation is not specifically limited.

In step S39, the I / P conversion unit 122 controls the display image output buffer 123 to read the frame picture stored in the previous step S38 only once, and thereby the display image output buffer 123. The picture stored in the frame is read only once under the control of the I / P conversion section 122, and is supplied (outputted) from the display image output buffer 123 to the picture frame size conversion section 124.

After the process of step S39, it progresses to step S36 and the image frame size conversion part 124 converts the image frame size of the frame picture from the display image output buffer 123, and the D / A conversion part 29 Through the display 103 (FIG. 6). Then, returning from step S36 to S31, the next picture supplied from the screen rearrangement buffer 121 to the I / P converter 122 is regarded as the picture of interest, and the same processing is performed in the following with respect to the picture of interest. Is done.

On the other hand, in step S32, the pic_struct of the picture of interest is 3 indicating that the top field and the bottom field are displayed in that order by the picture of interest, or indicating that the bottom field and the top field are displayed in that order. When it is determined as 4, the process proceeds to step S40, and the I / P conversion unit 122 determines whether the picture of interest is a picture of sequential scanning or interlaced scanning based on the ct_type (scanning flag) of the picture of interest.

In step S40, when the picture of interest is determined to be a picture of sequential scanning, that is, when the ct_type of the picture of interest indicates "sequential scanning," the process proceeds to step S33, and the I / P conversion unit 122 described above is performed. As described above, the picture of interest is recognized as a picture of progressive scanning, and the flow proceeds to step S34.

In step S34, the I / P conversion unit 122 supplies the picture of interest, which is a picture of sequential scanning, as it is to the display image output buffer 123 as a frame picture, and proceeds to step S35.

In step S35, the I / P conversion unit 122 controls the display image output buffer 123 to repeatedly read the frame picture stored in the previous step S34, the number of times corresponding to the pic_struct of the picture of interest. Accordingly, the frame picture stored in the display image output buffer 123 is repeatedly read out as many times as the number of times under the control of the I / P converter 122, and the frame size converting unit 124 from the display image output buffer 123 is read. ) Is supplied (output).

That is, at present, the pic_struct of the picture of interest is either 3 or 4, and the frame picture stored in the display image output buffer 123 is repeatedly read only two times at any value, and the picture frame size conversion unit 124 Supplied to.

After the process of step S35, it progresses to step S36 and the image frame size conversion part 124 converts the image frame size of the frame picture from the display image output buffer 123, and the D / A conversion part 29 The display 103 is supplied to and displayed on the display 103 (FIG. 6). Then, returning to step S31 from step S36, the same processing is performed on the next picture of interest.

On the other hand, when it is determined in step S40 that the picture of interest is the picture for interlaced scanning, that is, when the ct_type of the picture of interest indicates "interlaced scanning", the process proceeds to step S37, where the I / P converter 122 As described above, the picture of interest is recognized as a picture of interlaced scanning, and the flow advances to step S38.

In step S38, the I / P conversion unit 122 generates a frame picture by field interpolating a picture of interest that is a picture of interlaced scanning, supplies it to the display image output buffer 123, stores it, and proceeds to step S39. .

That is, since the pic_struct of the picture of interest is either 3 or 4, the picture of interest, which is the picture of interlaced scanning, is a frame picture. According to such a frame picture of interlaced scanning, the top field and the bottom field are originally displayed one after another. , Or bottom field and top field are displayed in sequence. In addition, in this embodiment, the display 103 (FIG. 6) is a display which displays the pictures of progressive scanning.

Therefore, the I / P converter 122 interpolates each of the top field and the bottom field of the picture of interest, which is a frame picture, to generate a frame picture. When the pic_struct of the picture of interest is 3, the I / P converter 122 sequentially supplies the frame picture generated from the top field and the frame picture generated from the bottom field to the display image output buffer 123. To remember. In addition, when the pic_struct of the picture of interest is 4, the I / P converter 122 sequentially supplies the frame picture generated from the bottom field and the frame picture generated from the top field to the display image output buffer 123. To remember.

In step S39, the I / P conversion unit 122 controls the display image output buffer 123 to read each of the two frame pictures stored in the previous step S38 only once, and thereby the display image output buffer. The two frame pictures stored in the 123 are read out once in the storage order under the control of the I / P conversion unit 122, and are read from the display image output buffer 123 to the picture frame size conversion unit 124. Supplied (output).

After the process of step S39, it progresses to step S36 and the image frame size conversion part 124 converts the image frame size of the frame picture from the display image output buffer 123, and the D / A conversion part 29 Through the display 103 (FIG. 6). Then, returning from step S36 to S31, the next picture supplied from the screen rearrangement buffer 121 to the I / P converter 122 is regarded as the picture of interest, and the same processing is described below for the picture of interest. Is performed.

As described above, the image decoding apparatus 102 recognizes which picture is interlaced or sequential scanning based on the ct_type and pic_struct of the picture in the I / P conversion unit 122, so that the picture is obtained. When it is recognized as a picture for interlaced scanning, the frame is generated by field interpolation, the frame picture is generated, output to the display image output buffer 123, and stored. When the picture is recognized as a picture for progressive scanning, the picture is stored. The image is output to the display image output buffer 123 and stored as a frame picture. In the image decoding device 102, the frame picture stored in the display image output buffer 123 is repeatedly output as many times as the number corresponding to pic_struct.

Therefore, when the pic_struct of the progressive scan picture is 3 or 4, the pictures of the progressive scan are separated into a top field and a bottom field and output as a field to be interlaced to prevent the vertical resolution of one field from being lowered. Can be.

Here, among the definitions of O to 8 of the pic_struct in Fig. 7 (FIG. 7) in H.264 / AVC, 3 and 4 are conscious of performing the time phase difference display of the field in the display of interlaced scanning, How to display is not specified.

In contrast, in the image decoding apparatus 102, for interlaced frame pictures with a pic_struct of 3 or 4, a frame picture is generated from each of the top field and the bottom field, and each of the two frame pictures is generated only once. Supplied to the display 103. In addition, for a frame picture of sequential scanning having a pic_struct of 3 or 4, the frame picture is repeatedly supplied twice to the display 103. Therefore, in the display 103 which is a display of progressive scanning, the frame picture supplied as mentioned above from the image decoding apparatus 102 can be displayed.

In the image decoding device 102 of FIG. 11, the I / P conversion unit 122 can perform the conversion of the image frame size by the image frame size conversion unit 124. In this case, the image decoding device 102 can be configured without providing the image frame size conversion device 124, and the processing can be further simplified.

Next, with reference to FIG. 13, the process of the image coding apparatus 101 of FIG. 9 and the image decoding apparatus 102 of FIG. 11 is further demonstrated.

For example, an NTSC type picture having a frame rate of 30 Hz (field rate of 60 Hz), obtained by 2-3 pull-down of a picture of a movie film having a frame rate of 24 Hz, is shown first from the top of FIG. 13. It is assumed that it is edited as described above and supplied to the image encoding apparatus 101 as an encoding target.

Here, the mth from the left of the field pictures which are the pictures of the first NTSC system from the top of FIG. 13 is called the mth field.

In the field picture of the first NTSC system from the top of FIG. 13, cut editing is performed between the second field and the third field, and cut editing is also performed between the fourth field and the fifth field.

Currently, in the 2-3 detection unit 2 (and the identification unit 111) of the image encoding apparatus 101 of FIG. 9, the first field and the second field are pictures of sequential scanning of one frame, and the third field and It is assumed that the fourth field is a picture of interlaced scanning, and the fifth to seventh fields are pictures of a sequential scan of one frame.

In the image encoding apparatus 101, a picture of sequential scanning is encoded as a frame picture, and a picture of interlaced scanning is encoded as a picture having a higher coding efficiency among frame pictures or field pictures.

Therefore, the first field and the second field, which are pictures of progressive scanning, are encoded as one frame picture, as shown second from the top of FIG. Similarly, the fifth to seventh fields, which are pictures of sequential scanning, are also encoded as one frame picture, as shown second from the top of FIG. 13.

In the identification unit 111 of the image encoding apparatus 101, ct_type as a scan flag of a frame picture having the first field and the second field as one frame and a frame picture having the fifth to seventh fields as one frame. In all, "sequential scanning" is set.

In addition, since the identification unit 111 needs to display two fields of the first field and the second field with a frame picture having the first field and the second field as one frame, the pic_struct is a frame picture. Is set to 3 or 4 of 0, 3 to 6 (FIG. 7), which can be set to the pic_struct of.

In addition, since the identification unit 111 needs to display three fields of the fifth to seventh fields with a frame picture having the fifth to seventh fields as one frame, the pic_struct is a pic_struct of the frame picture. Is set to 5 or 6 of 0, 3 to 6 (FIG. 7) which can be set to.

On the other hand, although the third and fourth fields which are interlaced pictures are encoded as the picture having the higher coding efficiency among the frame picture or the field picture, if the coding efficiency of the field picture is high, for example, Fig. 13 As shown in the second from the top, the video is encoded as a field picture.

In the identification unit 111 of the picture coding apparatus 101, "interlaced scanning" is set to both the field picture of the third field and ct_type as the scanning flag of the field picture of the fourth field.

In addition, since the identification unit 111 needs to display one field of the third field by the field picture of the third field, the pic_struct can be set to 1 or 2 (Fig. 7) that can be set as the pic_struct of the field picture. Set to). Similarly, since the field picture of the fourth field needs to display one field of the fourth field, the pic_struct is set to 1 or 2 (Fig. 7) which can be set to the pic_struct of the field picture.

In this case, in the image decoding device 102 of FIG. 11, as shown in the third from the top of FIG. 13, the picture is decoded.

That is, 3 or 4 is set in the pic_struct for the frame picture having the first field and the second field as one frame, but since "sequential scanning" is set in the ct_type, the image decoding apparatus 102 uses FIG. 12. As described in the flowchart of FIG. 2, the frame pictures of sequential scanning in which the first field and the second field are set to one frame are repeatedly outputted only twice in correspondence to the pic_struct in which 3 or 4 are set, and the display 103 ( 6).

In addition, since 1 or 2 is set in the pic_struct for the field picture of the third field, the image decoding apparatus 102 performs frame interpolation by field interpolating the field picture of the third field as described in the flowchart of FIG. 12. The picture is generated. The frame picture is output and displayed on the display 103 only once.

Similarly to the field picture of the third field, 1 or 2 is set in the pic_struct with respect to the field picture of the fourth field. Therefore, in the image decoding apparatus 102, the frame picture obtained by field interpolating the field picture of the fourth field is obtained. Only once, it is outputted and displayed on the display 103.

Since a frame picture having the fifth to seventh fields as one frame is set to 5 or 6 in the pic_struct, the image decoding apparatus 102, as explained in the flowchart of FIG. A frame picture of a sequential scan having seven fields as one frame is repeatedly output three times corresponding to the pic_struct in which 5 or 6 is set and displayed on the display 103 (Fig. 6).

In addition, when the encoding efficiency is higher among the frame picture or the field picture, for example, as a frame picture, for the third and fourth fields that are interlaced pictures, the image encoding apparatus 101 generates a first image. The frame picture consisting of the third and fourth fields is encoded.

In this case, in the identification part 111 of the image coding apparatus 101, "interlaced scanning" is set to ct_type as a scanning flag of a frame picture consisting of the third and fourth fields.

In addition, since the identification unit 111 needs to display two fields of the third field and the fourth field by the frame picture composed of the third and fourth fields, the pic_struct is set to the pic_struct of the frame picture. May be set to 3 or 4 of 0, 3 to 6 (FIG. 7) (second from top of FIG. 13).

In this case, in the image decoding device 102 of FIG. 11, the frame picture consisting of the third and fourth fields is decoded as follows. That is, 3 or 4 is set in the pic_struct for the frame picture composed of the third and fourth fields, but since "interlaced scanning" is set in ct_type, the image decoding apparatus 102 uses the flowchart of FIG. As described, two frame pictures are generated by field interpolating each of the top field and the bottom field of the frame picture consisting of the third and fourth fields. The two frame pictures are output and displayed on the display 103 only once, respectively.

As described above, in the display 103, pictures having a frame rate of 60 Hz are displayed by sequential scanning.

In addition, as described with reference to FIG. 5, in the 2-3 detection unit 2 (and therefore also in the identification unit 111), when an arbitrary picture is repeatedly determined to be an interlaced picture or a sequential scanning picture. There can be. In this case, the image encoding apparatus 101 can give priority to either of a determination result called a picture of interlaced scanning and a determination result called a picture of sequential scanning.

That is, FIG. 14 shows the picture coding apparatus 101 and the picture decoding apparatus 102 when giving priority to a determination result of a picture of interlacing scanning when it is determined that the picture of interlacing scanning is both a picture of interlaced scanning and a picture of progressive scanning. It is a figure explaining the process of).

For example, an NTSC picture having a frame rate of 30 Hz (field rate of 60 Hz), obtained by 2-3 pull-down of a picture of a movie film having a frame rate of 24 Hz, is shown first from the top of FIG. It is assumed that it is edited as described above and supplied to the image encoding apparatus 101 as an encoding target.

Here, as in the case of FIG. 13, the m th field from the left of the field picture, which is the picture of the first NTSC system from the top of FIG. 14, is referred to as the m th field.

Currently, in the 2-3 detection unit 2 (and the identification unit 111) of the image encoding apparatus 101, the first field and the second field are pictures of sequential scanning of one frame, and the third field and the fourth field. It is assumed that the pictures of interlaced scanning are determined, and the fourth to sixth fields are respectively determined to be pictures of sequential scanning of one frame.

In this case, it is determined that the fourth field is both a picture of interlaced scanning and a picture of sequential scanning. In the embodiment of Fig. 14, in the case of overlapping determination, a determination result of a picture of interlaced scanning is given priority, and eventually, the first field and the second field are pictures of sequential scanning of one frame, and the third field and the first are made. Four fields are determined as pictures for interlaced scanning, and fifth and sixth fields as pictures for sequential scanning in one frame, respectively.

In the image encoding apparatus 101, the first field and the second field which are pictures of progressive scanning are encoded as one frame picture, as shown second from the top of FIG. Similarly, the fifth and sixth fields, which are pictures of progressive scanning, are also encoded as one frame picture, as shown second from the top of FIG.

In the identification unit 111 of the image encoding apparatus 101, ct_type as a scan flag of a frame picture in which the first field and the second field are in one frame, and a frame picture in which the fifth and sixth fields are in one frame. In all, "sequential scanning" is set.

In addition, since the identification unit 111 needs to display two fields of the first field and the second field with a frame picture having the first field and the second field as one frame, the pic_struct is a frame picture. Is set to 3 or 4 of 0, 3 to 6 (FIG. 7), which can be set to the pic_struct of.

In addition, since the identification unit 111 needs to display two fields of the fifth and sixth fields with a frame picture having the fifth and sixth fields as one frame, the pic_struct is also the pic_struct of the frame picture. 3 or 4 of 0, 3 to 6 (FIG. 7), which can be set to.

On the other hand, although the third and fourth fields which are interlaced pictures are encoded as the picture with the higher coding efficiency among the frame picture or the field picture, if the coding efficiency of the field picture is high, for example, Fig. 14 As shown in the second from the top, the video is encoded as a field picture.

In the identification unit 111 of the image encoding apparatus 101, "interlaced scanning" is set to both ct_type as a scanning flag of the field picture of the third field and the field picture of the fourth field.

In addition, since the identification unit 111 needs to display one field of the third field by the field picture of the third field, the pic_struct can be set to 1 or 2 (Fig. 7) is set. Similarly, since the field picture of the fourth field needs to display one field of the fourth field, the pic_struct is set to 1 or 2 (Fig. 7) which can be set to the pic_struct of the field picture.

In this case, in the image decoding device 102 of FIG. 11, as shown in the third from the top of FIG. 14, the picture is decoded.

That is, 3 or 4 is set in the pic_struct for the frame picture having the first field and the second field as one frame, but since "sequential scanning" is set in the ct_type, the image decoding apparatus 102 uses FIG. 12. As described in the flowchart of FIG. 2, the frame pictures of sequential scanning in which the first field and the second field are set to one frame are repeatedly outputted only twice in correspondence to the pic_struct in which 3 or 4 are set, and the display 103 ( 6).

In addition, since 1 or 2 is set in the pic_struct for the field picture of the third field, the image decoding apparatus 102 performs frame interpolation by field interpolating the field picture of the third field as described in the flowchart of FIG. 12. The picture is generated. The frame picture is output and displayed on the display 103 only once.

Similarly, 1 or 2 is set in the pic_struct for the picture of the fourth field, so that in the image decoding apparatus 102, the frame 103 obtained by field interpolating the field picture of the fourth field is displayed only once on the display 103. The output is displayed.

Since a frame picture having the fifth and sixth fields as one frame is set to 3 or 4 in the pic_struct, the image decoding apparatus 102 includes a frame picture having the first and second fields as one frame. Similarly, a frame picture of sequential scanning using the fifth and sixth fields as one frame is repeatedly outputted only two times corresponding to the pic_struct in which 3 or 4 is set and displayed on the display 103 (Fig. 6).

As described above, in the display 103, pictures having a frame rate of 60 Hz are displayed by sequential scanning.

In addition, when the encoding efficiency of the third and fourth fields, which are pictures of interlaced scanning, is higher than the frame picture or the field picture, for example, as a frame picture, the image coding apparatus 101 generates a first image. The frame picture consisting of the third and fourth fields is encoded.

In this case, in the identification part 111 of the image coding apparatus 101, "interlaced scanning" is set to ct_type as a scanning flag of a frame picture composed of the third and fourth fields.

In addition, since the identification unit 111 needs to display two fields of the third field and the fourth field by the frame picture composed of the third and fourth fields, the pic_struct is set to the pic_struct of the frame picture. 0, 3 to 6 (FIG. 7) which may be set to 3 or 4 (second from top of FIG. 14).

In this case, 3 or 4 is set in the pic_struct of the frame picture composed of the third and fourth fields, but "interlaced scanning" is set in ct_type. Therefore, in the image decoding apparatus 102, the image decoding apparatus 102 described in the flowchart of FIG. As described above, two frame pictures are generated by field interpolating each of the top field and the bottom field of the frame picture consisting of the third and fourth fields. The two frame pictures are output and displayed on the display 103 only once, respectively.

Next, FIG. 15 shows an image encoding apparatus 101 and an image decoding apparatus (1) when giving priority to a determination result of a sequential scanning picture when it is determined that the interlaced picture is both a picture of interlaced scanning and a picture of sequential scanning. 102 is a diagram for explaining the processing.

For example, an NTSC system picture having a frame rate of 30 Hz (field rate of 60 Hz), obtained by 2-3 pull-down of a picture of a movie film having a frame rate of 24 Hz, is shown first from the top of FIG. 15. It is assumed that it is edited as described above and supplied to the image encoding apparatus 101 as an encoding target.

Here, as in FIGS. 13 and 14, the m th field from the left among the field pictures which are the first NTSC picture from the top of FIG. 15 is called the m th field.

The field picture of the 1st NTSC system from the top of FIG. 15 is the same as the field picture shown from the top of FIG. 14, and therefore, the 2-3 detection part 2 (of the image encoding apparatus 101) ( And the identification unit 111, the first field and the second field are pictures of sequential scanning of one frame, the third field and the fourth field are pictures of interlaced scanning, and the fourth to sixth fields are of one frame. It is determined by the pictures of sequential scanning, respectively. In other words, it is determined that the fourth field is both a picture of interlaced scanning and a picture of sequential scanning.

In the embodiment of Fig. 15, in the case of overlapping determination, a determination result called a picture of sequential scanning is given priority, and eventually, the first field and the second field are pictures of one frame of sequential scanning, and the third field is interlaced. As pictures for scanning, the fourth to sixth fields are respectively determined as pictures of sequential scanning of one frame.

In the image encoding apparatus 101, the first field and the second field which are pictures of progressive scanning are encoded as one frame picture as shown second from the top of FIG. Similarly, the fourth to sixth fields, which are pictures of progressive scanning, are also encoded as one frame picture, as shown second from the top of FIG. 15.

In the identification unit 111 of the image encoding apparatus 101, a frame picture having the first field and the second field as one frame and ct_type as the scan flag of the frame picture having the fourth to sixth fields as one frame are used. In all, "sequential scanning" is set.

In addition, since the identification unit 111 needs to display two fields of the first field and the second field with a frame picture having the first field and the second field as one frame, the pic_struct is a frame picture. Is set to 3 or 4 of 0, 3 to 6 (FIG. 7), which can be set to the pic_struct of.

In addition, since the identification unit 111 needs to display three fields of the fourth to sixth fields with a frame picture having the fourth to sixth fields as one frame, the pic_struct is a pic_struct of the frame picture. Is set to 5 or 6 of 0, 3 to 6 (FIG. 7) which can be set to.

On the other hand, since there are no other fields constituting the frame picture, the third field which is the interlaced picture is encoded as a field picture as shown second from the top of FIG. 15.

In the identification unit 111 of the picture coding apparatus 101, "interlaced scanning" is set to ct_type as a scanning flag of the field picture of the third field.

In addition, since the identification unit 111 needs to display one field of the third field by the field picture of the third field, the pic_struct can be set to 1 or 2 (Fig. 7) is set.

In this case, in the image decoding device 102, as shown in the third from the top of FIG. 15, the picture is decoded.

That is, as in the case of Fig. 14, the frame picture of the sequential scan having the first field and the second field as one frame is repeated twice only for the frame picture having the first field and the second field as one frame. The output is displayed on the display 103 (Fig. 6).

In addition, since 1 or 2 is set in the pic_struct for the field picture of the third field, the image decoding apparatus 102 performs frame interpolation by field interpolating the field picture of the third field as described in the flowchart of FIG. 12. The picture is generated. The frame picture is output and displayed on the display 103 only once.

Since a frame picture having the fourth to sixth fields as one frame is set to 5 or 6 in the pic_struct, the image decoding apparatus 102 performs the sequential scanning using the fourth to sixth fields as one frame. The frame picture is repeatedly output three times corresponding to the pic_struct in which 5 or 6 is set and displayed on the display 103 (Fig. 6).

Next, in the above-described embodiment, ct_type in clock_timestamp_flag [0] in which 1 is set is used as the scan flag. In addition, for example, the scan flag is defined in the standard of H.264 / AVC. User-defined information in the user data SEI can be used.

That is, in H.264 / AVC, a syntax called user data SEI, into which a user-defined syntax can be inserted, is prepared. In the user data SEI, two kinds of syntaxes are defined, a user data registered by ITU-T Recommendation T.35 SEI and a user data unregistered SEI. The syntax of the user data registered by ITU-T Recommendation T.35 SEI is shown in FIG. 16, and the syntax of the user data unregistered SEI is shown in FIG. 17, respectively.

Here, in H.264 / AVC, user data registered by ITU-T Recommendation T.35 SEI is described as follows.

User data registered by ITU-T Recommendation T.35 SEI message semantics

This message contains user data registered as specified by ITU-T Recommendation T.35, the contents of which are not specified by this Recommendation | International Standard.

itu_t_t35_country_code shall be a byte having a value specified as a country code by ITU-T Recommendation T.35 Annex A.

itu_t_t35_country_code_extension_byte shall be a byte having a value specified as a country code by ITU-T Recommendation T.35 Annex B.

itu_t_t35_payload_byte shall be a byte containing data registered as specified by ITU-T Recommendation T.35.

The ITU-T T.35 terminal provider code and terminal provider oriented code shall be contained in the first one or more byte s of the itu_t_t35_payload_byte, in the format specified by the Administration that issued the terminal provider code. Any remaining itu_t_t35_payload_byte data shall be data having syntax and semantics as specified by the entity identified by the ITU-T T.35 country code and terminal provider code.

In H.264 / AVC, user data unregistered SEI is described as follows.

User data unregistered SEI message semantics

This message contains unregistered user data identified by a UUID, the contents of which are not specified by this Recommendation | International Standard.

UUID_iso_iec_11578 shall have a value specified as a UUID according to the procedures of ISO / IEC 11578: 1996 Annex A.

user_data_payload_byte shall be a byte containing data havin g syntax and semantics as specified by the UUID generator.

In the user data registered by ITU-T Recommendation T.35 SEI shown in Fig. 16, user-defined information can be inserted for each picture as the element of the itu_t_35_payload_byte, so that the scanning flag can be inserted as the user-defined information. . In addition, when the scanning flag is inserted into itu_t_35_payload_byte, registration of a terminal (provider) based on ITU-T T.35 is required.

In addition, since user-defined information can be inserted for each picture as an element of the user_data_payload_byte in the user data unregistered SEI shown in FIG. 17, a scanning flag can be inserted as the user-defined information. In addition, when inserting a scan flag into user_data_payload_byte, it is necessary to specify the UUID to enable identification of the user data SEI.

FIG. 18 shows an example of syntax of an element when an injection flag is inserted as an element of itu_t_35_payload_byte or user_data_payload.

In Fig. 18, scan_information () is a user-defined element of itu_t_35_payload_byte or user_data_payload, and is composed of one bit of prog_inter_flag as a scan flag and 15 bits of reserved_for_future_use reserved. In the prog_inter_flag as the scan flag, for example, 0 as a value indicating "sequential scan" or 1 as a value indicating "interlaced scanning" is set in the identification unit 111.

The syntax used as the scan flag is not limited to the aforementioned ct_type, user data registered by ITU-T Recommendation T.35 SEI, and user data unregistered SEI. In addition, in the image encoding apparatus 101, the scanning flag can be recorded in the recording medium 104 or transmitted via the transmission medium 105 in a form attached to the encoded data in addition to being embedded in the encoded data. have.

By the way, in the image processing system of FIG. 6, NTSC whose frame rate is 30 Hz (the field rate is 60 Hz) obtained by 2-3 pull-down of the picture of the movie film whose frame rate is 24 Hz in the image coding apparatus 101. FIG. A picture of a system is encoded and encoded, and the resulting encoded data is decoded by a 60-degree sequential scan picture, and the 60-degree sequential scan picture is 60 s. Display is performed on the display 103 which performs display by sequential scanning.

In the picture coding apparatus 101, the 30 kHz NTSC system picture obtained by 2-3 pull-down of a 24 GHz movie film is set as the encoding target. Although it is common for the 30 Hz NTSC system picture obtained by 2-3 pull down to distribute, the picture of 24 GHz movie film is expected to be distributed as it is without being pulled down 2-3.

In this case, a picture of a 24 영화 movie film is input in the image encoding apparatus 101, and encoding is performed on the picture as an encoding target, and the image decoding apparatus 102 obtains the image in the image encoding apparatus 101. The encoded data is decoded into a picture of a 24 영화 movie film.

On the other hand, since the display 103 displays by 60 sequential scanning, the picture of the 24 ㎐ movie film obtained by the decoding in the image decoding device 102 is displayed on the display 103 as it is. Can not.

In addition, in H.264 / AVC, there is no method for determining the display interval of pictures in the case of displaying pictures of 24 sequential scans such as pictures of movie films on the display 103 of 60 sequential scans. .

Therefore, Fig. 19 is a configuration of an image processing system which encodes and decodes 24 sequential scan pictures as encoding objects, and displays 24 sequential scan pictures obtained by the decoding by 60 sequential scanning. An example is shown.

In addition, about the part corresponding to the case in FIG. 6 in the figure, the same code | symbol is attached | subjected and the description is abbreviate | omitted suitably below.

The picture coding apparatus 201 is supplied with pictures of 24 sequential scanning, for example, and the picture coding apparatus 201 basically conforms to the standard of H.264 / AVC, for example. A picture of 24 sequential scanning supplied therein is encoded, and the resulting encoded data is output.

The coded data output by the picture coding apparatus 201 is supplied to the recording medium 104 and recorded, or transmitted through the transmission medium 105, for example.

The image decoding device 202 receives the encoded data reproduced from the recording medium 104 or transmitted via the transmission medium 105, and the encoded data is basically H.264, for example. Based on the / AVC standard, decoding is performed with pictures of 24 sequential scanning. The image decoding device 202 also supplies the display 103 with 60 sequential scanning pictures by controlling the number of outputs of the 24 sequential scanning pictures to the display 103 on the rear end.

As a result, the 60-second progressive scan display 103 displays a 60-second progressive scan picture supplied from the image decoding device 202.

Next, FIG. 20 shows an example of the configuration of the picture coding apparatus 201 of FIG. 19. In addition, the same code | symbol is attached | subjected about the part corresponding to the case in FIG. 3 or FIG. 9 in the figure.

The picture coding apparatus 201 is basically configured similarly to the picture coding apparatus of FIG. 3 except that the 2-3 detection unit 2 is not provided. The picture coding apparatus 201 is supplied with 24 sequential scanning pictures, and the picture coding apparatus 201 encodes the pictures with 24 sequential scanning as in the case of FIG. 3. do.

Next, FIG. 21 shows a configuration example of the image decoding device 202 of FIG. In addition, the same code | symbol is attached | subjected about the part corresponding to the case in FIG. 4 or FIG. 11 in the figure.

In the image decoding apparatus 202, the screen rearrangement buffer 121 of FIG. 11 is provided in place of the screen rearrangement buffer 28, and the display image output buffer 211 and the output control unit 212 are provided. Except as newly installed, it is basically comprised similarly to the image decoding apparatus of FIG.

The display image output buffer 211 temporarily stores a picture supplied from the screen rearrangement buffer 121. Here, in the image encoding device 201 of FIG. 19, encoding is performed using pictures of 24 sequential scanning as encoding objects. Therefore, in the image decoding device 202 shown in FIG. 21, the picture 25 is supplied to the screen rearrangement buffer 121 from the adder 25 and stored in the picture of 24 sequential scanning which is the encoding target in the image coding device 201. The display image output buffer 211 temporarily stores a picture of 24 sequential scans.

The display image output buffer 211 reads the pictures stored therein a number of times according to the control of the output control unit 212, and supplies them to the display 103 (FIG. 19) through the D / A conversion unit 29. do.

That is, the output control unit 212 supplies the frame rate of the decoded picture that is supplied from the adder 25 to the screen rearrangement buffer 121, that is, the frame rate of the picture to be encoded, and the display 103 of sequential scanning. According to the display rate, the number of times of output of the picture stored in the display image output buffer 211 (the number of readings from the display image output buffer 211) is controlled.

Thereby, the picture of sequential scanning of the frame rate which matches the display rate of the display 103 is output from the display image output buffer 211, and is supplied to the display 103 via the D / A conversion part 29. do.

Here, the frame rate information of the frame rate of the picture to be encoded is, for example, multiplexed by the encoded data in the image encoding apparatus 201, and the output control unit 212 of the image decoding apparatus 202 determines the frame. The frame rate of the picture to be encoded is recognized from the rate information. In addition, the information of the display rate of the display 103 is supplied to the output control part 212 from the display 103, for example.

Next, with reference to FIG. 22, the control of the display image output buffer 211 by the output control part 212 is further demonstrated.

As described above, the output control unit 212 controls the number of times of output of the picture stored in the display image output buffer 211 according to the frame rate of the picture to be encoded and the display rate of the display 103 for sequential scanning.

In other words, the frame rate of the picture to be encoded is 24 Hz, and the display rate of the progressive scan display 103 is 60 Hz. In this case, as shown in FIG. 22, the output control unit 212 repeatedly outputs an arbitrary picture twice of the 24 kHz picture stored in the display image output buffer 211, and outputs the next picture three times. In addition, the next picture is repeatedly outputted two times, and the display image output buffer 211 is controlled so that the output of two repetitions and the output of three repetitions are likewise repeated alternately. As a result, the frame rate of the progressively scanned picture read from the display image output buffer 211 is 60 Hz.

In addition, the frame rate of the picture to be encoded may not be 24 Hz or the display rate of the display 103 for progressive scanning may not be 60 Hz.

For example, when the frame rate of the picture to be encoded is 24 Hz and the display rate of the display 103 for sequential scanning is 30 ms, the output control unit 212 can display the frame rate and the display of the sequential scan ( According to the display rate of 103, the number of times of output of the picture stored in the display image output buffer 211 is controlled, for example, as shown in FIG.

That is, the output control unit 212 outputs only the first to third pictures once for every four consecutive pictures of the 24 ㎐ pictures stored in the display image output buffer 211, and the second pictures are 2 times. The display image output buffer 211 is controlled to repeat the repeated output. As a result, the frame rate of the progressively scanned picture read from the display image output buffer 211 is 30 Hz.

In FIG. 23, the first to third pictures of only four consecutive pictures are outputted only once, and the fourth picture is outputted twice. However, the output pattern of the pictures is not limited to this. In short, any three pictures in the four consecutive pictures may be output only once, and the other one picture may be repeatedly outputted twice.

In addition, when the frame rate of the picture to be encoded is 24 Hz and the display rate of the progressive scan display 103 is 30 x A [Hz], which is an integer multiple of 30 Hz (A is an integer of 1 or more), the output control unit 212 ), An arbitrary three pictures of the four pictures are outputted A times for each of four consecutive pictures of the 24 ㎐ picture stored in the display image output buffer 211, and the other one picture is 2xA. The display image output buffer 211 can be controlled to repeat the output repeatedly. As a result, the frame rate of the progressively scanned picture read from the display image output buffer 211 is 30 x A [mm].

Next, the above-described series of processes may be performed by hardware or may be performed by software. When a series of processes are performed by software, the program which comprises the software is installed in a general purpose computer etc.

Therefore, FIG. 24 shows the structural example of one Embodiment of the computer in which the program which performs a series of process mentioned above is installed.

The program can be recorded in advance in the hard disk 305 or the ROM 303 as a recording medium built in a computer.

Alternatively, the program may be stored on a removable recording medium 311 such as a flexible disk, a compact disc read only memory (CD-ROM), a magneto optical (MO) disk, a digital versatile disc (DVD), a magnetic disk, or a semiconductor memory. It can be saved (recorded) temporarily or permanently. Such a removable recording medium 311 can be provided as so-called package software.

The program is not only installed on the computer from the removable recording medium 311 as described above, but also wirelessly transmitted from the download site to the computer via a satellite for digital satellite broadcasting, or from a local area network (LAN) or the Internet. By wired transmission to a computer via a network such as the above, the computer can be installed in the hard disk 305 which is received by the communication unit 308 and embedded therein.

The computer incorporates a CPU (Central Processing Unit) 302. The input / output interface 310 is connected to the CPU 302 via the bus 301, and the CPU 302 is configured by a user with a keyboard, a mouse, a microphone, or the like through the input / output interface 310. When an instruction is input by the input unit 307 being operated or the like, the program stored in the ROM (Read Only Memory) 303 is executed accordingly. Alternatively, the CPU 302 may be a program stored in the hard disk 305, a program transmitted from a satellite or a network, received by the communication unit 308, installed in the hard disk 305, or a drive 309. The program read from the attached removable recording medium 311 and installed on the hard disk 305 is loaded into the RAM (Random Access Memory) 302 and executed. As a result, the CPU 302 performs the processing according to the above-described flowchart or the above-described block diagram. The CPU 302 then outputs the processing result from the output unit 306 formed of a liquid crystal display (LCD), a speaker, or the like through, for example, the input / output interface 310, or the like. The data is transmitted from the communication unit 308 and the hard disk 305 is recorded.

Here, in the present specification, processing steps for describing a program for causing a computer to perform various processes do not necessarily need to be processed in time series in the order described as a flowchart, but are executed in parallel or separately ( For example, parallel processing or object processing).

The program may be processed by one computer or may be distributed by a plurality of computers. In addition, the program may be transmitted to a remote computer and executed.

In the present embodiment, as the picture to be encoded, an NTSC picture having a frame rate of 30 Hz (field rate of 60 Hz) or a picture having a progressive scan of 24 Hz is adopted, but the picture to be encoded is used. It is possible to employ pictures of different frame rates (field rates).

In addition, in the present embodiment, as the display 103, a display having a display rate of 30 Hz or 60 Hz is used as the display 103. However, the display 103 can adopt a display having a different display rate. .

In the present embodiment, basically, encoding and decoding in accordance with H.264 / AVC are performed. However, the present invention is, for example, MPEG, H.26x, or the like in addition to H.264 / AVC. Similarly, orthogonal transforms such as discrete cosine transform, Karuneen and Loewe transform, and other devices that perform encoding / decoding of images using motion compensation can be applied.

In Fig. 12, after determining the pic_struct, the ct_type serving as the scanning flag is determined. First, the ct_type may be determined, and then the pic_struct may be determined.

Claims (31)

An encoding device for encoding a picture, Encoding means for encoding the picture, Picture determination means for determining whether the picture is a picture of interlaced scanning or progressive scanning; Flag set means for setting a result of the determination by said picture determining means to a scanning flag indicating a scanning method of a picture; Information set means for setting information indicating a display method of the picture as display method indication information indicating a display method of a coded picture of interest; Multiplexing means for multiplexing and outputting the coding result of the picture and the scanning flag and the display mode indication information And, The scan flag is information for calculating a clockTimestamp, And when the picture is judged to be redundant, the encoding means gives priority to one of a decision result of an interlaced picture and a decision result of a sequential scan picture. The method of claim 1, The encoding means encodes a picture obtained from the 2-3 pulled down image according to the MPEG4 AVC / H.264 standard. And an encoding device. The method of claim 1, The display method indication information is a pic_struct defined in the H.264 / AVC standard. And an encoding device. The method of claim 1, The scan flag is ct_type specified in the H.264 / AVC standard. And an encoding device. 5. The method of claim 4, The scan flag is ct_type in clock_timestamp_flag [0] defined in the H.264 / AVC standard. And an encoding device. 5. The method of claim 4, The scan flag is a ct_type of clock_timestamp_flag [0] in which 1 is set, as defined in the H.264 / AVC standard. And an encoding device. The method of claim 1, The scan flag is user defined information in user data SEI defined in the H.264 / AVC standard. And an encoding device. An encoding method for encoding a picture, An encoding step of encoding the picture, A picture determination step of determining whether the picture is a picture of interlaced scanning or progressive scanning; A flag set step of setting a result of the determination in the picture determination step to a scan flag indicating a scanning method of a picture; An information set step of setting information indicating a display method of the picture as display method indication information indicating a display method of a coded picture of interest; A multiplexing step of multiplexing and outputting the encoding result of the picture and the scanning flag and display method indication information for each picture / RTI &gt; The scan flag is information for calculating a clockTimestamp, And when the picture is determined to be redundant, the picture is given priority to one of a decision result called an interlaced scan picture and a decision result called a sequential scan picture. delete A program recording medium on which a program for causing a computer to perform encoding processing for encoding a picture is recorded. An encoding step of encoding the picture, A picture determination step of determining whether the picture is a picture of interlaced scanning or progressive scanning; A flag set step of setting a result of the determination in the picture determination step to a scan flag indicating a scanning method of a picture; An information set step of setting information indicating a display method of the picture as display method indication information indicating a display method of a coded picture of interest; A multiplexing step of multiplexing and outputting the encoding result of the picture and the scanning flag and display method indication information for each picture / RTI &gt; The scan flag is information for calculating a clockTimestamp, And when the picture is judged to be redundant, the program records a program characterized in that the picture is given priority to one of a judgment result called an interlaced scan picture and a judgment result called a sequential scan picture. delete delete A decoding device for decoding encoded data obtained by encoding a picture, When the encoding result of the picture, a scanning flag indicating the scanning method for each picture, information for calculating a clockTimestamp, and display method indication information indicating a display method of the coded picture of interest are multiplexed, and the picture is determined to be redundant. Decoding means for decoding the encoded data in which the picture is one of a determination result of an interlaced scan picture and a decision result of a sequential scan picture; Recognition means for recognizing which picture is interlaced scanning or sequential scanning based on the scanning flag and display mode indication information of the picture; When the picture is recognized as a picture for interlaced scanning, the picture is generated and outputted by field interpolation by field interpolation, and when the picture is recognized as a picture for sequential scanning, the picture is output as a frame picture. A frame picture output means for performing Repeat output means for repeatedly outputting the frame picture output from the frame picture output means a number of times corresponding to the display mode indication information Decoding apparatus comprising a. The method of claim 13, The coded data is obtained by encoding a picture obtained from a 2-3 pulldown image according to the H.264 / AVC standard. A decoding device, characterized in that. The method of claim 13, The display method indication information is a pic_struct defined in the H.264 / AVC standard. A decoding device, characterized in that. The method of claim 13, The scan flag is ct_type specified in the H.264 / AVC standard. A decoding device, characterized in that. The method of claim 16, The scan flag is ct_type in clock_timestamp_flag [0] defined in the H.264 / AVC standard. A decoding device, characterized in that. The method of claim 16, The scan flag is a ct_type of clock_timestamp_flag [0] in which 1 is set, as defined in the H.264 / AVC standard. A decoding device, characterized in that. The method of claim 13, The scan flag is user defined information in user data SEI defined in the H.264 / AVC standard. A decoding device, characterized in that. The method of claim 13, The recognition means, When the display mode indication information of the picture indicates that the top field or the bottom field is to be displayed by the picture, the picture is recognized as a picture for interlaced scanning, The display mode indication information of the picture is displayed by the picture in order of displaying a frame, displaying a top field, a bottom field, and a top field in that order, or displaying a bottom field, a top field, and a bottom field in that order. When instructing to do so, the picture is recognized as a picture of sequential scanning, On the basis of the scan flag, when the display mode indication information of the picture indicates that the picture indicates displaying a top field and displaying a bottom field, or displaying a bottom field and displaying a top field, Recognizing whether the picture is an interlaced scan or a sequential scan A decoding device, characterized in that. A decoding method for decoding coded data obtained by encoding a picture, When the encoding result of the picture, a scanning flag indicating the scanning method for each picture, information for calculating a clockTimestamp, and display method indication information indicating a display method of the coded picture of interest are multiplexed, and the picture is determined to be redundant. A decoding step of decoding the coded data in which the picture is a priority of one of a determination result of an interlaced scan picture and a determination result of a sequential scan picture; A recognition step of recognizing which picture is interlaced scanning or sequential scanning based on the scanning flag and display mode indication information of the picture; When the picture is recognized as a picture for interlaced scanning, the picture is generated and outputted by field interpolation by field interpolation, and when the picture is recognized as a picture for sequential scanning, the picture is output as a frame picture. A frame picture output step, A repeating output step of repeatedly outputting the frame picture output in the frame picture output step a number of times corresponding to the display mode indication information Decoding method comprising a. delete A program recording medium on which a program for causing a computer to perform decoding processing for decoding coded data obtained by encoding a picture is recorded. When the encoding result of the picture, a scanning flag indicating the scanning method for each picture, information for calculating a clockTimestamp, and display method indication information indicating a display method of the coded picture of interest are multiplexed, and the picture is determined to be redundant. A decoding step of decoding the coded data in which the picture is a priority of one of a determination result of an interlaced scan picture and a determination result of a sequential scan picture; A recognition step of recognizing which picture is interlaced scanning or sequential scanning based on the scanning flag and display mode indication information of the picture; When the picture is recognized as a picture for interlaced scanning, the picture is generated and outputted by field interpolation by field interpolation, and when the picture is recognized as a picture for sequential scanning, the picture is output as a frame picture. A frame picture output step, A repeating output step of repeatedly outputting the frame picture output in the frame picture output step a number of times corresponding to the display mode indication information And a program recording medium having a program recorded thereon. A reproduction apparatus for reproducing coded data obtained by encoding a picture, When the encoding result of the picture, a scanning flag indicating the scanning method for each picture, information for calculating a clockTimestamp, and display method indication information indicating a display method of the coded picture of interest are multiplexed, and the picture is determined to be redundant. Decoding means for decoding the encoded data in which the picture is one of a determination result of an interlaced scan picture and a decision result of a sequential scan picture; Recognition means for recognizing which picture is interlaced scanning or sequential scanning based on the scanning flag and display mode indication information of the picture; When the picture is recognized as a picture for interlaced scanning, the picture is generated and outputted by field interpolation by field interpolation, and when the picture is recognized as a picture for sequential scanning, the picture is output as a frame picture. A frame picture output means for performing Repeat output means for repeatedly outputting the frame picture output from the frame picture output means a number of times corresponding to the display mode indication information And a reproducing apparatus. 25. The method of claim 24, The coded data is obtained by encoding a picture obtained from a 2-3 pulldown image according to the H.264 / AVC standard. A playback apparatus, characterized in that. 25. The method of claim 24, The display method indication information is a pic_struct defined in the H.264 / AVC standard. A playback apparatus, characterized in that. 25. The method of claim 24, The scan flag is ct_type specified in the H.264 / AVC standard. A playback apparatus, characterized in that. 28. The method of claim 27, The scan flag is ct_type in clock_timestamp_flag [0] defined in the H.264 / AVC standard. A playback apparatus, characterized in that. 28. The method of claim 27, The scan flag is a ct_type of clock_timestamp_flag [0] in which 1 is set, as defined in the H.264 / AVC standard. A playback apparatus, characterized in that. 25. The method of claim 24, The scan flag is user defined information in user data SEI defined in the H.264 / AVC standard. A playback apparatus, characterized in that. 25. The method of claim 24, The recognition means, When the display mode indication information of the picture indicates that the top field or the bottom field is to be displayed by the picture, the picture is recognized as a picture for interlaced scanning, The display mode indication information of the picture is displayed by the picture in order of displaying a frame, displaying a top field, a bottom field, and a top field in that order, or displaying a bottom field, a top field, and a bottom field in that order. When instructing to do so, the picture is recognized as a picture of sequential scanning, On the basis of the scan flag, when the display mode indication information of the picture indicates that the picture indicates displaying a top field and displaying a bottom field, or displaying a bottom field and displaying a top field, Recognizing whether the picture is an interlaced scan or a sequential scan A playback apparatus, characterized in that.

Images