JP3232082B2 - Image decoding method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION The present invention is related <br/> the image decoding how, in particular, those concerning the decoding processing of the coded image signal including information about the display timing of each frame of the image .

[0002]

2. Description of the Related Art In recent years, the multimedia era in which voices, images, and other data are handled in an integrated manner has been entered, and conventional information media, that is, means for transmitting information such as newspapers, magazines, televisions, radios, and telephones to humans. Has become a feature of multimedia. Generally, multimedia means not only characters but also graphics, sounds, and especially images, etc. are simultaneously associated with each other. To make the above-mentioned conventional information media the target of multimedia, the information must be expressed in digital form. Is an essential condition.

However, when the information amount of each of the above information media is estimated as a digital information amount, the information amount per character is 1 to 2 bytes in the case of a character, while the information amount per character is 64 kbits per second in the case of a voice. (Telephone quality), and more than 100 Mbits per second (current television broadcast quality) for moving pictures, it is not realistic to handle such enormous information as it is in a digital form in the above-mentioned information media such as television. . For example, a videophone is an integrated services digital network (ISDN) having a transmission rate of 64 kbps to 1.5 Mbps.
It has already been put into practical use by the Ital Network, but it is impossible to send the video of a television camera directly through ISDN.

Therefore, what is required is an information compression technique. For example, in the case of a videophone, ITU-T
(International Telecommunication Union Telecommunication Standardization Sector) The H.261 standard moving image compression technology is used. In addition, according to the information compression technology of the MPEG1 standard, it is possible to store image information together with audio information in a normal music CD (compact disc).

Here, MPEG (Moving Picture Exper)
ts Group) is an international standard relating to a technology for compressing and expanding an image signal for a moving image.
This is a standard that compresses data to one-half. In addition, since the transmission speed for the MPEG1 standard is mainly limited to about 1.5 Mbps, in MPEG2 standardized to meet the demand for higher image quality, moving image data has a size of 2 to 1 Mbps.
It is compressed to 5 Mbps.

As described above, the M which has already been put to practical use
In the image signal compression / expansion technology corresponding to the PEG1 and PEG2 standards, basically, only a fixed frame rate is used as an image display timing interval for each frame, that is, a frame rate. Therefore, in MPEG2, a flag (frame rate code) transmitted together with the encoded data is used.
, The table designated by the flag is selected from several frame rates (frame rate values) with reference to the table shown in FIG.

At present, ISO (International)
Organization for Standardization)
To be specific, a working group (ISO / IEC JTC1 / SC29 / WG1) who has been working on the standardization of MPEG1 and MPEG2
According to 1), standardization work of MPEG4, which is the latest image coding method, is underway. MPEG4 enables encoding processing and signal operation on an object basis, and realizes new functions required in the multimedia age. At first, MPEG4 aimed at standardization of low bit rate image processing, but the standardization target is now
It has been extended to more general-purpose image processing, including high-bit-rate image processing that also supports interlaced images.

In MPEG4, if a table similar to the MPEG2 table (see FIG. 13) is added to the head of the video object layer (corresponding to the MPEG2 video sequence), the frame rate can be expressed by the table. In MPEG4, since a wide range of image signals from a low-bit-rate image signal to a high-bit-rate high-quality image signal is to be processed, the necessary frame rates are innumerable.
In G4, it is difficult to determine the frame rate using a table as a practical problem.

[0009] Therefore, in MPEG4, in order to support an infinite number of fixed frame rates, and to correspond to an image in which the intervals of the image display timing and the decoding process timing for each frame are variable, they are inserted for each frame. A data structure including the display time data of each frame is adopted.

FIG. 14 shows an example of the data structure of a conventional coded image signal 200. As shown in FIG. An image coded signal 200 having a conventional data structure includes one image (1 in MPEG4).
Frame sequence constituting an image corresponding to one object), and following the header H at the beginning,
Each frame F (0), F (1), F (2),..., F
The code string Sa0, Sa1, Sa2,... Corresponding to (n)
, San are arranged in transmission order. Here, n is a frame sequence constituting one image.
A number corresponding to the transmission order of data of each frame,
The number n of the first frame is set to 0.

In this example, the code sequence Sa0,
.., San, display time data Dt0, Dt1, indicating the display time of the frame.
Dt2,..., Dtn are arranged, and image coded data Cg0, Cg1, Cg are provided following each display time data.
2,..., Cgn are arranged. Since the display time data indicates a relative time with respect to the reference time, as the number of frames constituting the image (the number of frames) increases, the amount of information necessary to represent the display time,
That is, the number of bits of the display time data increases.

On the decoding side of the coded image signal, the display time data is designated based on the display time data Dt0 to Dtn inserted in the code strings Sa0 to San corresponding to the respective frames. At that time, an image is displayed for that frame.

FIG. 15 shows the transmission order and display order of the image coded data corresponding to each frame in a frame sequence constituting one image. Here, n is a number indicating the transmission order as described above, and n 'is a number indicating the display order (the number n' of the first frame is set to 0). Also, frames F (n) (F (0) to F (18))
Are arranged based on the order of frames (that is, transmission order) in the data structure shown in FIG. 14, and the frames F ′ (n ′) (F ′ (0) to F ′ (18)) are
The frames F (n) arranged in the transmission order are rearranged and arranged so that the arrangement order is the display order as shown by the arrow in FIG. Therefore, the frames F (n) and F '(n') associated with the arrows are the same, for example, the frames F (0), F (1), and F (1).
(2) and F (3) are the frames F '(0),
This is the same frame as F '(3), F' (1), and F '(2).

Here, the frames F arranged in the transmission order
(N) Frame F of (F (0) to F (18))
(0) and F (13) are I (Intra-Picture) frames (hereinafter also referred to as I-VOP), and the frame F
(1), F (4), F (7), F (10), F (16)
Is a P (Predictive-Picture) frame (hereinafter P-VO)
Also called P. ), And frames F (2), F (3),
F (5), F (6), F (8), F (9), F (1
1), F (12), F (14), F (15), F (1
7), F (18) is B (Bidirectionally predictive)
-Picture) frame (hereinafter also referred to as B-VOP).

As described above, the transmission order (IPBBPB
BPBBPBBIBBBBB)
(N) (F (0) to F (18)) in display order (IBBPB
The order n ′ when rearranged to (BPBBPBBPBBIBBP) is represented by frame numbers B (n) (B (0) to B (18)) corresponding to each frame F (n). That is, the value of the frame number B (n) represents the number n ′ indicating the display order. Specifically, as shown in FIG. 15, B (0) = 0, B (1) = 3 ..., B (17)
= 16, B (18) = 17. Therefore, here
The cycle L of the I-VOP in the image display is L = 15, and the cycle M of the VOP including the I-VOP and the P-VOP in the image display is M = 3.

Then, the frame number B (n) = n '
Is represented by the following equations (1) to (3) using n. B (n) = n = 0 (when n = 0) (1) B (n) = n + M-1 (when n = M × i + 1) (2) where i and M are integers of 0 or more (0, 1, 2, ...)
It is. B (n) = n-1 (when n is any other value) (3) Here, the above condition (n = 0) corresponds to the first I-VOP, and the above condition (n = M × i + 1) is the first I-
Those corresponding to both I-VOP and P-VOP other than VOP, and the above conditions (when n is any other value) are as follows:
It corresponds to B-VOP.

However, the above formulas (1) to (3) are
This defines the relationship B (n) = n 'between the display order n' and the transmission order n when the code sequence of the frame corresponding to the VOP, P-VOP, and B-VOP is transmitted periodically. When the code sequence of the frame corresponding to each of the I-VOP, P-VOP, and B-VOP is not transmitted at a fixed period, the display order n ′ and the transmission order n are calculated by the above equations (1) to (4). One-to-one correspondence is obtained by another relational expression or method different from (3).

FIG. 16 is a diagram for explaining an example of an image display method in which the interval of image display timing corresponding to each frame is variable. In the figure, t '(n') (t '(1),
t ′ (2), t ′ (3), t ′ (4)...) are the time at which the image of the frame F ′ (n′−1) is displayed and the image of the frame F ′ (n ′). The time interval from the time when the display is performed is shown, and h ′ (1), h ′ (2), and h ′ (3) are frames F ′ (0)
The time at which the image display of the frames F ′ (1), F ′ (2), and F ′ (3) is performed with reference to the time h ′ (0) at which the image is displayed. H (n) (h (1), h (2), h
(3), h (4),...) Are based on time h ′ (0) at which image display of frame F (0) = F ′ (0) is performed. 1), F (2), F (3), F (4),...). Therefore, here, the display time h ′ (n ′) of the frames F ′ (n ′) arranged in the display order is h ′ (n ′) = h ′ (n′−1) + t ′ (n ′) It is. Note that h '(0) is set to h' (0) = 0.

Next, a situation in which an image coded signal having the image data structure shown in FIG. 14 is decoded and an image is displayed will be briefly described with reference to FIG. On the decoding side, when the image coded signal 200 shown in FIG. 14 is input, each frame F (0), F (
(1), F (2),... Coded image data Cg0, Cg
, Cg2,..., And the image display corresponding to each frame F (0), F (1), F (2),. Data Dt0,
Image display time h (0) based on Dt1, Dt2,...
, H (1), h (2),.

In this way, the coded image signal (image coded signal) not only has a fixed image display timing interval (image display cycle) for each frame but also has a variable image display timing. On the decoding side, an image coded signal is decoded and an image is displayed at a predetermined timing.

When the encoded image signal has a fixed image display interval with respect to each frame, the frame F (0) is used similarly to the case where the image display cycle is variable. , F (1), F (2),... Correspond to the display time data Dt0, Dt of each frame.
, Dt2,... Based on the image display time h (0), h
(1), h (2),...

By the way, the frame rate (the number of frames displayed in one second) is the frequency used in television broadcasting, for example, 29.97... Even if it is simply expressed by k (natural number) bits.・ Hz, exactly 30
Values such as 000/1001 Hz cannot be expressed.

Therefore, such a frame rate can be expressed as follows. That is, a predetermined time interval (1 modulo time) is set to, for example, 1 second, and this is set to N (N
The display time of each frame for an image with a variable frame rate and an image with a fixed frame rate, where the minute unit time (1 / N) equally divided is a unit time (1 time tick). Is represented.

More specifically, as shown in FIG. 17A, each frame F '(0), F' (1) ', F' (2),
Images VOP0, VOP1, VOP2, VOP of F '(3)
The display time of No. 3 is represented, for example, by collecting (y / N) y (VOP rate increment) of minute unit times (1 / N) with reference to time X. For example, image VOP
For 0, VOP1, VOP2, and VOP3, the value y is y = y'0, y = y'1, y = y'2, respectively.
y = y′3.

FIG. 17C shows the above-mentioned minute unit time (1 / N).
An image coded signal 200a having a data structure indicating the image display timing of each frame is indicated by (second) and the value y. In this image encoded signal 200a, minute unit time data D indicating N (natural number) corresponding to the minute unit time
k, followed by a frame H (n) (F
Code strings Sbn (Sb0, Sb1, Sb2, ...) corresponding to (0), F (1), F (2), ...) are arranged, and each code string Sbn has a reference time. The display time h (n) (h (0), h (1), h (2),...) measured using the above minute unit time (1 / N) and the number y with reference to x. Display cycle multiplier data Dyn (Dy0, Dy1, Dy
2,...) Are included.

In FIG. 17C, Cgn (Cg0, Cg0
g1, Cg2,...) are the above frames F (n)
(F (0), F (1), F (2),...).

However, as shown in FIG.
P0 is an I-VOP (I frame), VOP1 and 2 are B-
When VOP (B frame) and VOP3 are P-VOP (P frame), as shown in FIG. 17C, in the bit stream as the image encoded signal 200a, I-V
As a code string of frames F (1) and F (2) following a code string of frame F (0) corresponding to OP (VOP0),
Items corresponding to P-VOP (VOP3) and B-VOP (VOP1) are arranged.

[0028]

First, problems in the image signal data structure described with reference to FIGS. 14 to 16 will be described. As described above, when the image encoding signal is obtained by encoding an image signal in which the display interval of the frame is a constant value T, the image display timing h (n) corresponding to each frame is h (n) ) = N ′ × T. Here, n 'is a number indicating the display order, and n' =
B (n). In other words, such an image coded signal whose frame display interval is a constant value T (that is, an coded signal corresponding to an image whose frame rate is fixed) has a time corresponding to the constant display interval on the decoding side. If T is known, the display time h (n) of the n-th frame F (n) in the transmission order is determined by setting the constant display interval T to n '(= B
(N)), the frame F (n) (F (0)
, F (1), F (2),...) Inserted into the image coded signal corresponding to the display time data Dtn (Dt0, Dt1, Dt2,. However, there has been a problem that complicated display processing has to be performed.

Next, the problem of the image signal data structure described with reference to FIG. 17 will be described. As described above, in the data structure of an image signal proposed in the current MPEG4, even if the frame rate is fixed, the value (size) of the frame rate cannot be known unless several frames are decoded. Therefore, there is a problem that it is difficult to simplify a circuit configuration for realizing the actual decoding process.

Briefly, as shown in FIG. 17B, VOP0 is an I-VOP (I frame), VOP1,
2 is B-VOP (B frame), VOP3 is P-VOP
(P frame), in the bit stream as the image encoded signal 200a, as shown in FIG. 17C, a frame F corresponding to an I-VOP (I frame)
Following (0), a frame F (1) corresponding to a P-VOP (P frame) and a frame F (2) corresponding to a B-VOP (B frame) are arranged.
Until the frame F (2) corresponding to the VOP (B frame) is transmitted, the frame display cycle (1 fixed VO)
P increment), that is, in this case, there is a problem that the interval between the display timing of the I-VOP and the display timing of the B-VOP (B frame) to be displayed next is not known.

The present invention has been made in order to solve the above-mentioned problems, and has been made in consideration of the frame rate (for each frame).
Depending on whether the image display cycle is fixed ,
The display processing against the image code No. Cassin, and to obtain <br/> the picture decoding method can be performed satisfactorily by a simple hardware configuration.

[0032]

[0033]

Means for Solving the Problems] image decoding method according to the inventions, the image coded data corresponding to a plurality of frames
An image decoding method for decoding an encoded image signal including
Therefore, the image coded signal is included in the image coded signal.
Whether the image display interval of all the included frames is fixed
A display period identifier indicating whether the image display data is strange, and decoding the image-encoded data to generate image-decoded data, and when the display period identifier indicates that the image display interval is fixed, a predetermined time Obtained by dividing the interval by N (natural number)
Natural number used to represent the size of the minute unit time
The minute unit time data indicating N and the image display interval
Indication that it is M (natural number) times the minute unit time
Extracting the cycle multiplier data from the image encoded signal,
Using the minute unit time data and the display cycle multiplier data
The display interval calculated by the image table of the decoded image data.
Display interval, and the display cycle identifier is the image display interval.
If it is indicated that it is variable, that of the plurality of frames
The display timing data set for each
The display timing indicating the display timing of the plurality of frames.
Extracting and extracting the imaging data from the image coded signal
Time indicated by the displayed display timing data
Is defined as the image display timing of the decoded image data.
It is characterized in that that.

[0034]

[0035]

[0036]

[0037]

Embodiments of the present invention will be described below. Embodiment 1 FIG. FIG. 1A shows a data structure of an image coded signal 100a having a fixed (constant) frame display period according to the first embodiment of the present invention, and FIG. 1B shows the first embodiment of the present invention. 5 shows a data structure of an image encoded signal 100b whose frame display cycle is variable.

The image encoded signal 100a (see FIG. 1A) encodes an image signal having a fixed frame display cycle corresponding to one image (an image corresponding to one object in MPEG4). After the header H at the beginning, each frame F (0), F
, F (n) corresponding to (1), F (2),..., F (n) are arranged in transmission order. Here, n is a number corresponding to the data transmission order of each frame in a frame sequence constituting one image.

In the image coded signal 100a, a display cycle identifier (fixed display cycle identifier) Df indicating that the frame display cycle is fixed and display cycle data Dp indicating the frame display cycle are inserted into the header H. Code strings Sa0, Sa1, Sa2,..., S corresponding to the frames
At the beginning of an, frame number data B0, B1, B2,..., Bn indicating a frame number B (n) corresponding to the display order n ′ of the frame are inserted.
Also, the code strings Sa0, Sa corresponding to each of the above frames
, Sa2,..., And San include image encoded data Cg0, Cg obtained by encoding the image signal of each frame.
, Cg2,..., Cgn.

FIG. 2 is a diagram for explaining an example of an image display method in which the interval of image display timing corresponding to each frame is fixed. In the figure, the same reference numerals as those in FIG. Here, T is a frame display cycle of an image in which the display timing interval of each frame is fixed. In the image coded signal 100a, as shown in FIG. 2, frames F (n) (n = 0, 1, 2,...) Arranged in the transmission order
Is the display time h (n) of the frame F (0).
Assuming that (= h ′ (0)) is h (0) = 0, h (n) = B (n) ×
It is represented by T. Specifically, the display time h (2) of the frame F (2) is h (2) = B (2) × T, and the display time h (3) of the frame F (3) is h (3) = B ( 3) × T, display time h (1) of frame F (1) is h (1) = B (1) × T, display time h (4) of frame F (4) is h (4) = B ( 4) xT.

Accordingly, in the reproduction process of the image coded signal 100a, the decoded image data obtained by decoding the image coded data corresponding to each frame is displayed at the display time h.
(N) are sequentially displayed. Here, the frame number B (n) representing the number n ′ indicating the display order is
As described in the related art, the above equation (1) to (3) is used to determine the transmission order as a function of the number n.

On the other hand, the image coded signal 100b (FIG. 1)
(b) is obtained by encoding an image signal having a variable frame display cycle corresponding to one image (an image corresponding to one object in MPEG4). And each frame F (0), F
(1), F (2),..., F (n) have a structure in which code strings Sb0, Sb1, Sb2,.

In the image coded signal 100b, a display cycle identifier (display cycle variable identifier) Df indicating that the frame display cycle is variable is inserted in the header H, and the frames F (0) and F (1) are inserted. , F (2), ..., F
Code strings Sb0, Sb1, Sb2,... Corresponding to (n)
, At the beginning of Sbn, display time data (display timing data) Dt0 indicating display times h (0), h (1), h (2),..., H (n) at which the frame is displayed. , Dt1,
Dt2,..., Dtn are inserted. Also, the code strings Sb0, Sb1, Sb2,..., Sbn corresponding to each frame in the image coded signal 100b include:
.., And Cgn obtained by encoding the image signal of each frame. The image display by the reproduction processing of the image coded signal 100b is performed in the same manner as the image display of the image coded signal 200 having the conventional data structure shown in FIG.

Next, the function and effect will be described. In the first embodiment of the present invention, as shown in FIG. 1 (a), an image encoded signal 100a having a fixed frame display cycle has a fixed frame display cycle in a header portion of the entire image data. , And display cycle data Dp indicating the frame display cycle, and each frame number B (0), B
(1), B (2),..., B (n) have a data structure in which frame number data B0, B1, B2,. In the image encoded signal 100a having such a data structure, the frame display cycle T is known from the display cycle data Dp, and the frame number data Bn corresponds to what number frame each frame in one image is counted in display order. Therefore, the display time h (n) of each frame F (n) can be uniquely determined by these data Dp and Bn.

On the other hand, as shown in FIG. 1 (b), an image coded signal 100b having a variable frame display cycle has a display cycle variable indicating that the display interval is variable in a header portion of the entire image data. Identifier Df is inserted, and display times h (0), h (1), h (2),.
.., display time data Dt0, Dt1, D indicating h (n)
The data structure has t2,..., Dtn inserted.

Therefore, when reproducing the image coded signal 100b, the image display corresponding to each of the frames F (0) to F (n) is displayed at the display time h () indicated by the display time data Dt0 to Dtn. 0) to h (n).

As described above, by inserting the display cycle identifier Df indicating whether the frame display cycle is fixed or variable into the header portion of the image coded signal, the picture having the variable frame display cycle can also be inserted. Can respond. Furthermore, for an image having a fixed frame display cycle, the display time data Dt0 to Dtn of each frame having a large amount of information is not referred to, but based on the display cycle data Dp and the frame number data Bn having a small amount of information. An image of each frame can be displayed, and the image processing circuit on the decoding side can have a simple circuit configuration.

Hereinafter, the image coded signal 100 as described above will be described.
The encoding process of the image signal for generating a and 100b and the decoding process thereof will be described. FIG. 3 is a diagram showing a flow of the encoding process. First, in the encoding process,
It is determined whether or not the frame display cycle of the image signal corresponding to the input predetermined image is fixed (step S11). If the result of this determination is that the frame display period is fixed, a display period fixed identifier Df indicating that the frame display period is fixed is added to the header H of the bit stream corresponding to the image signal (step S
11a), a number n indicating the transmission order of each frame is used as a counter value, and this counter value n is n = 0
Is set to (step S12a). Subsequently, display cycle data Dp indicating the fixed frame display cycle T is:
It is added to the header H of the bit stream corresponding to the image signal (step S13a).

Next, as the code string Sa0 corresponding to the first frame F (0) in the transmission order of the predetermined image, the corresponding frame number data Bn (= B0) and the coded image data Cgn (= Cg0) Is the header H
(Steps S14a and S15a). Thereafter, it is determined whether or not the processing target frame in the image signal encoding processing is the last frame in the transmission order of the predetermined image (step S16a).
If the frame to be processed is not the last frame, the counter value n is incremented by one (step S1).
7a) The processing in steps S14a to S17a is performed on the subsequent frame F (1).

The processes in steps S14a to S17a are repeated until it is determined in step S16a that the frame to be processed is the last frame.

On the other hand, if the result of determination in step S11 is that the frame display cycle is variable, a display cycle variable identifier Df indicating that the frame display cycle is variable is stored in the bit stream corresponding to the image signal. (Step S11b), a number n indicating the transmission order of each frame is used as a counter value n, and the counter value n is set to n = 0 (step S12b). Next, as a code string corresponding to the first frame F (0) in the transmission order of the predetermined image, the corresponding display time data Dtn (= Dt0) and image coded data Cgn (= Cg0) are sequentially stored in the header. (Steps S13b and S14b). afterwards,
It is determined whether or not the frame to be processed in the image signal encoding process is the last frame in the predetermined image (step S15b). If the frame to be processed is not the last frame, the counter value n is 1
Is incremented by one (step S16b), and for the subsequent frame F (1), the above steps S13b to S1
The processing in 6b is performed.

The processes in steps S13b to S16b are repeated until it is determined in step S15b that the frame to be processed is the last frame. FIG. 4A is a block diagram illustrating a configuration of an image encoding device 1000 as hardware for performing the encoding process according to the first embodiment.

The image encoding apparatus 1000 encodes an input image signal Sg to generate encoded image data Cgn, and a frame display cycle based on the input image signal Sg. A determiner 1131 that determines whether or not the display period is fixed (that is, whether the display period is fixed or variable) and outputs a display period identifier Df indicating whether or not the display period is constant; A display cycle data generator (first data generator) 1132 that generates display cycle data Df indicating a fixed frame display cycle T based on the input image signal Sg.

The image encoding apparatus 1000 generates frame number data Bn indicating the order of each frame in the transmission order (frame number B (n)) based on the input image signal Sg. A display time h of each frame F (n) based on the generator (second data generator) 1133 and the input image signal Sg.
and a display time data generator (third data generator) 1134 for generating display time data Dtn indicating (n).

Further, the image coding apparatus 1000 includes:
Based on the display cycle identifier Df from the determiner 1131, the display cycle data D from the data generator 1132 is displayed.
open / close switch 1141 that switches between a conducting state for conducting p and a blocking state for blocking the display cycle data Dp.
And a selection switch 1 for selecting and outputting one of the frame number data Bn from the data generator 1133 and the display time data Dtn from the data generator 1134 based on the display cycle identifier Df from the determiner 1131.
142.

Then, the image coding apparatus 1000
The display cycle identifier Df from the determiner 1131, the coded data Cgn from the encoder 1110, the display cycle data Dp from the open / close switch 1141, and the output of the selection switch 1142 are multiplexed to form a multiplexed bit stream M1. A multiplexing unit (MUX) 1120 for generating the multiplexed bit stream M1 is used to convert the multiplexed bit stream M1 into an image coded signal 100a having a fixed frame display period or an image coded signal 100b having a variable frame display period.
Is output.

The operation of the image coding apparatus 1000 will be briefly described below. First, the image encoding device 10
When an image signal Sg corresponding to a predetermined image is input to 00, the determiner 1131 determines whether or not the frame display cycle of the image signal Sg is variable, and a display cycle identifier indicating the determination result. Df is output. At this time, based on the image signal Sg, the first to third data generators 1132 to 1134 respectively generate the display cycle data Dp, the frame number data Bn, and the display time data Dtn. The encoder 1110 encodes the image signal Sg and encodes the encoded image data Cgn
Is output as

Then, the display cycle identifier Df and the coded image data Cgn are output to the multiplexer 1120.
At this time, the display cycle data Dp is output to the multiplexer 1120 via a switch 1141 that is opened and closed based on the display cycle identifier Df, and the frame number data Bn and the display time data Dtn are based on the display cycle identifier Df. Switch 11 for selecting one of these data
The signal is output to the multiplexer 1120 via the P.42.

That is, when an image signal having a fixed image display cycle corresponding to each frame F (n) is input as the image signal Sg, the display cycle fixed identifier Df
In addition, display cycle data Dp indicating a cycle of image display corresponding to each frame, and frame number data Bn indicating a front-back relationship of the frame corresponding to each frame.
Are output to the multiplexer 1120. Then, in the multiplexer 1120, the image coded data Cgn, the display cycle fixed identifier Df, the display cycle data Dp, and the frame number data Bn are multiplexed, and the image coded signal 100a
Is output as

On the other hand, when an image signal whose image display cycle corresponding to each frame F (n) is variable is input as the image signal Sg, the display cycle variable identifier D
Together with f, the timing (display time) h (n) at which image display corresponding to each frame is performed, which is set relatively to one or a plurality of required reference times according to each frame described above. The indicated display time data Dtn is output to the multiplexer 1120. Then, the multiplexer 1120
In the above, the coded image data Cgn, the display cycle variable identifier Df, and the display time data Dtn are multiplexed and output as a coded image signal 100b.

Next, a decoding process for decoding the coded image signals 100a and 100b having the data structure of the present embodiment will be described with reference to FIG. First, in the decoding process, the multiplexed bit stream M1 (the image encoded signal 100a or 100
By detecting the display cycle identifier Df in b), it is determined whether or not the display cycle of the image coded signal is fixed (step S21). As a result of this determination, if it is determined that the display cycle is fixed, the count value n corresponding to the order of each frame F (n) in the transmission order is set to 0 (step S21a), and then the image coded signal , Display cycle data Dp indicating the display cycle T is read from the header portion H (step S22a).

Next, frame number data Bn indicating the frame number B (n) is read from the header of each frame (step S23a), and the display time h of each frame is read.
(N) is obtained by a calculation formula h (n) = B (n) × T (step S24a).

Then, decoding processing of the image coded data Cgn corresponding to the frame F (n) is performed, and the frame F (n) is decoded.
The decoded image data corresponding to (n) is displayed at the display time h (n).
(Step S25)
a). Thereafter, it is determined whether or not the processing target frame F (n) is the last frame in the transmission order of the predetermined image (step S26a), and the processing target frame is the last frame in the transmission order of the predetermined image. If it is a frame, the decoding process ends. If this is not the last frame, the count value n is incremented by one (step S27a), and thereafter, the step S23a
Steps 26a to 26a are performed until it is determined in step S26a that the frame to be processed is the last frame.

On the other hand, if it is determined in step S21 that the display cycle is variable, the count value n corresponding to the order of each frame F (n) in the transmission order is set to 0 (step S21b). Thereafter, display time data Dtn indicating the display time h (n) of the frame F (n) is read from the header portion of each frame (step S22b).
On the basis of the display time data Dtn, the frame F (n)
Is determined (step S23b).
Subsequently, decoding processing of the image coded data Cgn corresponding to the frame F (n) is performed, and the decoded image data of the frame F (n) subjected to the decoding processing is displayed at the display time h.
The image data is displayed as (n) (step S2).
4b).

Thereafter, it is determined whether or not the processing target frame F (n) is the last frame in the transmission order of the predetermined image (step S25b). If the processing target frame F (n) is the last frame, decoding is performed. The conversion process ends. On the other hand, if the processing target frame is not the last frame, the count value n is incremented by one (step S26b), and thereafter, the step S22b
Steps S26b to S26b are performed until it is determined in step S25b that the frame to be processed is the last frame. As described above, the coded image signal having the data structure shown in FIGS. 1A and 1B is decoded by the processing procedure shown in FIG.

FIG. 6A is a block diagram showing the configuration of an image decoding device as hardware for performing the decoding process in the first embodiment. This image decoding device 200
Reference numeral 0 denotes a configuration in which reproduction processing including decoding processing and display processing is performed on the multiplexed bit stream M1 as the image encoded signal 100a or 100b output from the image encoding apparatus 1000.

That is, the image decoding apparatus 2000
Are the image coded data Cgn, the display cycle identifier Df, and the display cycle data D from the multiplexed bit stream M1.
p, and a data separator (DEMU) that extracts and outputs frame number data Bn or display time data Dtn.
X) 2110, and a decoder 2120 that decodes the encoded image data Cgn and outputs decoded image data Rg.
And

Further, the image decoding apparatus 2000 is provided with an open / close switch which switches between a conduction state for conducting the display cycle data Dp and a cutoff state for blocking the display cycle data Dp based on the display cycle identifier Df. 214
0, and a selection switch 2150 that selects and outputs one of the frame number data Bn and the display time data Dtn based on the display cycle identifier Df.

Further, the image decoding apparatus 2000 includes:
There is provided a display device 2130 that displays the decoded image data Rg at a predetermined display timing based on the display cycle identifier Df and the outputs of the switches 2140 and 2150.

The operation of the image decoding apparatus 2000 will be briefly described below. First, the image decoding device 20
00, when the multiplexed bit stream M1 from the image encoding apparatus 1000 is input, the data separator 2110
, The display cycle identifier Df and the display cycle data Dp are separated from the multiplexed bit stream M1, and further, from the multiplexed bit stream M1, the image encoded data Cgn, the frame number data Bn, or the display time data Dtn are separated for each frame. Are separated.

The image coded data C of each frame
gn is decoded by the decoder 2120 and output to the display device 2130 as decoded image data Rg. Also,
The display cycle data Dp is supplied to the display device 21 via an open / close switch 2140 that is opened and closed by the display cycle identifier Df.
30 and the frame number data Bn of each frame.
Alternatively, the display time data Dtn is the display cycle identifier Df
Select switch 2 for selecting one of these data
The data is output to the display device 2130 via the display device 150.

In the display device 2130, the image of each frame corresponding to the decoded image data Rg having a fixed display cycle is displayed at a predetermined display timing based on the display cycle data Dp and the frame number data Bn. The image of the frame corresponding to the decoded image data Rg whose display cycle is variable is displayed at a predetermined display timing based on the display time data Dtn.

As described above, in the image signal data structure according to the first embodiment, an image coded signal obtained by coding an image signal is used to determine whether or not the image display processing cycle corresponding to each frame is variable. When the image display cycle for each frame is fixed, the display time data (display number) having a large amount of information (the number of bits) for each frame is displayed when the image display cycle for each frame is fixed. The display processing of the decoded image data Rg can be performed based on the display cycle data Dp and the frame number data Bn having a small information amount (number of bits) without referring to the (timing data) Dtn.

The image coded signal 100a having a fixed display cycle includes display cycle data Dp indicating a cycle T of image display and a frame number B indicating the anteroposterior relationship of frames.
Since the data (frame position data) Bn indicating (n) is included, the image display timing corresponding to each frame is
It can be determined by a simple calculation of T × B (n).

Further, the image coded signal 100b having a variable display cycle has one or more reference times h ′ (0) (see FIG. 2) corresponding to one or a plurality of reference times according to each frame. , The display time data (display timing data) Dtn indicating the display time (display timing) h (n) at which the image display corresponding to each frame is performed.
When the cycle of the image display corresponding to each frame is variable, the display time data Dtn is used similarly to the conventional data structure.
, The image display timing h (n) corresponding to each frame F (n) can be determined.

Further, in the image coding apparatus 1000 according to the first embodiment, the determining unit 1131 for generating the display cycle identifier Df indicating whether the display cycle of the image is variable or not is based on the input image signal. When an image signal having a fixed display cycle is input, the display cycle identifier Df,
Display cycle data Dp indicating the cycle of image display and frame number data Bn indicating the anteroposterior relationship of frames are multiplexed with the image encoded data Cgn and output. When an image signal having a variable image display cycle is input, Display cycle identifier Df
The display time data Dtn indicating the display time h (n) of each frame F (n) is multiplexed with the coded image data Cgn and output, so that the image display cycle for each frame is variable. Is fixed, the data for determining the display timing of each frame is output together with the image coded data Cgn. As a result, the number of bits required to determine the display time when the frame display cycle is fixed can be reduced, and an image with a variable frame display cycle can be displayed in the same manner as before.

Further, in the image decoding apparatus 2000 of the first embodiment, the display cycle identifier Df and the display indicating the image display cycle included in the multiplexed bit stream M1 sent from the image coding apparatus 1000 are described. Period data Dp, frame number data B indicating the context of the frame
n, a data separator 2110 for separating display time data Dtn indicating the display time h (n) of each frame and image encoded data Cgn, and decodes the image encoded data Cgn to output image decoded data Rg. Decoder 2120
And the decoded image data Rg having a fixed display cycle
Is displayed at a predetermined display timing based on the display cycle data Dp and the frame number data Bn, and the image decoding data Rg having a variable display cycle is displayed on the display time data Dt.
Since the display is performed at a predetermined display timing based on n, the decoded image data Rg of each frame can be displayed at the correct display timing regardless of whether or not the frame display cycle is variable.

In the image signal data structure described in the first embodiment, the display cycle identifier Df is placed at the beginning of the image data (multiplexed bit stream), and the frame is placed at the beginning of the frame data (the code sequence of each frame). Number data Bn, display time data Dtn, etc. are inserted,
The display cycle identifier, frame number data, display time data, and the like do not necessarily need to be inserted at the beginning of the corresponding header. The display cycle identifier and the display cycle data are included in the header portion of the image data (image encoded signal). The frame number data, the display time data, and the like may be inserted after the synchronization signal or the like as long as they are inserted into the header portion of the data (code string) corresponding to the frame.

In the first embodiment, the display cycle data Dp is inserted immediately after the display cycle identifier Df. However, the display cycle data Dp is always inserted immediately after the display cycle identifier so as to be continuous with the display cycle identifier. It is not necessary that the display cycle data be inserted after the display cycle identifier in the header portion of the image data.

Further, in the first embodiment, the number n '(= B (n)) indicating the display order in FIG.
Although serial numbers are assigned from the beginning in the display order of image data, it is not necessary to assign serial numbers,
A plurality of numbers from a predetermined head number to a tail number may be periodically assigned as frame numbers.

For example, when a frame number is represented by 4 bits, a number from 0 to 15 is periodically assigned to a frame. In this case, for the display time, h ′ (n ′) = h
It is represented by p ′ (15) + (n ′ + 1) × T. Where h
p ′ (n ′) is the frame number B in the immediately preceding cycle
(N) (= n ') represents the display time, and therefore, in this case, h' (n ') is the frame number B (n) (in the next cycle of hp' (n '). = N '). Note that hp '(1
5) indicates a display time corresponding to the last frame in the immediately preceding cycle.

Further, in the first embodiment, the frame is specified by the frame number data. However, the present invention is not limited to this. If the data specifies the context of the frame, the frame is specified by a predetermined rule. Indicating the context of a frame, data defining the context of a frame with reference to a predetermined table, or the like.

The display time data in the first embodiment indicates relative times with respect to a plurality of reference times. For example, one reference time may be set for a plurality of frames. The display time of the previous frame may be set as the reference time. Further, one or more reference times are set in advance, and based on a certain rule or signal,
The reference time may be referred to to determine the frame display time.

Further, in the first embodiment, the display cycle identifier and the display cycle for setting the display timing of each frame are used as additional data for determining the timing of the reproduction process for each frame on the decoding side. Although the data structure of the image and the image coded signal including the frame number data and the display time data has been described, the data structure of the image coded signal may be replaced by the decoding timing of each frame instead of the display timing of each frame. Additional data for determining the timing, that is, data including a decoding cycle identifier, decoding cycle data, and frame number data or decoding time data may be included. Hereinafter, such a data structure will be described as a modification of the first embodiment. .

(Modification of First Embodiment) The data structure of a modification of the first embodiment is such that the display cycle identifier Df and the display cycle data Dp in the coded image signal 100a of the first embodiment are And display time data Dtn in the coded image signal 100b of the first embodiment is replaced with decoding time data.

Here, the decoding cycle identifier indicates whether or not the cycle of the decoding process for decoding the image coded signal corresponding to each frame is variable. It is inserted as a fixed decoding period identifier into a fixed image encoded signal, and as a variable decoding period identifier into an image encoded signal having a variable decoding period.

The decoding cycle data is data indicating a decoding processing cycle DT corresponding to each frame, and the decoding time data is one corresponding to one or a plurality of reference times corresponding to each frame. This is data indicating a timing (decoding time Dh (n)) at which decoding processing corresponding to each frame is performed, which is set relatively to the frame.

The encoding process for generating an encoded image signal having the data structure of the modification of the first embodiment is shown in FIG.
Steps S11, S11a, S1 in the flow shown in FIG.
The processing in steps 1b, S13a, and S13b can be realized by substituting as follows.

That is, the process of determining the display cycle in step S11 is replaced with the process of determining whether or not the decoding cycle is fixed, and the process of adding the display cycle fixed identifier Df in step S11a is replaced with the decoding cycle fixed identifier Df. Is added to the process of adding the display cycle variable identifier Df in step S11b to the process of adding the decoding cycle variable identifier. Further, the process of adding the display cycle data Dp in step S13a is replaced with the process of adding the decode cycle data, and the process of adding the display time data Dtn in step S13b is replaced with the process of adding the decode time data.

FIG. 4B shows a configuration of an image encoding device 1000a as hardware for performing the encoding process of the modification of the first embodiment. The image coding apparatus 1000a has a constant decoding processing cycle corresponding to a frame based on the input image signal Sg, instead of the determiner 1131 of the image coding apparatus 1000 in the first embodiment. It is determined whether or not the decoding cycle is fixed (that is, whether the decoding cycle is fixed or variable), and the decoding cycle DT is determined.
Is provided with a decision unit 1131a that outputs a decoding cycle identifier DEf indicating whether or not is constant.

The image coding apparatus 1000a
Display cycle data generator 1132 and display time generator 113 in image encoding apparatus 1000 of the first embodiment.
4, instead of the input image signal Sg,
Each frame is determined based on a decoding cycle data generator (first data generator) 1132a that generates decoding cycle data DEp indicating a cycle (fixed cycle) DT of a frame decoding process and the input image signal Sg. And a decryption time data generator (third data generator) 1134a that generates decryption time data DEtn indicating the decryption time. Other configurations are the same as those of the image coding apparatus 1000 according to the first embodiment.

The image coding apparatus 1000 having such a configuration
In a, in the multiplexer (MUX) 1120, the decoding cycle identifier DEf from the decision unit 1131a, the image coded data Cgn from the encoder 1110, the decoding cycle data DEp from the open / close switch 1141, and the selection The output of the switch 1142 is multiplexed, and the multiplexed bit stream M1a is output as an image coded signal with a fixed decoding cycle or an image coded signal with a variable decoding cycle.

On the other hand, the decoding process for decoding the image coded signal having the data structure of the modification of the first embodiment is performed in steps S21, S22a, S22 in the flow shown in FIG.
22b, S23a, S23b, S24a, S24b, S
This can be realized by replacing the process of 25a as follows.

More specifically, the process of determining the display cycle in step S21 is replaced with a process of determining whether the decoding cycle is fixed, and the process of reading the display cycle data Dp indicating the display cycle T in step S22a is performed. Is replaced with the process of reading the decoding cycle data DEp indicating the decoding cycle DT, and the display time h in step S22b is replaced.
The process of reading the display time data Dtn indicating (n)
Decoding time data DEtn indicating the decoding time Dh (n)
Replace with the process of reading.

The frame number data Bn in steps S23a and S24a are read and the display time h
The process of determining (n) is performed based on the decoding cycle data DEp, determining the decoding time Dh (n) corresponding to the image encoded data of each frame that is sequentially input, and determining the decoding time Dh (n) based on the frame number data Bn. The process is replaced with a process for determining the display time h (n) for each frame.

The process of determining the display time h (n) based on the display time data Dtn in step S23b is performed in accordance with the decoding time D (n) based on the decoding time data DEtn.
h (n) is determined and replaced with a process of determining the display time h (n) based on the data DEtn.

Further, the processing of decoding the image coded data Cgn of the frame F (n) and displaying the decoded data at the display time h (n) in step S25a is described above.
The encoded image data Cgn of (n) is decoded at the decoding time Dh (n).
And the process of decoding the image coded data Cgn of the frame F (n) and displaying the same at time h (n) in step S24b is replaced with the process of displaying at time h (n) in step S24b. The image coded data Cgn of F (n) is decoded at a decoding time Dh (n), and is replaced with a process of displaying at a display time h (n).

FIG. 6B shows a configuration of an image decoding device 2000a as hardware for performing the decoding process according to the modification of the first embodiment. The image decoding device 2000a is configured to perform a reproduction process including a decoding process and a display process on the multiplexed bit stream M1a output from the image encoding device 1000a.

That is, the image decoding apparatus 2000a
Is the image decoding device 2000 according to the first embodiment.
Data separator 2110 for extracting and outputting image coded data Cgn, decoding cycle identifier DEf, decoding cycle data DEp, and frame number data Bn or decoding time data DEtn from the multiplexed bit stream M1a. (DEMUX) 2110a.

Further, the image decoding device 2000a
Based on the decoding cycle identifier DEf, a conduction state for making the decoding cycle data DEp conductive and the decoding cycle data D
A first open / close switch 2140a that switches between a blocking state for blocking Ep, a conduction state for conducting the frame number data Bn based on the decoding cycle identifier DEf, and a blocking state for blocking the frame number data Bn. Between the second open / close switch 2150a that switches between the first and second decryption time data DEtn and the second decryption time data DEt based on the decryption cycle identifier DEf.
and a third open / close switch 2160a that switches between a shut-off state for shutting off n.

In the image decoding apparatus 2000,
The decoding cycle data DEp output from the first open / close switch 2140a and the decoding time data DEtn output from the third open / close switch 2160a are supplied to the decoder 2120a and the display device 2130a, and the second open / close switch 2150a Is supplied to only the display device 2130a.

The decoder 2120a decodes the coded image data Cgn having a fixed decoding cycle for each frame at a timing (decoding time Dh (n)) determined based on the decoding cycle data DEp. On the other hand, the image encoding data Cgn having a variable decoding cycle is decoded for each frame at a timing (decoding time Dh (n)) determined based on the decoding time data DEtn.

Further, in the display device 2130a, the image decoding data Rg having a fixed decoding cycle is converted into each frame at a timing (display time h (n)) determined based on the decoding cycle data DEp and the frame number data Bn. The image decoding data Rg having a variable decoding cycle is displayed for each frame at a timing (display time h (n)) determined based on the decoding time data DEtn. . Other configurations are the same as those of the image decoding apparatus 2000 of the first embodiment.

Hereinafter, the operation of the image decoding apparatus 2000a according to the modification of the first embodiment will be briefly described. In the image decoding device 2000a having such a configuration,
When the multiplexed bit stream M1a is input, the data separator 2110a separates the encoded image data Cgn, the decoding cycle identifier DEf, the decoding cycle data DEp, and the frame number data Bn or the decoding time data DEtn.

Then, in the decoder 2120a, when the decoding cycle of the input decoded picture signal is fixed, the picture coded data Cgn is determined based on the decoding cycle data DEp (decoding time Dh ( n)), the decoded image data Rg output from the decoder 2120a is decoded at the timing (display time h (n) determined based on the decoding cycle data DEp and the frame number data Bn. ) Is displayed for each frame.

On the other hand, when the decoding cycle of the input picture decoding signal is variable, the picture coded data Cgn is generated at each timing (decoding time Dh (n)) determined based on the decoding time data DEtn. Decrypted every time,
The decoded image data Rg output from the decoder 2120a is displayed for each frame at a timing (display time h (n)) determined based on the decoding time data DEtn.

In such a modification of the first embodiment, as in the first embodiment, an image-encoded signal obtained by encoding an image signal is converted into a variable image decoding cycle corresponding to each frame. Decoding cycle identifier DE indicating whether or not
Since the configuration includes f, when the period of the image decoding process for each frame is fixed, the decoding time data DEtn having a large amount of information (the number of bits) for each frame is referred to by a simple circuit configuration. Without this, there is an effect that the decoding processing of the image coded data can be performed based only on the decoding cycle data DEp.

In the image decoding apparatus according to the modification of the first embodiment, the decoding of each frame is performed based on data for determining the timing of the decoding process for each frame included in the coded image signal. Although the image display of each frame is performed together with the processing, the image decoding apparatus performs the image display of each frame based on the data for determining the timing of the display processing for each frame included in the image coded signal. The decoding process of each frame may be performed together with the display process.

In this case, the decoding timing at which the decoding process corresponding to each frame is performed is set based on the display timing data of a plurality of frames including the target frame to be decoded. That is, based on the display timing data of the target frame and the display timing data of the next frame to which the next data is transmitted, the decoding timing of the target frame is determined by the earlier display timing of the two frames. Set the timing earlier by the offset time.

More specifically, if the display timing of the target frame to be decoded is earlier than the display timing of the next frame to which data is transmitted after the target frame, the offset time is set to the target frame. Is set to a size equal to or longer than the time required for the decoding process for.
On the other hand, a target frame (for example, P
-VOP), the next frame (for example, BV
If the display timing of OP) is earlier, the offset time is set to be equal to or greater than the total time of the time required for the decoding process for the target frame and the time required for the decoding process for the next frame.

Embodiment 2 FIG. 7A shows a data structure of an image coded signal 120a having a constant frame display period according to the second embodiment of the present invention. The image encoded signal 120a is one image (1 in MPEG4).
Are obtained by encoding an image signal having a fixed frame display cycle corresponding to an image corresponding to one object).
, F (n) corresponding to (0), F (1), F (2),..., F (n) are arranged in transmission order. In the image coded signal 120a, in the header H, the display cycle identifier Df indicating whether the frame display cycle is fixed or not, and the frame display cycle is M (natural number) times the minute unit time (1 / N). The display cycle multiplier data Dm, which indicates this by the multiplier M,
And a value N for obtaining the minute unit time (1 / N).
(A natural number) is inserted as minute unit time data Dk,
The code strings Sc0, Sc1, Sc2,.
At the beginning of Scn, the display time y'0, y 'of the frame
Display time data Dy0, Dy1, Dy2,..., D indicating 3, y′1,..., Y′n ′ (see FIG. 17A).
yn is inserted. It should be noted that the image coded signal 12
In the header H of 0a, the minute unit time data Dk, the display cycle identifier Df, and the display cycle multiplier data Dm are arranged to be transmitted in this order.

The code strings Sc0, Sc of each frame
, Scn, following the display time data Dy0, Dy1, Dy2,..., Dyn, image coded data Cg0, Cg1, Cg2,.
n has been inserted.

In the image coded signal 120a, if the reference time is x (see FIG. 17A), VOP0, VOP
3, VOP1,... Corresponding to each frame F (0),
The display times h (0), h (1), F (1), F (2),.
h (2),... are display time data Dy0, Dy1, D
x + y / N (y = y′0,
y′3, y′1,...).

However, the coded image signal 120a includes the minute unit time data Dk and the display cycle multiplier data Dm.
, The display time data Dy0, Dy
Without using 1, Dy21,..., A frame is displayed based on the minute unit time (1 / N) obtained from the minute unit time data Dk and the value of M (natural number) obtained from the display cycle multiplier data Dm. The period T (= M × 1 / N) is obtained, and the image of each frame is displayed at the original display time h (n) (= x + y × M × 1 / N) of each frame F (n) determined by the reference time x. can do.

FIG. 7 (b) shows an image coded signal 12 having a variable frame display period according to the second embodiment of the present invention.
0b shows the data structure. This image encoded signal 1
20b has a data structure in which the display cycle multiplier data Dm of the header portion H in the image coded signal 120a is removed.

Hereinafter, the image coded signal 120 as described above will be described.
encoding processing of an image signal for generating a or 120b,
And its decoding process will be described. FIG. 8 is a diagram showing a flow of the encoding process. First, in the encoding process, the minute unit time data Dk is added to a header portion of a bit stream corresponding to an input image signal corresponding to a predetermined image (step S30). It is determined whether or not the display cycle of the image signal is fixed (step S31). If the result of this determination is that the display cycle is fixed, a display cycle fixed identifier Df indicating that the display cycle of the image signal is fixed
Is added to the header of the bit stream so as to follow the minute unit time data Dk (step S32),
Further, the display cycle multiplier data Dm is added to the header so as to follow the display cycle fixed identifier Df (step S33). Thereafter, the counter value n corresponding to the number n indicating the transmission order of each frame F (n) constituting the predetermined image is set to n = 0 (step S35).

Next, as a code string corresponding to the first frame F (0) in the transmission order, the display time data Dyn (= Dy0) and the coded image data Dgn (= Cg0) of the corresponding frame are sequentially transferred to the header. H (steps S36 and S37). Thereafter, it is determined whether or not the processing target frame in the image signal is the last frame in the transmission order (Step S).
38), if the frame to be processed is not the last frame,
The frame F (n) (= F
(0)) is incremented by 1 (step S39), and the subsequent frame F (n + 1)
(= F (1)), the above steps S36 to S39
Is performed.

The processing in steps S36 to S39 is repeated until it is determined in step S38 that the frame to be processed is the last frame. As a result, the image coded signal 120a is generated. On the other hand, if the result of the determination in step S31 is that the display cycle is variable, the display cycle variable identifier Df indicating that the display cycle of the image signal is variable is included in the header of the bit stream corresponding to the image signal. Is added following the minute unit time data Dk (step S34). After that, the processes of steps S35 to S39 are performed,
The coded image signal 120b is generated.

FIG. 9A shows an image encoding apparatus 120 as hardware for performing the encoding processing of the second embodiment.
FIG. 3 is a block diagram showing a configuration of a 0. The image coding apparatus 1200 according to the second embodiment has an input image signal S like the image coding apparatus 1000 according to the first embodiment.
Based on an encoder 1110 that encodes g to generate encoded data Cgn, and the input image signal Sg,
It is determined whether or not the display cycle of the frame is constant (that is, whether the display cycle is fixed or variable), and the display cycle identifier Df indicating whether or not the display cycle is constant is determined. 1131.

Further, the image encoding apparatus 1200 includes a minute unit time data generator (first data generator) 1232 for generating minute unit time data Dk based on the input image signal Sg, Image signal Sg
, A display cycle multiplier data D indicating a numerical value M for expressing a frame display cycle in units of minute unit time.
display time multiplier (second data generator) 1233 for generating m and display time data (display timing data) indicating the display time h (n) of each frame based on the input image signal Sg. ) A display time data generator (third data generator) 1234 for generating Dyn.

Further, the above-mentioned image coding apparatus 1200
Based on the display cycle identifier Df from the determiner 1131, an open / close switch 1241 that switches between a conduction state for conducting the display cycle multiplier data Dm and a cutoff state for blocking the display cycle multiplier data Dm is provided. .

Then, the image encoding apparatus 1200
The minute unit time data Dk from the first data generator 1232 and the display cycle identifier D from the determiner 1131
f, display cycle multiplier data Dm from the open / close switch 1241, display time data Dyn from the third data generator 1234, and image coded data Cgn from the encoder 1110 are multiplexed to form a multiplexed bit stream M 2.
And a multiplexer (MUX) 1220 for generating
The multiplexed bit stream M2 is converted to the image encoded signal 12 described above.
0a or an image coded signal 120b.

The operation of the image coding apparatus 1200 will be briefly described below. First, the image encoding device 12
When the image signal Sg corresponding to the predetermined image is input to 00, the determiner 1131 determines whether or not the display cycle of the image signal Sg is variable, and the display cycle identifier Df indicating the determination result. Is output. At this time, based on the image signal Sg, the first to third data generators 1232
1234, the minute unit time data D
k, display cycle multiplier data Dm, and display time data Dy
n is generated, and the encoder 1110 encodes the image signal Sg and outputs it as encoded image data Cgn.

At this time, the minute unit time data D
k, the display cycle identifier Df, the display time data Dyn, and the image coded data Cgn are always output to the multiplexer 1220, and the display cycle multiplier data Dm is the display cycle identifier Dm.
The signal is output to the multiplexer 1220 via the open / close switch 1241 that has been turned on by f.

That is, when an image signal having a fixed image display cycle corresponding to each frame is input as an image signal, the minute time data Dk, display cycle identifier Df, display cycle multiplier data Dm, and Display time data Dtn and image encoded data C corresponding to each frame
gn is output to the multiplexer 1220. Then, in the multiplexer 1220, the minute unit time data Dk,
The display cycle identifier Df, the display cycle multiplier data Dm, the image coded data Cgn, and the display time data Dyn are multiplexed, and an image coded signal 120a is output as a multiplexed bit stream M2.

On the other hand, when an image signal whose image display cycle corresponding to each frame is variable is input as the image signal, the open / close switch 1241 is cut off by the display cycle identifier Df, and the minute unit Along with the time data Df and the display cycle identifier Df, the display time data Dyn and the image coded data Cg of each frame are described.
n is output to the multiplexer 1220. Then, in the multiplexer 1220, the display time data Dyn and the image coded data Cgn of each frame are multiplexed together with the minute unit time data Df and the display cycle identifier Df, and the image coded signal 120b is multiplexed as a multiplexed bit stream M2.
Is output.

Next, a decoding process for decoding an image coded signal having an image signal data structure according to the second embodiment will be described with reference to FIG. FIG. 10 shows the second embodiment.
FIG. 9 is a diagram showing a flow of a decoding process in. First, in the decoding process, the multiplexed bit stream M2 (the image encoded signal 120a or 120
The minute unit time data Dk in b) is read (step S40), and by detecting the display cycle identifier Df, it is determined whether or not the display cycle of the image coded signal is fixed (step S41). As a result of this determination, if it is determined that the display cycle is fixed, from the header portion H of the image coded signal, the display cycle T is changed to a minute unit time (1 / N).
The display cycle multiplier data Dm indicating the multiplier M (natural number) is read (step S42a),
Subsequently, based on the read minute unit time data Dk and display cycle multiplier data Dm, a frame display cycle T is obtained by calculation T = (1 / N) × M (step S43a).

Thereafter, each frame F 'shown in the display order is displayed.
The count value n 'corresponding to the order n' from the first frame of (n ') is set to 0 (step S44a).
The display time h ′ (n ′) of each frame F ′ (n ′) is obtained by the calculation formula h ′ (n ′) = n ′ × T (step S45a). At this time, each frame F
The decoding processing of the encoded image data Cgn corresponding to (n) is performed, and the decoded image data Rg corresponding to the frame F (n) is generated.

Thereafter, it is determined whether or not the processing target frame F ′ (n ′) counted in the display order is the last frame in the predetermined image (step S46).
a), if the frame to be processed is the last frame, the decoding process ends; otherwise, the counter value n 'is incremented by one (step S47).
a) The processes of steps S45a to 47a are repeatedly performed until it is determined in step S46a that the frame to be processed is the last frame.

In the above-described decoding processing, the decoded image data Rg corresponding to each of the decoded frames F ′ (n ′) is displayed in a predetermined display order n ′ at a corresponding display time h ′ (n). ').

On the other hand, if it is determined in step S41 that the display cycle is variable, the count value n corresponding to the order n of each frame F (n) in the transmission order is set to 0 (step S42b). ). Subsequently, each frame F
Display time data Dyn indicating the display time h (n) of the frame F (n) is read from the header portion H of (n) (step S43b), and each frame F (n) is further based on the display time data Dyn. Is obtained (step S44b). At this time, decoding processing of the image coded data Cgn corresponding to each frame F (n) is performed in the transmission order.

Thereafter, it is determined whether or not the processing target frame F (n) counted in the transmission order is the last frame in the predetermined image (step S44b), and if the processing target frame is the last frame, The decryption process ends. On the other hand, if the frame to be processed is not the last frame, the count value n in this decoding process
Is incremented by one (step S46b), and thereafter, the processing of steps S42b to S46b is performed until it is determined in step S45b that the frame to be processed is the last frame.

In the above-described decoding processing, the decoded image data Rg corresponding to each of the decoded frames F (n) is obtained.
Are displayed at a display time h (n) corresponding to each frame F (n) in a predetermined display order n ′.

FIG. 11A is a block diagram showing the configuration of an image decoding device as hardware for performing the decoding process according to the second embodiment. This image decoding device 2200
Is configured to decode and reproduce the multiplexed bit stream M2, which is the encoded image signal 120a or 120b output from the image encoding device 2000.

That is, the image decoding apparatus 2200
From the multiplexed bit stream M2, the minute unit time data Dk, the display cycle identifier Df, the display cycle multiplier data Dm, the display time data Dyn, and the image encoded data C
Data divider (DEMUX) that extracts and outputs gn
2210, and a decoder 2220 that decodes the coded image data Cgn and outputs decoded image data Rg.

Further, the image decoding apparatus 2200 switches between a conduction state in which the display cycle multiplier data Dm is conducted and a interruption state in which the data Dm is blocked based on the display cycle identifier Df. Open / close switch 22
40, a conduction state for conducting the display time data Dyn based on the display cycle identifier Df, and the data Dyn.
and a second open / close switch 2250 that switches between a shut-off state for shutting off yn.

Furthermore, the image decoding apparatus 2200
The minute unit time data Dk and the decoded image data Rg
And display time multiplier data Dm and display time data Dty via the switches 2240 and 2250, respectively.
A display device 2230 that displays an image at a predetermined display timing based on these data is provided.

The operation of the image decoding apparatus 2200 will be briefly described below. First, the image decoding device 22
00, when the multiplexed bit stream M2 from the image encoding device 1200 is input, the data separator 2210
, The minute unit time data Dk, the display cycle identifier Df, and the display cycle multiplier data Dm are separated from the multiplexed bit stream M2, and the display time data Dyn and the image coded data Cgn are further separated from the multiplexed bit stream M2 for each frame. Are separated.

Then, the image coded data C of each frame
gn is decoded by the decoder 2220 and output to the display device 2230 as decoded image data Rg. At this time, the minute unit time data Dk is directly transmitted to the display device 2.
230, the display cycle multiplier data Dm is output to the display device 2230 via a first open / close switch 2240 that is opened and closed by the display cycle identifier Df, and the display time data Dyn of each frame is displayed by the display cycle identifier Df. Is output to the display device 2230 via a second open / close switch 2250 that is opened and closed by the switch. Here, when the multiplexed bit stream M2 is an image coded signal 120a having a fixed display cycle, the first and second on / off switches 2240 and 2250 are in a conductive state, and the multiplexed bit stream M2 is displayed at a display cycle. Is a variable image coded signal 120b, the first and second open / close switches 2240 and 2250 are turned off.

As a result, in the display device 2230,
The image of each frame corresponding to the decoded image data Rg having a fixed display cycle is displayed at a predetermined display timing based on the minute unit time data Dk and the display cycle multiplier data Dm. In this case, the display timing of each frame is a display time h ′ (n) ′ = determined by an arithmetic expression T × n ′ (T = (1 / N) × M). On the other hand, an image of a frame corresponding to the decoded image data Rg having a variable display cycle is displayed at a predetermined display timing based on the display time data Dty. In this case, the predetermined display timing is the display time h (n) determined by the display time data Dty.

As described above, according to the second embodiment, the image encoded signal is added to the display cycle identifier Df indicating whether or not the display cycle corresponding to each frame is variable, and the predetermined time interval is set to N ( Minute unit time (1)
/ N), the minute unit time data Dk indicated by the natural number N, and the fixed frame display period T is indicated by how many times (M) the minute unit time (1 / N) corresponds. Since the data structure includes the display cycle multiplier data Dm, the value (magnitude) of the frame rate of the coded image signal having a fixed frame rate can be detected in advance before decoding each frame. In addition, there is an effect that various hardware configurations for realizing the display processing can be simplified.

In the second embodiment, as the additional data for determining the timing of the reproduction processing for each frame on the decoding side, the minute unit time data Dn for setting the display timing of each frame and the display data Although the data structure of the image coded signal including the cycle identifier Df, the display cycle multiplier data Dm, and the display time data Dyn has been described, the data structure of the image coded signal is different from the display timing of each frame in each frame. Additional data for determining the timing of the decoding processing of the data, that is, data including minute unit time data, decoding cycle identifier, decoding cycle multiplier data, and decoding time data may be included. A description will be given of a second modification.

(Modification of the Second Embodiment) The data structure of the modification of the second embodiment is such that the display cycle identifier Df and the display cycle multiplier data Dp in the image coded signal 120a of the second embodiment are converted to the decoding cycle. The display time data Dyn in the image coded signal 120b according to the second embodiment is replaced with the decoding time data DEyn, instead of the identifier DEf and the decoding cycle multiplier data DEp.

Here, the decoding cycle identifier DEf indicates whether or not the cycle of the decoding process for decoding the image coded signal corresponding to each frame is variable. It is inserted as a decoding cycle fixed identifier into an image coded signal having a fixed DT, and as a decoding cycle variable identifier into an image coded signal having a variable decoding cycle DT.

The decoding cycle multiplier data DEm is:
Data indicating the cycle DT of the decoding process corresponding to each frame by a multiplier value M of the above-mentioned minute unit time (1 / N), that is, how many times (M) the cycle corresponds to the minute unit time, The decoding time data DEyn is data indicating the timing at which the decoding process corresponding to each frame is performed.

The encoding process for generating an encoded image signal having the data structure of the modification of the second embodiment is shown in FIG.
Steps S31, S32, S3 in the flow shown in FIG.
It can be realized by replacing the processing of steps 3, S34 and S36 as follows. That is, the process of determining the display cycle in step S31 is replaced with the process of determining whether the decoding cycle is fixed, and the display cycle fixed identifier Df and the display cycle variable identifier Df in steps S32 and S34.
Are added to the decoding cycle fixed identifier DE.
f, the process is replaced with a process of adding the variable decoding period identifier DEf. Further, the process of adding the display cycle multiplier data Dm in step S33 is performed according to the decoding cycle multiplier data DE.
The processing for adding the display time data Dyn in step S36 is replaced with the processing for adding the decoding time data DEyn.

FIG. 9B shows the configuration of an image encoding device 1200a as hardware for performing the encoding process according to the modification of the second embodiment. In the image encoding device 1200a, the cycle of the decoding process corresponding to the frame is constant based on the input image signal Sg, instead of the determiner 1131 of the image encoding device 1200 according to Embodiment 2. There is provided a determiner 1131a that determines whether or not there is (ie, whether the decoding cycle is fixed or variable) and outputs a decoding cycle identifier DEf indicating whether or not the decoding cycle is constant.

Further, the image encoding device 1200a
Instead of the display cycle multiplier data generator 1233 and the display time data generator 1234 in the image encoding device 1200 according to the second embodiment, the cycle of the frame decoding process is based on the input image signal Sg, respectively. Is a decoding cycle multiplier data generator (second data generator) 1 that generates decoding cycle multiplier data DEm as a multiplier value M indicating how many times the minute unit time (1 / N) corresponds.
233a and the input image signal Sg,
A decoding time data generator (third decoding time data generator) that generates decoding time data DEyn indicating the decoding time Dh (n) of each frame F (n)
Data generator) 1234a.

In the image encoding device 1200a, the multiplexer 1220a converts the minute unit time data Dk, the decoding cycle multiplier data DEm, and the decoding time data DEty into the image encoding data of each frame F (n). C
gn, and outputs an image coded signal with a fixed decoding cycle or an image coded signal with a variable decoding cycle as a multiplexed bit stream M2a. Other configurations are the same as those of the image coding apparatus 1200 according to the second embodiment.

The operation of the image encoding apparatus 1200a according to the modification of the second embodiment will be briefly described below. In the image encoding device 1200a having such a configuration, when the image signal Sg is input, the determiner 1131a determines whether or not the decoding cycle of the image signal Sg is variable. A decoding cycle identifier DEf is output,
In the first data generator 1232a, the minute unit time data Dk is converted into the second and third data generators 1233a.
In a and 1234a, decoding cycle multiplier data DEm and decoding time data DEyn are respectively generated. In the encoder 1110, the image signal Sg is encoded and output as encoded image data Cgn.

The multiplexer (MUX) 1220a has the decoding cycle identifier DE from the decision unit 1231a.
f, the encoded data Cgn from the encoder 1110,
The data Dk and DEyn from the first and third data generators 1232 and 1234a are input, and the decoding cycle multiplier data DE from the second data generator 1233a.
m is input via the open / close switch 1241a. Then, from the multiplexer 1220a, these data are multiplexed, and an image coded signal having a fixed decoding cycle or an image coded signal having a variable decoding cycle is output as the multiplexed bit stream M2a. To

On the other hand, the decoding process for decoding the image coded signal having the data structure of the modification of the second embodiment is performed in the steps S41 and S42 in the flow shown in FIG.
a, S43b, S44a, S44b, S45a, S47
This can be realized by replacing the processing of a as follows.

Specifically, the process of determining the display cycle in step S41 is replaced with a process of determining whether or not the decoding cycle is fixed, and the process of reading the display cycle multiplier data Dm in step S42a is replaced with the above-described decoding cycle. Step 43 is replaced with processing for reading the multiplier data DEm.
b, the process of reading the data Dyn indicating the display time h (n) is performed by the data DEyn indicating the decoding time Dh (n).
Replace with the process of reading.

The processing for obtaining the display time h (n) based on the data Dyn in step S44b is replaced with the processing for obtaining the decoding time Dh (n) based on the data DEyn, and the display time is counted in the display order in step S44a. The processing in which the count value n 'corresponding to the order n' from the first frame of each frame F '(n') is set to 0 is performed in each frame F '(n') counted in the transmission order.
The processing is replaced with the processing of (n) in which the count value n corresponding to the order n from the first frame is set to 0.

Further, the process of obtaining the display time h ′ (n ′) of each frame F ′ (n ′) in step S45a by the calculation formula h ′ (n ′) = n ′ × T is performed in each frame F ( n) The decoding time Dh (n) is Dh by the decoding cycle DT and the number n indicating the transmission order of the frame.
The process in which the count value n 'is incremented in step S47a is replaced with the process in which the count value n is incremented.

FIG. 11B shows the configuration of an image decoding device 2200a as hardware for performing a decoding process according to a modification of the second embodiment. The image decoding device 2200a is configured to perform a reproduction process including a decoding process and a display process on the multiplexed bit stream M2a output from the image encoding device 1200a.

That is, the image decoding apparatus 2200a
Is the image decoding apparatus 2200 according to the second embodiment.
Data to extract and output minute unit time data Dk, decoding cycle identifier DEf, decoding cycle multiplier data DEm, decoding time data DEyn, and image encoded data Cgn from the multiplexed bit stream M2a in place of the data separator 2210 of FIG. A separator (DEMUX) 2210a is provided.

The image decoding apparatus 2200a
Instead of the first and second open / close switches 2240 and 2250 in the image decoding device 2200 according to the second embodiment,
First open / close switch 22 that controls conduction / non-conduction of decoding cycle multiplier data DEm based on decoding cycle identifier DEg.
40a and a second open / close switch 2250a for controlling conduction / non-conduction of the decoding time data DEyn based on the decoding cycle identifier DEg.

In the image decoding apparatus 2200a, the minute unit time data Df from the data separator 2210a, the decoding cycle multiplier data DEm output from the first and second switches 2240a and 2250a, and the decoding time data DEyn are output. , Decoder 2220a and display device 2230
a.

The decoder 2220a outputs the coded image data Cgn of each frame F (n) having a fixed decoding cycle.
At a timing (decoding time Dh) determined based on the minute unit time data Dk and the decoding cycle multiplier data DEm.
(n) = DT × n), and decodes each frame, and the encoded image data C of each frame F (n) whose decoding cycle is variable
gn is decoded for each frame at a timing (decoding time Dh (n)) determined based on the decoding time data DEyn.

The display device 2230a converts the decoded image data Rg of each frame F (n) having a fixed decoding cycle into timing (display) determined based on the minute unit time data Dk and the decoding cycle multiplier data DEm. Time h
(n)), and each frame F whose decoding cycle is variable
The image decoding data Rg of (n) is set at a timing (display time h) determined based on the decoding time data DEyn.
(n)). Other configurations are the same as those of the image decoding device 2200 according to the second embodiment.

Hereinafter, the operation of the image decoding apparatus 2200a according to the modification of the second embodiment will be briefly described. In the image decoding device 2200a having such a configuration,
When the multiplexed bit stream M2a is input, the data separator 2210a separates the minute unit time data Dk, the decoding cycle multiplier data DEm, the decoding cycle identifier DEf, the decoding time data DEyn, and the encoded image data Cgn. .

In the decoder 2220a, when the decoding cycle of the input image coded signal is fixed, the image coded data Cgn is converted to the minute unit time data Dk and the decoding cycle multiplier data DEm. When the decoding is performed for each frame at a timing determined based on the decoding timing and the decoding period of the input decoded image signal is variable, the image coded data Cgn is determined based on the decoding time data DEyn (decoding time). Dh (n)). Here, the decoding time of an image coded signal having a fixed decoding cycle is determined by the product of the number n indicating the transmission order and the decoding cycle DT = (1 / N) × M,
The decoding time of the image encoded signal whose decoding cycle is variable is determined by the decoding cycle data DEyn.

Further, in the display device 2230a, the image of each frame F (n) corresponding to the decoded image data Rg whose decoding cycle is fixed is determined based on the minute unit time data Dk and the decoding cycle multiplier data DEm. , And the image of the frame F (n) corresponding to the decoded image data Rg whose decoding cycle is variable,
It is displayed at a predetermined display timing based on the decoding time data DEty.

In such a modification of the second embodiment, similarly to the above-described second embodiment, image encoding data Cgn obtained by encoding an image signal is converted into a cycle of an image decoding process corresponding to each frame. The decoding period identifier DEf indicating whether the frame is variable or not, the minute unit time data Dk indicating the fixed decoding period, the decoding period multiplier data DEm, and the decoding time data DEyn indicating the decoding time are included. When the period of the image decoding process is fixed, a small circuit corresponding to one image can be obtained by a simple circuit configuration, that is, without referring to the decoding time data DEyn having a large amount of information (number of bits) for each frame. It is possible to easily perform a decoding process of an image coded signal based only on the unit time data Dk and the decoding cycle multiplier data DEm having a small amount of information (number of bits). Can.

When the period of the image decoding process for each frame is variable, the decoding process of the image coded signal is performed by referring to the decoding time data DEyn for each frame, as in the related art. There is an effect that can be. In the image decoding device 2200a according to the modification of the second embodiment, the decoding process of each frame is performed based on the data for determining the timing of the decoding process for each frame included in the image coded signal. The image decoding apparatus performs image display of each frame, but the image decoding apparatus performs display processing of each frame based on data for determining timing of display processing for each frame included in the image encoded signal. At the same time, a decoding process for each frame may be performed.

In this case, the decoding timing at which the decoding process corresponding to each frame is performed is set based on the display timing data of a plurality of frames including the target frame to be decoded. That is, based on the display timing data of the target frame and the display timing data of the next frame to which the next data is transmitted, the decoding timing of the target frame is determined by the earlier display timing of the two frames. Set the timing earlier by the offset time.

Specifically, when the display timing of the target frame to be decoded is earlier than the display timing of the next frame to which data is transmitted after this target frame, the offset time is set to the target frame. Is set to a size equal to or longer than the time required for the decoding process for
On the other hand, a target frame (for example, P
-VOP), the next frame (for example, BV
If the display timing of OP) is earlier, the offset time is set to be equal to or greater than the total time of the time required for the decoding process for the target frame and the time required for the decoding process for the next frame.

Further, an encoding processing program or a decoding processing program for performing image processing by software in the image encoding apparatus or the image decoding apparatus shown in each of the above-described embodiments and modifications thereof may be provided by a floppy disk or the like. By recording the data on the data storage medium described above, the processing described in each of the above embodiments can be easily realized by an independent computer system.

FIG. 12 shows that the encoding process or the decoding process in each of the above-described embodiments and its modifications is performed by a computer system using a floppy disk storing the encoding program or the decoding program. It is a figure for explaining a case. FIG. 12 (a)
Fig. 12 (b) shows the external appearance, cross-sectional structure, and floppy disk main body viewed from the front of the floppy disk.
Shows an example of the physical format of the floppy disk body.

[0171] The floppy disk FD is composed of the floppy disk main body D and a floppy disk case FC.
A plurality of tracks Tr are formed concentrically from the outer circumference toward the inner circumference on the surface of the floppy disk main body D, and each track Tr has 16 sectors Se in the angular direction. Is divided into Therefore,
In the floppy disk FD in which the program is stored, the floppy disk body D has the data as the program recorded in an area (sector) Se allocated thereon.

FIG. 12C shows a configuration for recording the program on the floppy disk FD and performing image processing by software using the program stored in the floppy disk FD.

The above program is stored on a floppy disk FD
, The data as the program is written from the computer system Cs to the floppy disk FD via the floppy disk drive FDD.
When the image encoding device or the image decoding device is constructed in the computer system Cs by the program recorded on the floppy disk FD, the program is read from the floppy disk FD by the floppy disk drive FDD, and the computer system C
Load to s.

In the above description, the description has been made using a floppy disk as a data storage medium. However, even when an optical disk is used, encoding or decoding by software can be performed in the same manner as in the case of the floppy disk. it can. Further, the data storage medium is not limited to the above-mentioned optical disk or floppy disk, but may be any type of IC card, ROM cassette, etc., as long as it can record a program. As in the case where a floppy disk or the like is used, encoding or decoding by software can be performed.

Further, the image coded signal stored in the data storage medium such as a floppy disk is formed into the image signal data structure of the first and second embodiments or the modified examples thereof, whereby the image from the floppy disk can be read. When decoding a coded signal and displaying an image, a simple circuit configuration can be used to execute a reproduction process including a decoding process and a display process of the coded image signal when the frame display period or the decoding process period is fixed. It can be carried out.

[0176]

According to the image decoding method according to the inventions as described above, according to the present invention, an image decoding for decoding a coded image signal including an image code cade <br/> over data corresponding to a plurality of frames Encoding method, wherein the image coded signal is the image coded signal
The image display interval of all frames included in is fixed
Or including a display period identifier indicating whether it is variable, to generate image decoded data by decoding the image encoded data,
When the display cycle identifier indicates that the image display interval is fixed, the predetermined time interval is divided into N (natural numbers) equally.
The self-timer used to represent the magnitude of the obtained minute unit time.
Minute unit time data indicating the number N, and the image display interval
Is M (natural number) times the minute unit time.
Extracting display cycle multiplier data from the image encoded signal
And the minute unit time data and the display cycle multiplier data
Is used to calculate the display interval of the decoded image data.
An image display interval, wherein the display cycle identifier is the image display
When indicating that the interval is variable, the plurality of frames
Display timing data set for each of
Indicating the display timing of the plurality of frames.
Extracting display timing data from the encoded image signal
And is indicated by the extracted display timing data.
The image display timing of the decoded image data.
The feature is that the frame rate (each
Frame display cycle) is fixed or not
Accordingly, there is an effect that display processing for an image coded signal can be favorably performed with a simple hardware configuration.

[0177]

[0178]

[0179]

[Brief description of the drawings]

FIG. 1 illustrates a data structure of an image coded signal according to Embodiment 1 of the present invention in a case where a frame display cycle is fixed (FIG.
(a)) and when the frame display cycle is variable (Fig. (b))
It is a figure which shows and contrasts.

FIG. 2 is a diagram illustrating a state of image display based on an image coded signal having a fixed frame display period according to the first embodiment.

FIG. 3 is a diagram showing a flow of an encoding process for generating an encoded image signal having a data structure according to the first embodiment.

FIG. 4 is a block diagram illustrating a configuration (FIGS. (A) and (b)) of an image encoding device that performs an encoding process according to the first embodiment and its modification.

FIG. 5 is a diagram showing a flow of a decoding process for decoding an image coded signal having a data structure according to the first embodiment.

FIG. 6 is a block diagram illustrating a configuration (FIGS. (A) and (b)) of an image decoding device that performs a decoding process according to the first embodiment and its modification.

FIG. 7 shows a data structure of an image-encoded signal corresponding to MPEG4 according to the second embodiment of the present invention in a case where the frame display period is fixed (FIG. 7A) and in a case where the frame display period is variable ( FIG. 3 (b) is a diagram for comparison with FIG.

FIG. 8 is a diagram showing a flow of an encoding process for generating an encoded image signal having a data structure according to the second embodiment.

FIG. 9 is a block diagram illustrating a configuration (FIGS. (A) and (b)) of an image encoding device that performs an encoding process according to the second embodiment and its modification.

FIG. 10 is a diagram showing a flow of a decoding process for decoding an image encoded signal having a data structure according to the second embodiment.

FIG. 11 is a block diagram illustrating a configuration (FIGS. (A) and (b)) of an image decoding device that performs a decoding process according to the second embodiment and a modification thereof.

FIG. 12 is a data storage medium (FIGS. (A) and (b)) storing a program for performing the encoding and decoding processing of each of the embodiments by a computer system, and the computer system (FIG. (C)). FIG.

FIG. 13 is a diagram showing a table of fixed frame rates in the conventional MPEG2.

FIG. 14 is a conceptual diagram for describing an image coded signal having a conventional image data structure.

FIG. 15 is a diagram showing a comparison between a data transmission order and a data display order in a frame sequence forming one image.

FIG. 16 is a diagram illustrating a conventional image display based on an image coded signal having a variable frame display period.

FIG. 17 shows each frame (V
OP) display time (FIGS. (A) and (b)) is represented by M
It is a figure for explaining the present data structure (Drawing (c)) of an image coded signal corresponding to PEG4.

[Explanation of symbols]

100a, 100b, 120a, 120b Image encoded signal 1000, 1000a, 1200, 1200a Image encoding device 1110 Encoder 1120, 1120a, 1220, 1220a Multiplexer (MUX) 1131, 1131a Judge 1132 Display period data generator (First data generator) 1132a decoding cycle data generator (first data generator) 1133 number data generator (second data generator) 1134 display time data generator (third data generator) 1134a Decoding time data generator (third data generator) 1141, 1241, 1241a, 2140 open / close switch 1142 selection switch 1232 minute unit time data generator (first data generator) 1233 display cycle multiplier data generator (second) Data generator) 1233 a Decoding cycle multiplier data generator (second data generator) 1234 Display time data generator (third data generator) 1234a Decoding time data generator (third data generator) 2000, 2000a, 2200, 2200a Image decoder 2110, 2110a, 2220, 2220a Data separator (DEMUX) 2120, 2120a, 2220, 2220a Decoder 2130, 2130a, 2230, 2230a Display device 2140a, 2240, 2240a First open / close switch 2150a, 2250, 2250a 2nd open / close switch 2160a 3rd open / close switch Df display cycle identifier DEf decoding cycle identifier Dp display cycle data DEp decoding cycle data Dm display cycle multiplier data DEm decoding cycle multiplier data Dn minute unit time data Dtn display time data DEtn decoding time data B0, B1, B2, Bn Frame number data B (0) to B (18), B (n) Frame number Cg0, Cg1, Cg2, Cgn Image encoded data Dt0, Dt1, Dt2 , Dtn display time data Dy0, Dy1, Dy2, Dyn display time data DEtn, DEtn 'decoding time data F (0) to F (18), F (n) frames F' (0) to F '(18), F '(N') frame H header h (0) to h (4), h '(0) to h' (3), h (n), h '(n')
Display time M1, M1a, M2, M2a Multiple bit stream Rg Image decoded data Sg Image signal Sa0, Sa1, Sa2, San, Sb0, Sb1, S
b2, Sbn, Sc0, Sc1, Sc2, Scn Code string t '(1) to t' (4), t '(n') Display interval

──────────────────────────────────────────────────続 き Continuation of the front page (56) References WO 95/15659 (WO, A1) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N ^7/ 24-7/68

Claims (1)

(57) [Claims] 1. An image decoding method for decoding an image coded signal including image coded data corresponding to a plurality of frames, wherein the image coded signal is the image coded signal
The image display interval of all frames included in is fixed
Or a display cycle identifier indicating whether it is variable, generates image decoded data by decoding the image encoded data, when the display cycle identifier indicates that the image display interval is fixed, a predetermined Divide the time interval into N (natural numbers)
The self-timer used to represent the magnitude of the obtained minute unit time.
Minute unit time data indicating the number N, and the image display interval
Is M (natural number) times the minute unit time.
Extracting display cycle multiplier data from the image encoded signal
And the minute unit time data and the display cycle multiplier data
Is used to calculate the display interval of the decoded image data.
The image display interval is used as the display cycle identifier and the image display interval is variable.
Indicates that each of the plurality of frames is
Display timing data set by the
Display timing data indicating the display timing of the frame
Data from the image coded signal and the extracted display
The timing indicated by the timing data is
An image decoding method characterized by using image display timing of image decoded data .