JPH10243419A - Method and device for stereoscopic vision image coding and decoding - Google Patents

Abstract

PROBLEM TO BE SOLVED: To code/decode image data obtained by picking up an image of an object from a plurality of directions with a simple circuit configuration employing one system of coding/decoding path. SOLUTION: A plurality of channels of image signals Si1, Si2,...Sin obtained by picking up an image of an object from a plurality of directions arranged sequentially for each frame or each field by a sequential processing means 301 and a multiplexer 302 to obtain an image signal, a coding means 303 codes the signal and the coded signal is outputted to a transmission line. A decoding means 305 decodes the coded and transmitted image data, the decoded signal is given to a demultiplexer 306, where the image signals for each channel are sequentially demultiplexed for each frame or each field, and the demultiplexed image signals of each channel are subject to simultaneous processing by a simultaneous processing means 307 and a plurality of channels of image signals So1, So2,...Son similar to the image signals obtained at the image pickup are outputted from the means 307.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a stereoscopic image encoding / decoding method and apparatus for encoding / decoding each image data obtained by photographing from a plurality of directions.

[0002]

2. Description of the Related Art In recent years, great expectations have been placed on a stereoscopic image system as a high-quality image system including digital broadcasting. Thereby, three-dimensional video recognition becomes possible.

When a human looks at three cylinders A, B, and C as shown in FIG. 15A, a displacement of an image position on the retina of the left eye and the right eye, ie, " Parallax ”occurs,
It is known that this allows a human to perceive depth. In the three-dimensional image system, an image in which an image shift corresponding to the parallax is generated is independently displayed on each of the right and left eyes to generate a three-dimensional effect.

That is, an image (or object) to be stereoscopically viewed is observed (photographed) from a plurality of directions, and parallax is generated using a plurality of images obtained from the observation points.

[0005] In the conventional example and the embodiment, two kinds of images (left image signal and right image signal) observed from two directions, left and right, will be mainly described for ease of explanation and understanding. However, it can be easily extended even when three or more types of images are used.

Here, a case is considered where a stereoscopic image system is applied to broadcasting and storage media. In this case, normally, two signals of a left image signal for the left eye and a right image signal for the right eye are transmitted or recorded.

By the way, a stereoscopic image requires two left and right plane images, and the amount of information is enormous. Therefore, when digitally transmitting or storing this, it is essential to compress the amount of information.

Conventionally, the compression method is based on the MPEG (Moving Picture Expert Group) standardized internationally. The MPEG image coding method is described in the latest book, “Latest MPEG Textbook” (ASCII).

In the MPEG image coding system, a hybrid system is adopted. In this method, in addition to intra-frame compression in which an image in a frame is subjected to DCT processing,
Inter-frame compression (prediction coding) for reducing redundancy in the time axis direction by using correlation between frames is also employed. In the inter-frame compression, the bit rate is further reduced by obtaining the difference between the preceding and succeeding frames and encoding the difference value (prediction error) using the property that a general moving image is very similar between the preceding and following frames. It is to let. In particular, motion-compensated inter-frame predictive coding that reduces the prediction error by estimating the motion of an image and calculating the inter-frame difference is effective.

As described above, in the hybrid system, in addition to the intra-frame encoding in which the image data of the predetermined frame is directly subjected to the DCT process and encoded, the image data of the predetermined frame and the reference image data of the frames before and after this frame are added. Predictive coding in which only the difference data is subjected to DCT processing and coding is employed. The prediction encoding method includes forward prediction encoding for obtaining a prediction error by performing motion compensation on temporally forward reference image data, and backward prediction for obtaining a prediction error by performing motion compensation on temporally backward reference image data. There are predictive coding and bidirectional predictive coding using either forward or backward or an average in both directions in consideration of coding efficiency.

To decode a frame encoded in the inter-frame compression mode, a reference image is required. That is, the original image cannot be restored only by the inter-frame compressed frame. On the other hand, a frame coded by intra-frame coding (hereinafter, referred to as an I-picture) is coded only by intra-frame information, and can be decoded only by single coded data. Therefore, M
In the PEG standard, I.P.
One picture is inserted in a fixed cycle (for example, 12 frames). According to the MPEG standard, an inter-coded frame (hereinafter, referred to as a P picture) is obtained by forward prediction coding using this I picture. Note that P
A picture can also be obtained by forward predictive coding of a preceding P picture. Also, a bidirectional prediction adaptive switching frame (hereinafter, referred to as B) is obtained by bidirectional prediction encoding using either the forward or rearward or bidirectional I and P pictures.
Picture).

[0012] Regarding the compression of stereoscopic images, the booklet "VI
EW VOL.15 NO.5 (1996) ”(edited by NHK Engineering Service), and JP-A-7-250350“ Stereoscopic Video Coding Apparatus ”(Sanyo Electric).

In a conventional stereoscopic image system, FIG.
As shown in FIG. 6 (a), the left and right image signals are compressed as separate image sequences (ie, left image sequences L1, L2, L3,... And right image sequences R1, R2, R3,...). Or, Figure 1
For example, as shown in FIG. 6B, the left image sequence (L1, L2, L3,...)
..) And the difference signal between the left and right image signals, that is, a parallax image sequence (L
1-R1, L2-R2, L3-R3,...). The parallax image signal is compressed because the left and right images are similar, that is, there is a correlation, so that the difference has a higher compression efficiency.

Conventionally, both left and right compression paths are required in both cases of FIGS. 16 (a) and 16 (b).

This will be described by taking a transmission system such as a broadcast as an example. FIG. 17 shows an outline of a conventional transmission system, and shows a basic configuration diagram of MPEG.

The MPEG encoder 10 supplies the image data input to the input terminal 11 to one input terminal (intra-frame encoding side terminal) a of the intra-frame / inter-frame encoding switch S1. It is supplied to one input terminal of the subtractor 12 and the motion compensating means 19. The output of the subtractor 12 is supplied to the other input terminal of the changeover switch S1.

When the intra-frame encoding mode is designated, the input image data is converted from the spatial coordinate components into frequency components by the DCT circuit 13 through the switch S1 by two-dimensional DCT processing of 8.times.8 blocks. Thereby, spatial correlation components can be reduced. Next, DCT
The output of the circuit 13 is provided to a quantization circuit 14, where the output of the circuit 13 is quantized with a predetermined quantization width to reduce the redundancy of one block of signal. The quantized data from the quantization circuit 14 is input to the Huffman encoding circuit 20, is subjected to variable length encoding based on a Huffman code table or the like, and is output to a variable length encoded transmission path.

When the inter-frame coding mode is designated, the switch S1 is switched to the other input terminal b (the output terminal of the subtractor 12), and the switch S2 is connected to the input terminal b. You. The input image data is supplied to a subtractor 12, which receives pixel data of a block (reference block) of a reference image which has been motion-compensated from a motion compensating unit 19, which will be described later, and outputs the pixel data of the block of the current frame. DCT circuit 13 using the difference as a prediction error
Output to The DCT circuit 13 performs DCT processing on the difference data. On the other hand, the reference block from the motion compensator 19 is obtained by decoding the quantized output. That is,
The output of the quantization circuit 14 is also supplied to an inverse quantization circuit 15, where it is inversely quantized, and further subjected to inverse DCT processing in an inverse DCT circuit 16 to return to original video data (this is difference data). The output of the inverse DCT circuit 16 is
Given to. The output of the adder 17 is
The adder 17 adds the difference data to the reference block from the motion compensator 19 to reproduce the block data of the current frame, and outputs the data to the frame delay circuit 18. The frame delay circuit 18
The block data from the adder 17 is stored in correspondence with the screen position. The motion compensating means 19 includes two motion compensating circuits for forward prediction and backward prediction, a means for obtaining an average output of both the two motion compensating circuits, a block data of the current frame and a frame delay circuit 18 A motion vector detecting circuit for detecting a motion vector from the reference block data from the CPU, and an output of either one of the two motion compensating circuits or both of the two motion compensating circuits corresponding to respective predictions of forward prediction, backward prediction and bidirectional prediction. And a selection circuit for selecting an average output as the motion-compensated reference block data.

The reference block data from the motion compensating means 19 is supplied to the subtractor 12, while being supplied to the adder 17 via the switch S2.

Thus, the motion-compensated reference block data is supplied to the subtractor 12, and the subtractor 12
The prediction error is calculated from the difference between the block of the current frame and the reference block data. Thereby, DCT processing is performed on the prediction error.

The variable length coded data transmitted from the encoder 10 via the variable length coded transmission path is supplied to the decoder 21. The intra-frame and inter-frame encoded data from the encoder unit 10 are sequentially input to the decoder unit 21, subjected to inverse variable-length encoding by the Huffman decoding circuit 22, and then inversely quantized by the inverse quantization circuit 23. It is quantized, further subjected to inverse DCT processing by an inverse DCT circuit 24, and supplied to an adder 25. In the adder 25, the inverse DC
The image data from the T circuit 24 and the motion-compensated reference image data from the motion compensator 27 are added and output to an output terminal 28 as output image data. Further, the output of the adder 25 is supplied to a frame delay circuit 26, which is supplied to the motion compensating means 27 after being frame-delayed, from which the motion-compensated reference image data is supplied to the adder 25.

In MPEG, DCT (Discrete Cosine Transform) is basically performed on a small block in an image, and the higher the frequency component, the less bit allocation is performed, and then the variable-length coding is performed. Then, in order to further increase the efficiency, compression in the amplitude direction by quantization (Quantize) and compression in the time axis direction by motion compensation between fields or between frames are performed. Only the information having the smaller amount of information is switched between the difference information remaining after the motion compensation and the information itself in the field or in the frame for transmission.

FIGS. 18 (a) and 18 (b) show the MP shown in FIG.
1 shows a conventional example in which a transmission system of a three-dimensional image is configured using an EG image compression method. These are shown in FIG.
(a) and (b) respectively.

In FIG. 18A, the left eye,
The right eye image signal is input and these are respectively
It is supplied to the MPEG encoder units 101 and 102 arranged for the right eye, and the obtained left and right encoded outputs are multiplexed.
And supplies it to the decoder side as one image signal. On the decoder side, the transmitted image signal is extracted separately for the left and right by the demultiplexer 203, and then decoded by the left- and right-eye MPEG decoder units 201 and 202, and output as left-eye and right-eye image signals. I do.

In FIG. 18 (b), the left eye,
An MPEG encoder unit which receives right-eye image signals and arranges these left-eye image signals as they are for the left.
The image signal for the left eye and the image signal for the right eye are input to a subtractor 104 to obtain a parallax image signal as a difference between the two, and an MPEG encoder 105 provided for the parallax.
And outputs the encoded outputs of left and parallax to the multiplexer 10
3 and transmitted to the decoder side as one image signal. On the decoder side, the transmitted image signal is extracted by the demultiplexer 203 for each of left and parallax, and then decoded by the left-eye and parallax MPEG decoders 201 and 202, respectively. Supplies the left-eye image signal to the subtractor 205, and supplies the left-eye image signal and the parallax image signal from the parallax MPEG decoder unit 204 to the subtractor 205. Output as the right eye image signal.

However, in a conventional transmission system as shown in FIGS. 18A and 18B, the encoder and the decoder in the MPEG are used for the left and right (or parallax) image signals.
There was a problem that had to have a system.

Also, as shown in FIGS. 18 (a) and 18 (b), when encoding is performed separately on the left and right sides in different systems, a time base signal is created when decoding, and a delay memory is arranged for each system. There is a problem that processing for synchronizing the left and right image signals becomes difficult.

Further, in the above-mentioned Japanese Patent Application Laid-Open No. 7-250350, there is described an embodiment in which one compression path is used for both left and right image signals.
The fourteenth to twentieth rows indicate that “each image data obtained by photographing from a plurality of directions is encoded for each image data by the encoding unit 403, and the encoded data of the image is encoded. By adding a code for identifying each imaging direction to each of the above, the decoding device can easily determine the imaging direction by referring to the identification code for each imaging direction. " I have. Therefore, it is necessary to wait until the encoding and compression of the other image signal is completed until the encoding and compression of the other image signal is completed, which is not suitable for a system where real time is important such as broadcasting. It is necessary to hold the image for the time to wait, the size of the holding circuit such as a delay memory added for time alignment becomes large, and the encoder side discriminates the image data in each shooting direction. Codes need to be generated and added, and the decoder side needs to identify each shooting direction of image data with reference to the identification code, which has a drawback that the circuit configuration becomes complicated.

[0029]

As described above, in the conventional three-dimensional image system, although the left and right image signals have a correlation, they are separately encoded, so that the compression efficiency deteriorates. If encoding is performed separately for the left and right, processing becomes difficult to synchronize the left and right image signals by producing a time base signal when decoding. Furthermore, since the left and right image signals are separately encoded, the circuit scale is about twice that in the case of encoding one system of image signals.

When encoding the left and right image signals, it is necessary to generate and add an identification code to the image data in each shooting direction, even if one encoding path is shared. There is a problem in that a waiting time is required to perform the compression, the real-time property is lacking, and the scale of a holding circuit added for time alignment becomes large.

In view of the above problems, the present invention can encode and decode image data from a plurality of directions with a simple circuit configuration using a single system of encoding and decoding paths. It is an object of the present invention to provide a stereoscopic image encoding / decoding method and apparatus.

[0032]

According to a first aspect of the present invention, there is provided a stereoscopic image encoding method in which image signals of a plurality of channels obtained by photographing from a plurality of directions are sequentially arranged for each frame or each field. Encode as one image signal.

In the stereoscopic image encoding method according to the second aspect of the present invention, two-channel image signals obtained by photographing from the left and right directions are alternately inserted for each frame or each field, and are used as one image signal. , B, and P pictures, the pictures are divided into GOPs each composed of N pictures, and the number of pictures N in one GOP is set to an odd number.

In the stereoscopic image encoding method according to the third aspect of the present invention, two-channel image signals obtained by photographing from the left and right directions are alternately inserted for each frame or one field, and are used as one image signal. , B or P picture, the period M in which the I or P picture appears in this picture sequence is set to an odd number.

In the stereoscopic image encoding method according to a fourth aspect of the present invention, N-channel image signals obtained by photographing from a plurality of directions are sequentially arranged for each frame or each field, and each image signal is output as one image signal. , B, and P pictures, image signals for N channels are divided into groups having the same number. First, each picture of I (or P) and P in the first group is I (or P). A picture is generated from the image signal of the first channel, and the picture signal of the other channel at the same time is predicted as a P picture from the picture. In the second group, similarly, the image signal of the second channel for I (or P) and P From the other channel of the same time at the same time in the Nth group. As such a predictive image signal, within each group predicts image signals of the other channels at the same time from an image signal of a channel different from each for each group.

In the stereoscopic image encoding method according to a fifth aspect of the present invention, N-channel image signals obtained by photographing from a plurality of directions are sequentially arranged for each frame or each field, and the image signals are output as I image signals. , B, and P pictures, the image signals for the N channels are divided into groups having the same number, and each of the groups arranged in the time axis direction is always assigned to the same two channels at the same time in any group. Image signals constitute I (or P) and P pictures, and the image signals of the remaining channels in the same group are encoded with B pictures.

According to a sixth aspect of the present invention, there is provided a stereoscopic image decoding method comprising the steps of: inputting an encoded image signal encoded by the stereoscopic image encoding method according to any one of the first to fifth aspects; The encoded image signal is decoded, and each frame or field is alternately extracted from the decoded image signal to obtain an image signal of a plurality of channels.

In the first aspect, image signals of a plurality of channels can be sequentially arranged for each frame or field, which is an image processing unit, and can be encoded as one image signal. In order to serialize a plurality of channel signals, it is only necessary to arrange the signals one frame or one field at a time, so that the control method is relatively simple.

In the second invention, in the case of MPEG which is divided into GOPs composed of N pictures and encoded, 1 GOP
If the number of pictures N is odd, the first picture (that is, the I picture) that appears for each GOP alternates between the left-eye image signal and the right-eye image signal. It is possible to suppress the error propagation on a GOP basis without transmitting the influence of the error of the channel of the first GOP to the next GOP.

According to the third aspect of the present invention, when the image signals of two channels for the left eye and the right eye are unified and encoded, the period M in which the I or P picture appears is made an odd number.
For an I or P picture, if the first I picture is an image signal for the left eye, the next P picture is an image signal for the right eye, and similarly, the next P picture is an image signal for the left eye,
The next P picture is a right-eye image signal,..., And so on, and P pictures predicted forward alternately appear in two channels of a left-eye image signal and a right-eye image signal, and are arranged between these I or P pictures. A B-picture to be predicted is predicted from both left-eye and right-eye image signals, that is, an image signal that always includes the same channel. When an error occurs in one channel, the degree of propagation of the error to the other channel can be reduced as compared with the case of predicting.

In the fourth invention, image signals for N channels are divided into groups having the same number, and each group has I signals.
Since (or P), P, the image signal of another channel at the same time is predicted from the image signal of a different channel for each group, there is an effect that prediction is easy to be performed and error propagation is suppressed.

In the fifth invention, the image signals for N channels are divided into groups having the same number, and within each group, I (or P) and P pictures are composed of two channels at the same time in the time axis direction for each group. Therefore, there is an effect that prediction is easy to hit.

According to the sixth aspect of the present invention, at the time of decoding, since it is known which channel is transmitted first after decoding as one image signal, one frame is transmitted from the time when the transmission signal is detected. Alternatively, by extracting image signals for each field, image signals of a plurality of channels can be easily decoded. This can be realized by a relatively simple control method without adding or decoding a code for identifying each channel.

[0044]

Embodiments of the present invention will be described with reference to the drawings. In the embodiment of the present invention, a case where two kinds of images (left image signal and right image signal) observed from two directions of left and right are used for ease of explanation and understanding, Even when three or more types of observed images are used, it can be easily extended.

FIG. 1 is a block diagram showing a transmission system according to the stereoscopic image encoding / decoding method and apparatus of the present invention.

In FIG. 1, in the coding apparatus, left-eye and right-eye image signals are input to input terminals 31 and 32, respectively, and the left-eye image signal is input to an input terminal a of a multiplexer 34 as it is. The right-eye image signal is input to the input terminal b of the multiplexer 34 via the delay memory 33 of one frame delay or one field delay. The delay memory 33 is
If the left and right image signals are frame images, the signal input for the frame time is delayed, and if it is a field image, the field time is delayed. The multiplexer 34 alternately switches between the a-side and the b-side in alternately arranged time units (every frame or field time). By this switching, two image signals are converted into one image signal and output. In the multiplexer 34, the a-side and the b-side are alternately turned on by the switching signal of the frame period or the field period from the control unit 36. The signal output as one image signal from the multiplexer 34 is subjected to variable-length coding by the MPEG encoder 35 and output to a transmission path (not shown).

In the decoding device, after the transmitted variable-length coded signal is decoded by the MPEG degoder 37, the decoded image signal is supplied to the input terminal of the demultiplexer 38. The demultiplexer 38 alternately switches between the two output terminals c and d in alternately arranged time units (one frame or one field time). The control means 42
A switching signal having a frame period or a field period is output from the time when the decoded signal from the EG decoder unit 37 is detected, and the demultiplexer 38 is switched between the c-side and the d-side. Since it is known in advance which of the left and right channels is the first image when the image is sequentially sequenced on the encoder side,
When detecting the decoded signal, the control means 42 alternately switches between the c side and the d side. The left image output from the output terminal c of the demultiplexer 38 is output to the output terminal 40 via the delay memory 39 of one frame delay or one field delay, and the right image output from the output terminal d of the demultiplexer 38 is directly output to the output terminal. 41. Delay memory 39
Also, similarly to the delay memory 33, if the left and right image signals are frame images, the input signal is delayed by the frame time,
The input signal is delayed for the field time even if it is a field image.

In such a configuration, at the time of encoding, an L-channel image signal obtained by photographing from the left direction,
A signal obtained by delaying the R channel image signal obtained by photographing from the right direction by one frame or one field by the delay memory 33 is alternately inserted for each frame or one field by the multiplexer, and one image signal is obtained. After doing
This is encoded by the MPEG encoder 35 and output to the transmission path.

At the time of decoding, the coded image signal coded as described above is inputted, and the MPEG decoder 37
, And each frame or field is alternately extracted by the demultiplexer 38 from the decoded image signal.
For the channel image signal, 1
By outputting with a frame or one field delay and outputting the R channel image signal as it is,
The image signals of the left and right channels are synchronized and output as in the case of shooting.

According to FIG. 1, since both channels are encoded as one left and right image signal, only one MPEG encoder 35 and one decoder 37 are required.
It is needless to say that the circuit scale can be reduced to about one half of the circuit size.

At the time of decoding (at the time of reproduction), L
If it is known which channel is the first image of the left and right image signals in which the R and R channel images are arranged, the control unit 3
8 and the demultiplexer 38 and the delay memory 39, it is possible to easily synchronize the left image signal and the right image signal, and there is no need for complicated processing such as addition or decoding of synchronous data as in the conventional example. It can be done with simple control.

The delay memory 33 in the apparatus shown in FIG.
The delay memory 39 is an example of a device when the first channel of the left and right image signals is L. When the first channel is R, the delay memory 33 is on the left image input signal side and the delay memory 39 is on the right image output side. The arrangement can be easily inferred, and the present invention includes this configuration.

In FIG. 1, a case has been described in which two-channel image signals obtained by photographing from two directions, left and right, are encoded and decoded. A similar transmission system can be configured for image signals of a plurality of channels obtained by photographing from different directions.

FIG. 2 is a block diagram in which the two-channel transmission system of FIG. 1 is extended to a case of a plurality of channels.

In the encoding apparatus shown in FIG.
Is a serializing means for sequentially delaying image signals Si1, Si2,... Sin of a plurality of channels obtained by photographing from a plurality of directions by one frame or one field for each channel. The image signal of each channel, which has been delayed one by one, is sequentially inserted by the multiplexer 302 for each frame or field, and
Are output as two image signals. The control means 304 outputs a switch signal of a frame cycle or a field cycle, and controls the multiplexer 302 so as to sequentially take out signals of a plurality of channels from the serialization means 301. One image signal from the multiplexer 302 is encoded by the encoding means 303.
And output to a transmission path (not shown).

In the decoding device, the decoding means 305 inputs and decodes the above-mentioned coded image signal. Thereafter, each frame or field is sequentially extracted from the decoded image signal by the demultiplexer 306, and the sequentially extracted image signal of each channel is sequentially synchronized by the synchronizing means 307 one frame or one field at a time. , Image signals So1, S of a plurality of channels similar to those at the time of shooting
o2, ... Output as Son.

According to the configuration of FIG. 2, image signals of a plurality of channels exceeding two channels can be easily encoded and decoded by one encoder unit and one decoder unit.
A simple circuit configuration is sufficient, and at the time of decoding (at the time of reproduction), if it is known which channel is the first image of the sequentially arranged image signals of a plurality of channels, the control means 30
8 and the demultiplexer 306 and the synchronizing means 307 make it possible to easily synchronize the left image signal and the right image signal without the need for complicated processing such as addition and decoding of synchronous data as in the conventional example. It can be done with simple control.

FIGS. 3A and 3B are diagrams showing a first embodiment of the stereoscopic image encoding method according to the present invention. FIG. 3A shows the picture assignment when the left and right image signals are encoded for each frame or field, and FIG. 3B shows the prediction direction in the case of encoding as shown in FIG. 3A. here,
A case in which encoding is performed using I, B, and P pictures will be described. The I-picture is an intra-coded frame, and the P-picture is an inter-coded frame predicted from an already-coded temporally previous I-picture. P
A picture can also be obtained by forward predictive coding of a subsequent P picture. The B picture is a bidirectional prediction adaptive switching frame obtained by performing bidirectional prediction coding on the I or P picture in either the forward or backward direction or in both directions.

FIG. 3A shows the encoding apparatus shown in FIG.
An example is shown in which a left image signal L and a right image signal R are alternately arranged, and a period M in which an I or P picture appears is set to M = 2, and encoding is performed with one image signal. The numbers below L and R are
The order of the numbers indicates the time order, and the same number indicates the same time. L and R are hereinafter referred to as channels. That is, the L channel is a left image signal before being alternately arranged, and the R channel is a right image signal before being alternately arranged. For example, L1 and R1 represent the first image signals of the L and R channels at the same time.

Further, as shown in FIG. 3 (b), since encoding is performed with M = 2, the image signal of each channel can be predicted from the image signal of another channel at the same time having a high degree of redundancy (correlation). . For example, R1 (B picture) is L1 at the same time.
(I picture) can be predicted from the image signal. Similarly, R2 (B picture) can be predicted from L2 (P picture) at the same time. This is equivalent to encoding the parallax image signal, and a compression efficiency equal to or higher than that of the conventional example (FIGS. 16 (b) and 18 (b)) can be obtained.

FIGS. 4A and 4B are views showing a second embodiment of the stereoscopic image encoding method according to the present invention. FIG. 4A shows the picture assignment when the left and right image signals are encoded for each frame or field, and FIG. 4B shows the prediction direction in the case of encoding as shown in FIG. 4A. As shown in FIG. 4B, the left image signal L and the right image signal R are alternately arranged.
Encoding is performed with one image signal by setting the cycle M of the I or P picture to M = 3. When M = even as in the embodiment of FIG. 3, the first channel of the left and right image signals (FIG.
Channels that are not L) as shown in (b) (R in FIG. 3)
Is always predicted from the image signal of the first channel (L). If an error is included in the first channel L at a certain time, the other channel R
The error cannot be removed from the right image signal unless the error is propagated and the left image signal is repaired. This is because, in FIG. 3, I, B, and P pictures are assigned when encoding the image signals of the L and R channels, but an I picture is assigned instead of a P picture, and an I, B picture is assigned. The same is true for the case of the configuration.

One of the methods for suppressing such error propagation is as follows.
One is that M = odd number shown in the second embodiment of FIG. Here, in FIG. 4, when encoding the image signals of the L and R channels, I, B and P pictures are assigned, and the L1 which is an I picture is an inter-frame coded image (P picture). Since R2 is predicted forward, L
If there is an error in the channel, the error may be propagated to the R channel. However, if an intra-frame coded image (I-picture) is assigned instead of a P-picture,
If it is composed of only pictures, there is no error propagation from L1 to R2. As can be seen from FIG. 4 (b), R1 and L2 as B pictures are L1 and R as I pictures.
2 can be predicted from one or both of L, R
Can be predicted from the same channel, and error propagation from other channels can be suppressed. That is, by setting M = odd number, an effect of suppressing error propagation is produced.

FIGS. 5A and 5B show a third embodiment of the stereoscopic image encoding method according to the present invention. The third embodiment shows another method for suppressing error propagation.

In MPEG, a GOP (Gr.
oup of Picture). Usually, the number of pictures constituting this GOP is represented by N. MPEG
According to the rules, the first picture of a GOP (note that it is the first picture to be encoded and not the first picture in the time axis direction) must be an I picture.

FIG. 5A shows an example where N = even number, and FIG.
Shows an example where N = odd number. In the case of FIG. 5 (a), L, R
In the case of the two channels, if the leading picture of a certain GOP is an L-channel image signal, the leading picture of the next GOP is also an L-channel image signal. Therefore, if an error occurs in the L-channel image signal during shooting, the error propagation cannot be kept within the GOP.

In the case of MPEG, as shown in FIG.
= Odd number makes it possible to alternately arrange the L and R channels of the first picture of a GOP for each GOP. If a certain GOP starts encoding from the L channel, the next GOP encodes from the R channel. Will be started. As a result, even if an error occurs in the L channel image signal of the first picture in a certain GOP, the error does not affect the R channel image signal of the first picture in the next GOP. That is, it is possible to suppress the influence of the error of one channel on a GOP basis.

Therefore, in the case of the left and right image columns in which the left image signal and the right image signal are alternately arranged as in the first and second embodiments, that is, in the case of a two-channel array, N is set to an odd number. Thus, L and R can be alternately arranged in the first picture of each GOP, and error propagation can be suppressed.

In the third embodiment shown in FIG. 5, M = 3 and N =
7 is assumed. Note that the present invention is not limited to a method of alternately arranging L and R in the first picture of each GOP, where N = odd.

FIGS. 6A and 6B are diagrams showing a fourth embodiment of the stereoscopic image encoding method according to the present invention. FIG. 6A shows the picture assignment when the left and right image signals are encoded for each frame or field, and FIG. 6B shows the prediction direction in the case of encoding as shown in FIG. 6A.

In MPEG, what is about to be generally used at present is MPEG2 of the second specification.
In MPEG2, the GOP structure is optional,
It is not always necessary to have a GOP structure. Therefore, the fourth embodiment is a method of suppressing error propagation without using a GOP as a method of forming a group including a plurality of pictures.

Here, in order to explain the fourth embodiment, the word redundant distance = JD is defined. What is a redundant distance?
This is a value indicating the redundancy (correlation) between two pictures, and increases by 1 when the channel changes, and increases by 1 when the picture changes in the time axis direction. In FIGS. 3 to 7 and 11, three types of solid lines, dotted lines, and alternate long and short dash lines are used to indicate the prediction directions, which are changed depending on the value of the redundant distance. Solid line: JD = 1, dotted line: JD = 2, dashed line: JD = 3.

For example, in FIG. 4B, JD = 2 for L1 and R2, and JD = 3 for R2 and L4.

The larger the redundant distance is, the lower the redundancy (interphase) is, the more difficult it is to predict, and the lower the compression efficiency.

Now, consider the case where two B pictures are predicted from two I or P pictures, as in M = 3 in FIG. 4 (a).

As shown in FIG. 4B, when L3 is encoded, J
The prediction is made from R2 with D = 2, from L4 with JD = 1, or both. Considering the redundant distance, L
4 is easier to predict, but L4 is JD = 3
L3 can be predictively coded from L4 of the same channel, but since L4 is predictively coded from R2 of another channel which is JD = 3 away, the suppression rate of error propagation is not so large. not high. That is,
The prediction efficiency is low because the redundant distance of the P picture, which is one of the reference pictures from which the B picture is based, is large.

Therefore, in the fourth embodiment shown in FIG. 6, image frames or fields of each channel are divided into groups including the same number, and the first LR channel at the same time in the time axis direction in each group indicated by a dotted frame. The image signals (L1 / R1, L3 / R3, L5 / R5,...) Are set to I and P pictures, and from these pictures, the image signals (L2 / R2, L4 / L4 /
R4, L6 / R6,...) Are predicted as B-pictures.
Reduce the redundant distance of the picture (minimum ie JD = 1
2) to increase the prediction efficiency and suppress error propagation. In FIG. 6, for example, L1 is encoded with an I picture, and R1 with a redundant distance JD = 1 is forward predicted as a P picture, and L2 and R2 are bidirectionally predicted from L1 and R1, which are I and P pictures. . This will be described in detail below.

In FIG. 6, (L1 / R1)
L2 / R2, L3 / R3 L4 / R4, L5
/R5.L6/R6),..., The two channels at the same time are made into one set, and two sets are made into one group. The I and P pictures serving as reference images for each group are as follows:
Since they are the first LR channel image signal in the time axis direction, they are L1 / R1, L3 / R3, L5 / R5,.

First, the first group (L 1 / R 1 · L 2
/ R2). After encoding L1 with an I picture, L1
, And R1 is encoded as a P picture. At this time, since the redundant distance of R1 from L1 is JD = 1, which is the minimum, the correlation is very high and the prediction is very easy to hit.

The image signals L2 and R2 of the remaining channels of the first group are predicted as B pictures from both the L1 and R1 channels. Prediction is possible, prediction is very easy to hit, and error propagation can be suppressed.

The second group (L 3 / R 3 · L 4 / R 4
In), either L3 or R3 is first coded with an I or P picture.

In the case of an I picture, the same prediction is performed as in the first group. In the case of a P picture, prediction is performed from the previous I or P picture in the first group. FIG.
In this example, R3 is set as the first picture of the second group, and prediction is performed from R1, which is a P picture of the previous group. At this time, the redundant distance JD from R1 to R3 is JD = 2.

In the fourth embodiment shown in FIG. 6, the leading pictures of each group are alternately arranged by L and R as L1, R3, L5, R7,. When all I pictures are used, the influence of an error of one channel can be suppressed, and the suppression rate of error propagation can be increased.

In the fourth embodiment shown in FIG. 6, one group is composed of two sets of LR channel image signals.
The present invention is not limited to two sets but can be extended to two or more sets.

FIGS. 7 (a) and 7 (b) show a fifth embodiment of the stereoscopic image encoding method according to the present invention. Fig. 7 (a)
FIG. 7B shows the picture assignment when the left and right image signals are encoded for each frame or field, and FIG. 7B shows the prediction direction in the case of encoding as shown in FIG. 7A. this is,
FIG. 6 shows a case where the fourth embodiment of FIG. 6 is extended to three sets of LR channel image signals.

Although the fourth and fifth embodiments shown in FIGS. 6 and 7 have been described with reference to the LR two-channel image signal, the present invention is not limited to the LR two-channel image signal, and the present invention is not limited thereto. Can be extended to the image signal of

FIG. 8 shows a sixth embodiment of the stereoscopic image encoding method according to the present invention. This is the fourth in FIG.
Is shown extended to three channels of L, C (center) and R. FIG. 8 (a) shows the observation points of three channels, and FIG. 8 (b) divides the image signal (image frame or field) of three channels into groups including two sets, and alternates L, C, and R. 2 shows an arrangement of image signals in the case of encoding.

First, I (or P), P in the first group
For each picture, an I picture is generated from the L channel image signal (L1), and an R channel image signal (R1) at the same time is predicted from the picture as a P picture. For the P) and P, the L-channel image signal (L3) at the same time is predicted from the R-channel image signal (R3). In the third group, the C-channel image signal (I or P) and P are similarly calculated. C5) to the L-channel image signal (L
As in 5), for I (or P) and P, image signals of other channels at the same time are predicted from image signals of different channels in each group.

Also in the case of the sixth embodiment shown in FIG. 8, since a channel at the same time is predicted from a certain channel in each group, the prediction is easy to hit. Since the head picture of each group is an image signal of a different channel alternately, if all the head pictures of each group are I pictures, an error occurs in the channel corresponding to the head picture at the time of shooting. However, the error propagation stays within the group, and the influence on other channels can be suppressed.

FIG. 9 shows a seventh embodiment of the stereoscopic image encoding method according to the present invention. This is the fourth in FIG.
Has been extended to the four channels L, LS, RS and R. FIG. 9 (a) shows observation points of four channels, and FIG. 9 (b) divides the image signal (image frame or field) of four channels into groups each including two sets of L, LS, RS, and R. Are alternately encoded on the L side and the R side.

First, I (or P), P in the first group
For each picture, an I picture is generated from the L channel image signal (L1), and an R channel image signal (R1) at the same time is predicted from the picture as a P picture. P) and P, the image signal (L3) of the L channel at the same time is predicted from the image signal (R3) of the R channel. In the third group, the image signals of the RS channel (I or P) and P are similarly calculated. RS5), the L-channel image signal (L5) at the same time is predicted. Similarly, in the fourth group, for I (or P) and P, the R-channel image signal at the same time from the LS channel image signal (LS7). In order to predict (R7), I (or P) and P in each group are obtained from image signals of different channels for each group and other channels at the same time. It will predict the image signal of the channel.

Also in the case of the seventh embodiment shown in FIG. 9, since a channel at the same time is predicted from a certain channel in each group, the prediction is easy. Since the head picture of each group becomes an image signal of a different channel alternately, if all the head pictures of each group are I pictures, even if an error occurs in the channel corresponding to the head picture at the time of shooting, , The error propagation stays within the group, and the influence on other channels can be suppressed.

FIG. 10 shows an eighth embodiment of the stereoscopic image encoding method according to the present invention. This shows a case where the fourth embodiment of FIG. 6 is extended to four channels of L, LS, RS, and R. However, in FIG. 10, the image signals (image frames or fields) of four channels at the same time are divided into groups each including one set, and L, LS,
This shows the arrangement of image signals when RS and R are alternately encoded on the L and R sides.

First, I (or P), P in the first group
For each picture, an I picture is generated from the L channel image signal (L1), and an R channel image signal (R1) at the same time is predicted from the picture as a P picture. P) and P, the LS channel image signal (LS2) at the same time is predicted from the R channel image signal (R2). Similarly, in the third group, the I (or P) and P RS channel image signals (L Rs3), the L-channel image signal (L3) at the same time is predicted. Similarly, in the fourth group, I (or P) and P are converted from the LS-channel image signal (LS4) to the Rs-channel image signal at the same time. In order to predict (RS4), I (or P) and P in each group are determined based on the image signals of different channels for each group. To predict the image signal of the tunnel.

In the case of the seventh embodiment shown in FIG.
The prediction is easy because a channel at the same time is predicted from a certain channel in each group. Since the first picture of each group becomes an image signal of a different channel alternately, if the first picture of each group is
In the case of a picture, even if an error occurs in the channel corresponding to the first picture at the time of shooting, the error propagation remains in the group, the effect on other channels can be suppressed, and the error occurs at a certain time. Even if it occurs,
Since the pictures in the group are only the image signals at the same time, the error propagation is only for the pictures at the same time, and the error does not propagate to the image signals at different times in other groups. .

According to the above-described embodiments of FIGS. 6 to 10, the image signals for N channels are divided into groups having the same number, and within each group, I (or P) and P are different for each group. Since the image signal of the other channel at the same time is predicted from the image signal of the changed channel, the prediction is easily performed, and there is an effect of suppressing error propagation.

In the embodiments shown in FIGS. 6 to 10, in each group, I (or P) and P are used to predict image signals of other channels at the same time from image signals of different channels for each group. Was to do,
The arrangement of the LR image signal sequence for I (or P) and P in each group may be configured to be unified.

FIGS. 11 (a) and 11 (b) show a ninth embodiment of the stereoscopic image encoding method according to the present invention. FIG.
FIG. 11A shows the picture assignment when the left and right image signals are encoded for each frame or field, and FIG.
It shows the prediction direction when encoding as shown in FIG.

The ninth embodiment of FIG. 11 shows an example in which the arrangement of the LR image signal strings of each group is unified. L
Regarding two channels of R, in any group,
For I (or P) and P, the first picture I (or P) is an L channel image signal, and the P picture predicted forward from this is an R channel image signal at the same time.

First, I (or P), P in the first group
For each picture, an I picture is generated from the L channel image signal (L1), and an R channel image signal (R1) at the same time is predicted from the picture as a P picture. P) and P, the image signal (R3) of the R channel at the same time is predicted from the image signal (L3) of the L channel, and similarly in the third group, the image signal (L) of the L channel for I (or P) and P L5) to the R-channel image signal (R
5) To predict the image signal of another channel (R) at the same time from the image signal of the same channel (L) at all times in each group for I (or P) and P, such as predicting I). become.

In this case, since only the image signal of one channel, such as the L channel to the R channel, becomes the reference image in the prediction, the prediction is easy to hit, but the error propagation suppression rate is not high.

In the ninth embodiment shown in FIG.
Although the description has been given of the image signal of two channels of R, the present invention can be extended to an image of two or more channels.

FIG. 12 shows a tenth embodiment of the stereoscopic image encoding method according to the present invention. This is shown in FIG.
Is shown extended to three channels of L, C (center) and R. FIG. 12 (a) shows observation points of three channels, and FIG. 12 (b) divides the image signal (image frame or field) of three channels into groups each including two sets, and alternates L, C, and R. 2 shows an arrangement of image signals in the case of encoding.

First, I (or P), P in the first group
For each picture, an I picture is generated from the L channel image signal (L1), and an R channel image signal (R1) at the same time is predicted from the picture as a P picture. P) and P, the image signal (R3) of the R channel at the same time is predicted from the image signal (L3) of the L channel, and similarly in the third group, the image signal (L) of the L channel for I (or P) and P L5) to the R-channel image signal (R
As in 5), for each of the groups I (or P) and P, the image signal of a predetermined other channel at the same time is predicted from the image signal of the same channel in any group. The image signals of the remaining channels in the same group are encoded with B pictures.

Also in the case of the tenth embodiment shown in FIG. 12, the prediction is easy to hit, but since the first picture of each group is always an image signal of the same channel, an error occurs in the channel corresponding to the first picture during shooting. If an error occurs, the error is propagated to other groups, so the error propagation suppression rate is not high.

FIGS. 13 (a) and 13 (b) show an eleventh embodiment of the stereoscopic image encoding method according to the present invention. This is different from the ninth embodiment shown in FIG. 11 in that L, LS, RS, R
4 shows a case where the four channels are expanded. FIG.
(a) shows the observation points of the four channels, and FIG.
(b) divides a four-channel image signal (image frame or field) into groups including two sets, and outputs L, LS,
This figure shows the arrangement of image signals when RS and R are alternately encoded on the L and R sides.

For each of the I (or P) and P pictures in each of the first, second,... Groups, the I (or P) picture is assigned to the L channel as in the tenth embodiment of FIG. An image signal is generated from an image signal, and an R-channel image signal at the same time is predicted from the picture as a P picture. In the first group, I (or P) and P are L1
, R1 in the second group, R3 in the second group, R3 in the third group, and R5 in the third group. From the other image signals at the same time. The image signals of the remaining channels in the same group are encoded with B pictures.

In the case of the eleventh embodiment shown in FIG. 13, the prediction is easy, but since the first picture of each group is always an image signal of the same channel, an error occurs in the channel corresponding to the first picture during shooting. If an error occurs, the error is propagated to other groups, so the error propagation suppression rate is not high.

FIG. 14 shows a twelfth embodiment of the stereoscopic image encoding method according to the present invention. This is shown in FIG.
9 shows a case where the ninth embodiment is extended to four channels L, LS, RS, and R. However, FIG. 14 shows an arrangement of image signals when the image signals (image frames or fields) of the four channels at the same time are divided into groups each including one set and encoded.

For each of the I (or P) and P pictures in each of the first, second,... Groups, the I (or P) picture is converted from the L channel image signal as in the embodiment of FIG. Then, the picture signal of the R channel at the same time is predicted from the picture as a P picture. First
In the group, I (or P) and P are from L1 to R1
And the second group predicts R2 from L2,
In the third group, for each of the groups I (or P) and P, the image signal of a predetermined other channel at the same time is predicted from the image signal of the same channel in any group, such as predicting R3 from L3. become. The image signals of the remaining channels in the same group are encoded with B pictures. Moreover, each group is composed only of the image signals of four channels at the same time.

In the case of the twelfth embodiment shown in FIG. 14, prediction is easy, but since the first picture of each group is always an image signal of the same channel, an error occurs in the channel corresponding to the first picture during shooting. If an error occurs, the error is propagated to other groups, so the error propagation suppression rate is not high. However, if the first picture of each group is an I picture, even if an error occurs at a certain time, the only picture in the group is an image signal at the same time. , Ie, only in the same group, and no error propagation occurs in other groups, ie, for image signals at different times.

[0111]

As described above, according to the present invention, image signals of a plurality of channels are sequentially arranged for each frame or field as an image processing unit and are encoded as one image signal. Therefore, it is possible to improve the encoding efficiency when encoding a stereoscopic image, and also, when decoding as a single image signal when decoding, one frame or one field at a time. By extracting the image signals, it is possible to easily synchronize the image signals of a plurality of channels. Further, since a single-system encoding path and decoding path are used and the synchronization circuit is simplified, the circuit scale of the stereoscopic image encoding / decoding device can be reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an image transmission system according to a stereoscopic image encoding / decoding method and apparatus of the present invention.

FIG. 2 is a block diagram in which the transmission system of FIG. 1 is applied to a plurality of channels.

FIG. 3 is a diagram showing a first embodiment of a stereoscopic image encoding method according to the present invention.

FIG. 4 is a diagram showing a stereoscopic image encoding method according to a second embodiment of the present invention.

FIG. 5 is a diagram showing a third embodiment of the stereoscopic image encoding method of the present invention.

FIG. 6 is a diagram showing a fourth embodiment of the stereoscopic image encoding method of the present invention.

FIG. 7 is a diagram showing a fifth embodiment of the stereoscopic image encoding method of the present invention.

FIG. 8 is a diagram showing a sixth embodiment of the stereoscopic image encoding method of the present invention.

FIG. 9 is a diagram showing a seventh embodiment of the stereoscopic image encoding method of the present invention.

FIG. 10 is a diagram showing an eighth embodiment of the stereoscopic image encoding method according to the present invention.

FIG. 11 is a diagram showing a ninth embodiment of the stereoscopic image encoding method according to the present invention.

FIG. 12 is a diagram showing a stereoscopic image encoding method according to a tenth embodiment of the present invention.

FIG. 13 is a diagram showing an eleventh embodiment of the stereoscopic image encoding method according to the present invention.

FIG. 14 is a diagram showing a twelfth embodiment of the stereoscopic image encoding method according to the present invention.

FIG. 15 is a view for explaining stereoscopic video recognition by humans.

FIG. 16 is a diagram showing a stereoscopic image system.

FIG. 17 is a diagram showing an outline of an image transmission system based on MPEG.

FIG. 18 is a block diagram illustrating a conventional stereoscopic image encoding / decoding method and apparatus.

[Explanation of symbols]

31 left-eye image signal input terminal, 32 right-eye image input terminal, 33 delay memory, 34 multiplexer, 35
MPEG encoder, 37 ... MPEG decoder, 3
8 demultiplexer, 39 delay memory, 40 left-eye image signal output terminal, 41 right-eye image output terminal, 301 serializing means, 302 multiplexer, 303 encoding means,
305 decoding means, 306 demultiplexer, 307 synchronization means, Si1, Si2, ..., Sin input image signals of a plurality of channels, So1, So2, ..., Son output image signals of a plurality of channels

Claims (8)

[Claims] 1. A stereoscopic image code, wherein image signals of a plurality of channels obtained by photographing from a plurality of directions are alternately inserted for each frame or each field and encoded as one image signal. Method. 2. When two-channel image signals obtained by photographing from the left and right directions are alternately inserted for each frame or field, and encoding is performed using I, B, and P pictures as one image signal. To the G consisting of N pictures
A stereoscopic image encoding method, wherein an image is divided into OPs, and the number N of pictures of one GOP is set to an odd number. 3. When two-channel image signals obtained by photographing from the left and right directions are alternately inserted for each frame or field, and encoding is performed using I, B or P pictures as one image signal. A period M in which an I or P picture appears in the picture sequence is set to an odd number. 4. An N-channel image signal obtained by photographing from a plurality of directions is inserted alternately for each frame or each field, and is encoded as one image signal using I, B, and P pictures. First, the image signals for the N channels are divided into groups having the same number.
For each picture (or P) and P, an I (or P) picture is generated from the image signal of the first channel, and from that picture, an image signal of another channel at the same time is predicted as a P picture. Similarly, I (or P),
For P, predict the image signal of the other channel at the same time from the image signal of the second channel. In the Nth group, similarly, for I (or P), P A stereoscopic image encoding method for predicting an image signal of another channel at the same time from an image signal of a different channel for each group within each group. 5. An N-channel image signal obtained by photographing from a plurality of directions is alternately inserted for each frame or each field, and is encoded by I, B, and P pictures as one image signal. In addition, the image signals for N channels are divided into groups having the same number, and for each group arranged in the time axis direction, the I (or P), P picture And encoding the image signals of the remaining channels in the same group with B pictures. 6. A coded image signal input by the stereoscopic image coding method according to claim 1, decoding the coded image signal, and decoding the decoded image. Alternately extract each frame or field from the signal,
A stereoscopic image decoding method characterized by obtaining image signals of a plurality of channels. 7. A means for sequentially delaying image signals of a plurality of channels obtained by photographing from a plurality of directions by one frame or one field for each channel, and an image of each channel delayed by one frame or one field A stereoscopic image encoding apparatus, comprising: means for alternately inserting signals and outputting as one image signal; and means for encoding the signal output as one image signal. 8. A means for inputting and decoding an encoded image signal encoded by the stereoscopic image encoding apparatus according to claim 7, and means for alternately extracting each frame or field from the decoded image signal. Means for sequentially delaying and synchronizing the image signal of each channel sequentially extracted for each frame or field by one frame or one field and outputting the image signal as a plurality of channel image signals. Image decoding device