JP2004048293A - Stereoscopic image compressing or decompressing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a stereoscopic image compressing or decompressing apparatus for efficiently compressing and transmitting image data comprising a stereoscopic image and a high resolution planar image in addition to the stereoscopic image. <P>SOLUTION: An image configuration conversion section 104 of a stereoscopic image pre-processing section 103 receives a left eye image signal 101 and a right eye image signal 102 and multiplexes them to configure one image, a frame/field conversion section 105 configures an interlace image, an MPEG compressor 106 compresses the interlace image according to the MPEG format, and a transmission/recording section 107 records the compressed image to a storage and transmits the compressed image to the outside. A compressed stereoscopic image signal received from the outside and reproduced from the storage is decompressed by an MPEG expansion section 109 in compliance with the MPEG via a reception/reproduction section 108, and the interlace image is reproduced. A field/frame conversion section 111 of a stereoscopic image post-processing section 110 reproduces a frame image from the interlace image, and a pixel configuration conversion section 112 reconfigures the frame image into a left eye image signal 113 and a right eye image signal 114 and outputs the resultant signals. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention provides a three-dimensional image, a three-dimensional image and a two-dimensional image, a three-dimensional image and a high-resolution two-dimensional image, and compresses or transmits image data composed of a three-dimensional image and a high-resolution additional image as efficiently as possible. The present invention relates to a stereoscopic image compression or decompression device.
[0002]
[Prior art]
As a conventional compression method for a stereoscopic image signal, for example, a “stereoscopic image transmission method and apparatus” disclosed in Japanese Patent Application Laid-Open No. H11-18111 has been patented.
[0003]
FIG. 20 shows a configuration example according to the present invention.
[0004]
The three-dimensional image 2001 includes a left-eye image (）) 2002 and a right-eye image (x) 2003. These are arranged on the display as shown at 2001. That is, the left-eye image and the
A right eye image is placed. In the example of FIG. 20, 176 pixels × 288 lines of the left eye image and 176 pixels × 288 lines of the right eye image are shown, for a total of 352 pixels × 288 lines.
[0005]
The stereoscopic image 2001 is converted like a converted image 2004. That is, on the screen of 352 pixels × 288 lines, 176 pixels × 288 lines of the left eye image are arranged in the left half, and 176 pixels × 288 lines of the right eye image are arranged in the right half.
[0006]
Then, processing such as MPEG compression 2005, transmission / recording 2006, reception / reproduction 2007, and MPEG decompression 2008 is performed on the converted image 2004.
[0007]
As the image signal output from the processing of the MPEG decompression 2008, as shown in 2009, the same stereoscopic image 2009 as the input signal of the stereoscopic image 2001 is reproduced.
[0008]
[Problems to be solved by the invention]
However, the conventional example has the following disadvantages.
[0009]
In the signal of the converted image 2004, a left-eye image is arranged in the left half and a right-eye image is arranged in the right half. Therefore, when performing MPEG compression, the correlation between the left-eye image and the right-eye image cannot be applied. That is, in the stereoscopic image, the near object is data in which the left-eye image and the right-eye image are separated, and the distant object is data in which the left-eye image and the right-eye image are close to each other. It is possible to increase the data compression efficiency by using the correlation between the image data and the right-eye image, but this cannot be realized by the compression method as in the conventional example.
[0010]
When a stereoscopic image is compressed and transmitted to be displayed on a stereoscopic image display, it is necessary to consider displaying the same signal on a high-resolution planar image display at the same time. That is, it is important to efficiently compress and transmit image data composed of a stereoscopic image and a high-resolution plane additional image in addition to the stereoscopic image, but in the conventional example, this is not considered at all.
[0011]
An object of the present invention is to provide a three-dimensional image compression or decompression device that efficiently compresses and transmits image data composed of a three-dimensional image and a high-resolution two-dimensional image in addition to a three-dimensional image.
[0012]
[Means for Solving the Problems]
The present invention has solved the above problems,
A stereoscopic image preprocessing unit that arranges the left-eye image on odd lines and the right-eye image on even lines, or the right-eye image on odd lines and the left-eye image on even lines, and uses one image as an interlaced image; Is selected as an optimal block from a block composed of data of odd lines and even lines, and a block composed of data of only odd lines or only even lines. When selecting an optimal prediction method from frame prediction from preceding and succeeding frames, or field prediction from preceding and succeeding fields, orthogonal transformation, quantization, and variable length coding are performed on the error between the block data predicted therefrom and the block data. As a result, by selecting a block and a prediction method that minimize the code amount of the block data, Image compression section for compression,
Alternatively, one of the left-eye image, the right-eye image, and the high-resolution additional image is one of an odd line of the (3n + 1) th frame (n is an integer of 0 or more), an even line of the (3n + 1) th frame, an odd line of the (3n + 2) th frame, an even line of the (3n + 2) th frame, The odd-numbered line of the 3n + 3 frame and the even-numbered line of the 3n + 3 frame,
The other one of the 3n + 1th frame odd line, the 3n + 1th frame even line, the 3n + 2 frame odd line, the 3n + 2 frame even line, the 3n + 3 frame odd line, and the 3n + 3 frame even line Place them in two,
Still another is not arranged among the 3n + 1th frame odd line, the 3n + 1th frame even line, the 3n + 2 frame odd line, the 3n + 2 frame even line, the 3n + 3 frame odd line, and the 3n + 3 frame even line. Place it on the other two,
A stereoscopic image preprocessing unit that uses an image arranged on an odd line and an image arranged on an even line of the same frame as an interlaced image, an image compression unit that compresses one image processed by the stereoscopic image preprocessing unit,
Accordingly, the left-eye image, the right-eye image, and the high-resolution additional image can be efficiently compressed using the respective correlations.
[0013]
(Action)
The present invention
A left-eye image is arranged on an odd-numbered line and a right-eye image is arranged on an even-numbered line, or a right-eye image is arranged on an odd-numbered line and the left-eye image is arranged on an even-numbered line.
From the block consisting of odd and even lines and the block consisting of only odd lines or even lines only, select the best block, and then predict the frame from the previous and next frames, or the previous and next fields When choosing the best prediction method from field predictions from,
An image to be compressed by selecting a block and a prediction method that minimizes the code amount of the block data as a result of orthogonal transformation, quantization, and variable-length encoding of the error between the block data predicted from the block data and the block data. Compression section,
Alternatively, one of the left-eye image, the right-eye image, and the high-resolution additional image is one of an odd line of the (3n + 1) th frame (n is an integer of 0 or more), an even line of the (3n + 1) th frame, an odd line of the (3n + 2) th frame, an even line of the (3n + 2) th frame, The odd-numbered line of the 3n + 3 frame and the even-numbered line of the 3n + 3 frame,
The other one of the 3n + 1th frame odd line, the 3n + 1th frame even line, the 3n + 2 frame odd line, the 3n + 2 frame even line, the 3n + 3 frame odd line, and the 3n + 3 frame even line Place them in two,
Still another is not arranged among the 3n + 1th frame odd line, the 3n + 1th frame even line, the 3n + 2 frame odd line, the 3n + 2 frame even line, the 3n + 3 frame odd line, and the 3n + 3 frame even line. Place it on the other two,
A stereoscopic image preprocessing unit that uses an image arranged on an odd line and an image arranged on an even line of the same frame as an interlaced image, an image compression unit that compresses one image processed by the stereoscopic image preprocessing unit,
Thereby, the left-eye image, the right-eye image, and the high-resolution additional image can be efficiently compressed by using the respective correlations.
It has the action of:
[0014]
BEST MODE FOR CARRYING OUT THE INVENTION
An embodiment of the present invention will be described with reference to FIGS.
[0015]
The configuration of the present invention will be described.
[0016]
FIG. 1 is a block diagram showing an example of a stereoscopic image compression / expansion apparatus for realizing the present invention in which a right-eye image having the same number of pixels and resolution as a left-eye image is input.
[0017]
The stereoscopic image compression / expansion device receives a left-eye image signal from 101 and a right-eye image signal from 102. These are progressive images each having a frame configuration. Here, when the stereoscopic image display can display 352 pixels × 288 lines, the left-eye image signals 101 and 102 are set to image signals of 176 pixels × 288 lines, respectively, and 176 pixels × 288 lines of the right-eye and left-eye can be viewed. .
[0018]
The left-eye image signal 101 and the right-eye image signal 102 are input to the image configuration conversion unit 104 of the stereoscopic image preprocessing unit 103, respectively. The image configuration conversion unit 104 multiplexes the left-eye image signal 101 and the right-eye image signal 102 to form one image, and then, in the frame / field conversion unit 105, virtually interlaces a Top Field and a Bottom Field. An image is composed. The operations of the image configuration conversion unit 104 and the frame / field conversion unit 105 in the stereoscopic image preprocessing unit 103 will be described later in detail again. The image signal output from the frame / field conversion unit 105 is compressed in an image compression format such as MPEG-2 or MPEG-4 by an MPEG (Moving Picture Expert Group) compression unit 106, and compressed by a transmission / recording unit 107. As a stereoscopic image signal, recording to a storage, transmission to the outside, and the like are performed.
[0019]
The compressed stereoscopic image signal received from the outside and reproduced from the storage is MPEG-decompressed by the MPEG decompression unit 109 via the reception / reproduction unit 108, and an interlaced image composed of Top Field and Bottom Field is reproduced. This interlaced image signal is returned to a frame-structured progressive image by the field / frame conversion unit 111 of the stereoscopic image post-processing unit 110, and then reconstructed into a left-eye image signal and a right-eye image signal by the pixel configuration conversion unit 112. Output from 113 and 114 respectively.
[0020]
FIG. 2 shows another example of a three-dimensional image compression / expansion apparatus for realizing the present invention, which displays a right-eye image having the same number of pixels and resolution as a left-eye image, and a normal planar image at twice the resolution. FIG. 3 is a block diagram illustrating a case where a high-resolution additional image having the same number of pixels and resolution is input.
[0021]
The stereoscopic image compression / expansion device receives a left-eye image signal 201, a right-eye image signal 202, and a high-resolution additional image signal 203. These are progressive images each having a frame configuration. Here, when the stereoscopic image display can display 352 pixels × 288 lines and the high-resolution flat display can display 352 pixels × 288 lines, the left-eye image signal 201, the right-eye image signal 202, and the high-resolution additional image signal 203 are, for example, Assuming that an image signal of 176 pixels × 288 lines is provided, a left-eye image signal 201 and a right-eye image signal 202 allow viewing of 176 pixels × 288 lines for each of the right and left eyes in a stereoscopic image display, and a left-eye image signal 201 and a high resolution It is assumed that 352 pixels × 288 lines can be viewed by the additional image signal 203.
[0022]
The left-eye image signal 201, the right-eye image signal 202, and the high-resolution additional image signal 203 are respectively input to the image configuration conversion unit 205 of the stereoscopic image preprocessing unit 204. The image configuration conversion unit 205 multiplexes the left-eye image signal 201, the right-eye image signal 202, and the high-resolution additional image signal 203 to form one image, and the frame / field conversion unit 206 virtually sets Top An interlaced image composed of Field and Bottom Field is configured, and the selection unit 207 selects a frame image and a field image. Operations of the image configuration conversion unit 205, the frame / field conversion unit 206, and the selection unit 207 in the stereoscopic image preprocessing unit 204 will be described later in detail again. The image signal output from the selection unit 207 is compressed in an image compression format such as MPEG-2 or MPEG-4 by an MPEG (Moving Picture Expert Group) compression unit 208, and is transmitted and recorded by a transmission / recording unit 209. , Recording to a storage, transmission to the outside, and the like are performed.
[0023]
The compressed stereoscopic image signal received from the outside and reproduced from the storage is MPEG-decompressed by the MPEG decompression unit 211 through the reception / reproduction unit 210, and an interlaced image composed of a progressive signal or a Top Field or Bottom Field is reproduced. You. This image signal passes through a branch circuit 213 of the three-dimensional image post-processing unit 212, is returned to a progressive image having a frame configuration by a field / frame conversion unit 214, and is then converted into a left-eye image signal and a right-eye image signal by a 215-pixel configuration conversion unit. , 216, 217, and 218 respectively.
[0024]
【Example】
The embodiment of the present invention and the operation of the present invention will be described with reference to FIGS.
[0025]
FIG. 3 shows a first embodiment.
[0026]
The three-dimensional image 301 includes a left-eye image (（) 302 and a right-eye image (x) 303, and the left-eye image 302 and the right-eye image 303 correspond to the left-eye image signal input 101 and the right-eye image signal input 102 in FIG. 1, respectively. These are arranged on the display as shown at 301. That is, a left-eye image and a right-eye image are alternately arranged for each vertical column. In the example of FIG. 3, 176 pixels × 288 lines of the left eye image and 176 pixels × 288 lines of the right eye image are shown, for a total of 352 pixels × 288 lines.
[0027]
The stereoscopic image 301 is converted by the image configuration conversion unit 104 like a first converted image 304. That is, 176 pixels × 288 lines of the left eye image are arranged on odd lines as they are translated, and 176 pixels × 288 lines of the right eye image are arranged on even lines as they are translated.
[0028]
The first converted image 304 is converted by the frame / field converting unit 105 into a second converted image 305 and a third converted image 306. That is, the second converted image 305 is converted into a 176 pixel × 288 line Top Field image, and the third converted image 306 is converted into a 176 pixel × 288 line Bottom Field image.
[0029]
Then, processing such as MPEG compression 307, transmission / recording 308, reception / playback 309, and MPEG decompression 310 is performed on the field images indicated by the second converted image 305 and the third converted image 306. These correspond to the MPEG compression unit 106, the transmission / recording unit 107, the reception / reproduction unit 108, and the MPEG decompression unit 109 in FIG.
[0030]
In the MPEG compression 307, the difference between the left-eye image and the right-eye image is detected in macroblock units, and when larger than a specified value, the field DCT is applied assuming that the correlation between the Top Field and Bottom Field is small. By applying the frame DCT assuming that the correlation of the Bottom Field is large, the compression is efficiently performed.
[0031]
The stereoscopic image preprocessing unit 103 arranges the left-eye image on odd lines and the right-eye image on even lines, or the right-eye image on odd lines and the left-eye image on even lines, and makes one image an interlaced image. One image processed by the stereoscopic image pre-processing unit 103 is converted into a block composed of data of odd lines and even lines and a block composed of data of only odd lines or only even lines by the MPEG compression unit 106. Among them, the most suitable block is selected, and further, when selecting the most suitable prediction method among the frame prediction from the previous and next frames or the field prediction from the previous and next fields, the block data predicted therefrom and the block data are selected. Are subjected to orthogonal transformation, quantization, and variable-length encoding, and as a result, a block that minimizes the code amount of the block data and a prediction method are selected for compression.
[0032]
The image signal output from the MPEG decompression 310 passes through the field / frame conversion unit 111 and the pixel configuration conversion unit 112, and the same stereoscopic image as the stereoscopic image 301 shown in the output stereoscopic image 311 is reproduced.
[0033]
FIG. 4 shows an example of a GOP (Group Of Pictures) configuration in the MPEG compression 307 of FIG.
[0034]
The GOP is composed of 15 pictures as indicated by a picture number 401, and is sequentially composed of BIBBBPBBPBBPBBP pictures as indicated by a picture type 402. Reference numeral 403 denotes a Top Field by T, and a Bottom Field by B. At this time, the compression shown in FIG. 3 can be realized by setting the top field as a left-eye image indicated by all ○ as shown in the picture contents 404 and the bottom field as a right-eye image indicated as all x in the picture contents 405. Reference numeral 406 indicates the relationship between the prediction source picture and the prediction destination picture by arrows.
[0035]
FIG. 5 shows a second embodiment.
[0036]
The stereoscopic image 501 is composed of a left-eye image (（) 502 and a right-eye image (x) 503, and the high-resolution plane image 504 is a high-resolution image for increasing the left-eye image (○) 502 and the left-eye image (○) 502 to a high resolution. It consists of an image (●) 505. The left-eye image (○) 502 of the three-dimensional image 501 and the left-eye image (○) 502 of the high-resolution planar image 504 indicate the same pixel. The left-eye image 502, the right-eye image 503, and the high-resolution additional image 505 correspond to the left-eye image signal input 201, the right-eye image signal input 202, and the high-resolution additional image signal input 203 in FIG. 2, respectively. These are arranged on the display as shown in a stereoscopic image 501 and a high-resolution planar image 504. That is, in the three-dimensional image 501, the left-eye image 502 and the right-eye image 503 are alternately arranged for each vertical column, and in the high-resolution planar image 504, the left-eye image 502 and the high-resolution additional image 505 are alternately arranged for each vertical column. . In the example of FIG. 5, in the stereoscopic image 501, the left eye image 176 pixels × 288 lines, the right eye image 176 pixels × 288 lines, a total of 352 pixels × 288 lines, and in the high resolution plane image 504, the left eye image 176 pixels × 288 lines, A high-resolution additional image of 176 pixels × 288 lines, that is, a total of 352 pixels × 288 lines is shown.
[0037]
The stereoscopic image 501 and the high-resolution plane image 504 are converted by the image configuration conversion unit 205 into a left-eye image 506 of the first picture, a right-eye image 507 of the first picture, a high-resolution additional image 508 of the first picture, and a left-eye image of the second picture. 509, the right-eye image 510 of the second picture, and the high-resolution additional image 511 of the second picture. That is, in the first image, the left-eye image 506 moves to the left on the screen, the right-eye image 507 moves to the center on the screen, and the high-resolution additional image 508 moves to the right on the screen. And a first picture 512 of 528 pixels × 288 lines is configured. In the second video, the same arrangement is performed on the left-eye image 509 of the second picture, the right-eye image 510 of the second picture, and the high-resolution additional image 511 of the second picture, and the second picture 513 of 528 pixels × 288 lines is formed. You.
[0038]
The first picture 512 of 528 pixels × 288 lines and the second picture 513 of 528 pixels × 288 lines do not pass through the frame / field conversion unit 206, that is, the output of the pixel configuration conversion unit 205 is directly selected by the selection unit 207. Output progressively.
[0039]
Processing such as MPEG compression 514, transmission / recording 515, reception / playback 516, and MPEG expansion 517 is performed on the frame image indicated by the first picture 512 of 528 pixels × 288 lines and the second picture 513 of 528 pixels × 288 lines. Done. These correspond to the MPEG compression unit 208, the transmission / recording unit 209, the reception / reproduction unit 210, and the MPEG decompression unit 211 in FIG.
[0040]
The image signal output from the MPEG decompression 517 is output from the branch circuit 213 through the pixel configuration conversion unit 215 directly without passing through the field / frame conversion unit 214, and is output as the stereoscopic image 501 and the high-resolution planar image 504. 518 and the output high-resolution planar image 519 are reproduced.
[0041]
FIG. 6 shows an example of a GOP (Group Of Pictures) configuration in the MPEG compression 514 of FIG.
[0042]
The GOP is composed of 15 pictures as shown by a picture number 601, and is composed of BBIBPBPBBPBBPBBP pictures in order as shown by a picture type 602. At this time, as shown in the picture contents 603, the compression shown in FIG. it can. 604 is a prediction source picture and a prediction picture.
The relationship between the destination pictures is indicated by arrows.
[0043]
FIG. 7 shows another example of the arrangement method shown in FIG. 5 for the first picture 512 of 528 pixels × 288 lines and the second picture 513 of 528 pixels × 288 lines.
[0044]
The left-eye image (O) 701 is on the screen, the right-eye image (X) 702 is on the center of the screen, and the high-resolution additional image (●) 703 is on the lower screen. When the three-dimensional image 501 and the high-resolution plane image 504 of FIG. 5 are arranged, the screen of FIG. 7 has 176 pixels × 864 lines.
[0045]
FIG. 8 shows another example of the arrangement method shown in the first picture 512 of 528 pixels × 288 lines in FIG. 5 as the second picture 513 of 528 pixels × 288 lines.
[0046]
The left-eye image (○) + high-resolution additional image (●) 801 are alternately arranged in a row from the left to the center of the screen, and the right-eye image (×) 802 is the right of the screen. When the three-dimensional image 501 and the high-resolution plane image 504 of FIG. 5 are arranged, the screen of FIG. 8 has 528 pixels × 288 lines.
[0047]
Although a plurality of arrangement examples have been described above, the left-eye image (○), the high-resolution additional image (●), and the right-eye image (×) do not have to be in this order.
[0048]
For example, in the first picture 512 of 528 pixels × 288 lines in FIG. 5, in the second picture 513 of 528 pixels × 288 lines, the order of the right-eye image (×), the left-eye image (、), and the high-resolution additional image (●) may be used. Or anything else.
[0049]
For example, in FIG. 7, the order of the right-eye image (×) 702, the left-eye image (○) 701, and the high-resolution additional image (●) 703 may be the order, or any other order.
[0050]
For example, in FIG. 8, the order of the right-eye image (×) 802 and the left-eye image + high-resolution additional image (画像, ●) 801 may be used.
[0051]
Further, a left-eye image + high-resolution additional image (（, ●) 801 and a right-eye image (×) 802 may be arranged vertically.
[0052]
FIG. 9 shows a third embodiment.
[0053]
The stereoscopic image 901 is composed of a left-eye image (（) 902 and a right-eye image (x) 903, and the high-resolution plane image 904 is a high-resolution addition for making the left-eye image (○) 902 and the left-eye image (○) 902 high-resolution. It is composed of an image (●) 905. The left-eye image 902, the right-eye image 903, and the high-resolution additional image 905 correspond to the left-eye image signal input 201, the right-eye image signal input 202, and the high-resolution additional image signal input 203 in FIG. 2, respectively. These are arranged on the display as shown in a stereoscopic image 901 and a high-resolution planar image 904. That is, in the three-dimensional image 901, the left-eye image and the right-eye image are alternately arranged for each vertical column, and in the high-resolution planar image 904, the left-eye image and the high-resolution additional image are alternately arranged for each vertical column. In the example of FIG. 9, in the stereoscopic image 901, the left eye image 176 pixels × 288 lines, the right eye image 176 pixels × 288 lines, a total of 352 pixels × 288 lines, and in the high resolution plane image 904, the left eye image 176 pixels × 288 lines, A high-resolution additional image of 176 pixels × 288 lines, that is, a total of 352 pixels × 288 lines is shown.
[0054]
The stereoscopic image 901 and the high-resolution plane image 904 are converted by the image configuration conversion unit 205 like an interlace conversion image 906. That is, 352 pixels × 288 lines of the high-resolution plane image 904 are arranged on odd-numbered lines as they are translated, and 176 pixels × 288 lines of the right-eye image 903 are arranged on even-numbered lines. However, since the number of pixels per line between the high-resolution planar image and the right-eye image does not match as it is, for the right-eye image, “0” is inserted for each pixel as shown in the interlace conversion image 906, and 352 pixels × 288 After making the line, it is translated and arranged on the even-numbered line as it is. This part will be described later in more detail. As a result, frame data of 352 pixels × 576 lines is arranged in the interlaced image 906.
[0055]
The interlace conversion image 906 is converted by the frame / field conversion unit 206 into a Top Field image 907 of 352 pixels × 288 lines and a Bottom Field image 908 of 352 pixels × 288 lines.
[0056]
Therefore, the output of the frame / field conversion unit 206 via the frame / field conversion unit 206, that is, the selection unit 207 outputs the Top Field image 907 of 352 pixels × 288 lines and the Bottom Field image 908 of 352 pixels × 288 lines. Selected and output in interlace.
[0057]
Then, processing such as MPEG compression 909, transmission / recording 910, reception / playback 911, and MPEG decompression 912 is performed on a field image represented by a Top Field image 907 of 352 pixels × 288 lines and a Bottom Field image 908 of 352 pixels × 288 lines. Done. These correspond to the MPEG compression unit 208, the transmission / recording unit 209, the reception / reproduction unit 210, and the MPEG decompression unit 211 in FIG.
[0058]
In the MPEG compression 909, the difference between the left-eye image and the right-eye image is detected in macroblock units, and when larger than a specified value, the field DCT is applied assuming that the correlation between the Top Field and Bottom Field is small. By applying the frame DCT assuming that the correlation of the Bottom Field is large, the compression is efficiently performed.
[0059]
The image signal output from the MPEG decompression 912 is re-converted from the branch circuit 213 to the frame configuration via the field / frame conversion unit 214, and then passed through the pixel configuration conversion unit 215 to obtain a three-dimensional image 901 and a high-resolution planar image. A stereoscopic image 913 and a high-resolution planar image 914 that are the same as those of the image 904 are reproduced.
[0060]
FIG. 10 shows how “0” is inserted for each pixel in the right eye image 903 over 8 pixels × 8 lines (DCT blocks). As shown in the underlined portion of 1001, “0” is inserted for each pixel, and as a result, the right-eye image is placed in the first, third, fifth, and seventh odd columns, and the right-eye image is placed in the second, fourth, sixth, and eighth even columns. “0” is inserted. These can be reversed. The value to be inserted does not have to be “0”.
[0061]
FIG. 11 shows how the same value as the previous pixel (left pixel) is inserted for each pixel in the right-eye image 903 over 8 pixels × 8 lines. As indicated by the underlined portion 1101, the same data as the previous pixel is inserted for each pixel. As a result, the right-eye image and the second, fourth, sixth, and eighth even numbers are inserted in the first, third, fifth, and seventh odd columns. The same data as the previous pixel is inserted in the column. These may be reversed, and may be the same data as the succeeding pixel (right pixel).
[0062]
FIG. 12 illustrates a state in which the average value of the previous pixel (left pixel) and the average value of the rear pixel (right pixel) are inserted for each pixel in the right-eye image 903 over 8 pixels × 8 lines. As indicated by the underlined portion 1201, the average value of the previous pixel (left pixel) and the average value of the rear pixel (right pixel) are inserted for each pixel. Further, as shown at 1202, “0” may be inserted for the rightmost end of the screen. As a result, the right-eye image is displayed in the first, third, fifth and seventh odd columns, and the average value of the previous pixel (left pixel) and the rear pixel (right pixel) is displayed in the second, fourth, sixth and eighth even columns. Is inserted at the right end.
[0063]
FIG. 13 shows a state in which the average value of the previous pixel (left pixel) and the average value of the rear pixel (right pixel) are inserted for each pixel in the right-eye image 903 again over 8 pixels × 8 lines. As shown by the underlined portion of 1301, the average value of the preceding pixel (left pixel) and the following pixel (right pixel) is inserted for each pixel. Further, as shown in 1302, the same data as the left pixel may be entered for the rightmost edge of the screen.
As a result, the right-eye image is displayed in the first, third, fifth and seventh odd columns, and the average value of the previous pixel (left pixel) and the rear pixel (right pixel) is displayed in the second, fourth, sixth and eighth even columns. The same data as that of the left pixel is inserted at the rightmost end of.
[0064]
FIG. 14 shows an example of a GOP (Group Of Pictures) configuration in the MPEG compression 909 of FIG.
[0065]
The GOP is composed of 15 pictures as indicated by a picture number 1401, and is sequentially composed of BIBBPBBPBBPBBP pictures as indicated by a picture type 1402. Reference numeral 1403 denotes a Top Field by T, and a Bottom Field by B. At this time, as shown in the picture contents 1404, the Top Field is a high-resolution plane image (○) obtained by combining the left-eye image (○) and the high-resolution additional image (●), and the Bottom Field is shown in the right-eye image as shown in the picture contents 1405. By performing data (× 0) in which (×) and the inserted “0” (0) are combined, the compression shown in FIG. 9 can be realized. An arrow 1406 indicates the relationship between the prediction source picture and the prediction destination picture.
[0066]
FIG. 15 shows a fourth embodiment.
[0067]
The stereoscopic image 1501 is composed of a left-eye image (（) 1502 and a right-eye image (x) 1503, and the high-resolution plane image 1504 is a high-resolution image for increasing the resolution of the left-eye image (○) 1502 and the left-eye image (○) 1502. It is composed of an image (●) 1505. The left-eye image 1502, the right-eye image 1503, and the high-resolution additional image 1505 correspond to the left-eye image signal input 201, the right-eye image signal input 202,
This corresponds to the resolution addition image signal input 203. These are arranged on the display as shown in a stereoscopic image 1501 and a high-resolution planar image 1504. That is, in the three-dimensional image 1501, a left-eye image and a right-eye image are alternately arranged for each vertical column, and in the high-resolution planar image 1504, a left-eye image and a high-resolution additional image are alternately arranged for each vertical column. In the example of FIG. 15, in the stereoscopic image 1501, the left eye image is 176 pixels × 288 lines, the right eye image is 176 pixels × 288 lines, and the total is 352 pixels × 288 lines. In the high resolution plane image 1504, the left eye image is 176 pixels × 288 lines. A high-resolution additional image of 176 pixels × 288 lines, that is, a total of 352 pixels × 288 lines is shown.
[0068]
The three-dimensional image 1501 and the high-resolution planar image 1504 are converted by the image configuration conversion unit 205 into a first frame Top Field image 1506, a first frame Bottom Field image 1507, a second frame Top Field image 1508, and a second frame Bottom Field image 1509. , A third frame Top Field image 1510, and a third frame Bottom Field image 1511. That is, the left-eye image 1502 of the first image and the right-eye image 1503 of the first image are the first frames 1506 and 1507, the high-resolution additional image 1505 of the first image, and the left-eye images 1508 and 1509 of the second image (not shown). , The right-eye image of the second video (not shown) and the high-resolution additional image of the second video are the third frames 1510 and 1511. That is, three frames are composed of two videos.
[0069]
1506 to 1511 are converted by the frame / field conversion unit 206 as 1512 to 1517. In other words, a Top Field image of 176 pixels × 288 lines as 1512, a Bottom Field image of 176 pixels × 288 lines as 1513, and a Top Field image of 176 pixels × 288 lines and a Bottom Field image in 1514, 1515, 1516, and 1517 below. , And a Top Field image and a Bottom Field image.
[0070]
Accordingly, the outputs 1512-1517 are output in an interlace manner via the frame / field conversion unit 206, that is, the output of the frame / field conversion unit 206 is selected by the selection unit 207.
[0071]
Then, processing such as MPEG compression 1518, transmission / recording 1519, reception / playback 1520, and MPEG decompression 1521 is performed on the field images denoted by 1512 to 1517. These correspond to the MPEG compression unit 208, the transmission / recording unit 209, the reception / reproduction unit 210, and the MPEG decompression unit 211 in FIG.
[0072]
In the MPEG compression 1518, the difference between the left-eye image and the right-eye image is detected in macroblock units, and when larger than a specified value, the field DCT is applied assuming that the correlation between the Top Field and Bottom Field is small. By applying the frame DCT assuming that the correlation of the Bottom Field is large, the compression is efficiently performed.
[0073]
The image signal output from the MPEG decompression unit 1521 is reconverted from the branch circuit 213 to the frame configuration via the field / frame conversion unit 214, and then passed through the pixel configuration conversion unit 215 to output a three-dimensional image 1501 and a high-resolution planar image. The same stereoscopic image 1522 and high-resolution planar image 1523 as 1504 are reproduced.
[0074]
FIG. 16 shows a GOP (Group Of Pictures) configuration example in the MPEG compression 1518 of FIG.
[0075]
A GOP is composed of 15 pictures as shown by a picture number 1601, and is composed of BBIBPBPBBPBBPBBP pictures in order as shown by a picture type 1602. In 1603, the Top Field is indicated by T and the Bottom Field is indicated by B. At this time, as shown in the picture contents 1604 and 1605, the combination of the Top Field and the Bottom Field is combined with the left-eye image (O) and the right-eye image (X), the high-resolution additional image (●), the left-eye image (O), and the right-eye image By using (x) and the high-resolution additional image (●), the compression shown in FIG. 15 can be realized.
[0076]
Now, here, one screen is shown by 1.5 frames including the three-dimensional image and the high-resolution plane image. Furthermore, since the fields of the 1.5 frame data indicating one screen and the 1.5 frame data indicating the next screen are reversed, the field configuration makes one cycle in three frames. For example, the sixth frame has the same field configuration as the third frame in FIG. 16 initially. As shown in FIG. 16, when a GOP is composed of 15 frames, the 15 frames are divisible by 3 frames, so that the frame configuration in each GOP is the same. That is, also in the next GOP shown in FIG. 16, in the third frame, the Top Field has the same configuration as the left-eye image (○) and the Bottom Field has the same configuration as the GOP shown in FIG.
[0077]
As described above, when the frame configuration is the same for each GOP, for example, in FIG.
-For I-pictures, Top Field is a left-eye image (o), Bottom Field is a right-eye image (x)
Therefore, the stereoscopic image (left-eye image, right-eye image) has a feature that the image quality is better than that of the high-resolution planar image (high-resolution additional image).
[0078]
An arrow 1606 indicates the relationship between the prediction source picture and the prediction destination picture.
[0079]
Here, in FIG. 16, the left-eye image (、), the right-eye image (×), and the high-resolution additional image (ず っ と) are arranged in this order, but the arrangement order can be changed. For example, a left-eye image (O) on the third frame odd line, a right-eye image (X) on the third frame even line, a high-resolution additional image (●) on the fourth frame odd line, and a high-resolution image on the fourth frame even line in 1601 The additional image (●), the left-eye image (○) on the fifth frame odd-numbered line, the right-eye image (×) on the fifth frame even-numbered line, and so on are repeated every three frames. Accordingly, the left-eye image is always an odd line (Top Field), and the right-eye image is always an even line (Bottom Field), so that the prediction efficiency, that is, the compression efficiency, can be increased.
[0080]
FIG. 17 shows another example of a GOP (Group Of Pictures) configuration in the MPEG compression 1518 of FIG.
[0081]
The GOP is composed of 12 pictures as shown by a picture number 1701, and is sequentially composed of BIBPBPBPBPBP pictures as shown by a picture type 1702. In 1703, the Top Field is indicated by T and the Bottom Field is indicated by B. At this time, as shown in the picture contents 1704 and 1705, the combination of the Top Field and the Bottom Field is combined with the left-eye image (O) and the right-eye image (X), the high-resolution additional image (●) and the left-eye image (O), and the right-eye image By using (x) and the high-resolution additional image (●), the compression shown in FIG. 15 can be realized.
[0082]
Now, when a GOP consists of 12 frames as shown in FIG. 17, since 12 frames are divisible by 3 frames, the frame configuration in each GOP is the same. That is, also in the next GOP shown in FIG. 17, in the second frame, the Top Field has the same configuration as the left-eye image (○) and the Bottom Field has the same configuration as the GOP shown in FIG.
[0083]
Reference numeral 1706 denotes an arrow indicating the relationship between the prediction source picture and the prediction destination picture.
[0084]
FIG. 18 shows another example of a GOP (Group Of Pictures) configuration in the MPEG compression 1518 of FIG.
[0085]
The GOP is composed of 12 pictures as shown by a picture number 1801, and is sequentially composed of BBBIBBBBPBBBP pictures as shown by a picture type 1802. Reference numeral 1803 denotes a Top Field, and B denotes a Bottom Field. At this time, as shown in the picture contents 1804 and 1805, the combination of the Top Field and the Bottom Field is combined with the left-eye image (O) and the right-eye image (X), the high-resolution additional image (●) and the left-eye image (O), and the right-eye image By using (x) and the high-resolution additional image (●), the compression shown in FIG. 15 can be realized.
[0086]
Now, when the GOP is composed of 12 frames as shown in FIG. 18, since 12 frames are divisible by 3 frames, the frame configuration in each GOP is the same. That is, even in the next GOP shown in FIG. 18, in the fourth frame, the Top Field has the same configuration as the left-eye image (○) and the Bottom Field has the same configuration as the GOP shown in FIG.
[0087]
An arrow 1806 indicates the relationship between the prediction source picture and the prediction destination picture.
[0088]
FIG. 19 shows another example of a GOP (Group Of Pictures) configuration in the MPEG compression 1518 of FIG.
[0089]
The GOP is composed of 16 pictures as shown by a picture number 1901, and is sequentially composed of BBBIBBBPBBBBBPBBBP pictures as shown by a picture type 1902. In 1903, T represents the Top Field and B represents the Bottom Field. At this time, as shown in the picture contents 1904 and 1905, the combination of the Top Field and the Bottom Field is combined with the left-eye image (O) and the right-eye image (X), the high-resolution additional image (●) and the left-eye image (O), and the right-eye image By using (x) and the high-resolution additional image (●), the compression shown in FIG. 15 can be realized.
[0090]
Now, when a GOP is composed of 16 frames as shown in FIG. 19, 16 frames cannot be divided by 3 frames, so that the frame configuration in each GOP is different. That is, in the next GOP shown in FIG. 19, in the fourth frame, the top field has a high-resolution additional image (●), and the bottom field has a left-eye image (○), which is different from the GOP shown in FIG.
[0091]
As described above, when the frame configuration is different for each GOP,
In the first GOP, the I-picture is a left-eye image (が) for Top Field and a right-eye image (×) for Bottom Field.
-In the second GOP, the I-picture is a top-field high-resolution additional image (●), and the Bottom Field is a left-eye image (O).
In the third GOP, the I-picture is a top-field right-eye image (x), and a bottom-field is a high-resolution additional image (●).
Since the images that make up the I picture change, the image quality is averaged for all three images.
[0092]
An arrow 1906 indicates the relationship between the prediction source picture and the prediction destination picture.
[0093]
7 shows another example of the embodiment of the present invention.
[0094]
On the compression side, a flag indicating a plane image or a stereoscopic image stream is inserted into the compressed transmission or recording stream.
[0095]
For example, in the MPEG format, PES # packet specifies 128-bit PES # private # data, and the flag may be placed here. After locating identification data such as “01011010” using the first 8 bits (B0 to B7) of the 128 bits as a stereoscopic video ID, the following 2 bits (B8, B9)
Plane 3d flag
00: Normal plane image
01: 3D image
10: pending
11: pending
And so on.
[0096]
On the decompression side, by detecting the planar stereoscopic flag, it is detected that the stream is a planar image or a stereoscopic image stream, and is decompressed as a planar image or a stereoscopic image and post-processed.
[0097]
This is illustrated by FIG.
[0098]
On the compression side, if the image is a planar image, the stereoscopic image preprocessing unit 103 is bypassed. At the same time, “00” is written in the first two bits (plane three-dimensional flag) of PES # private # data for the former and “01” for the latter, and output to the transmission / recording unit 107.
[0099]
On the decompression side, the data output from the reception / reproduction unit 108 is decompressed by the MPEG decompression unit 109, and the first two bits (plane three-dimensional flag) of PES # private # data are detected. At this time, if “00”, a three-dimensional image is output by bypassing the three-dimensional image post-processing unit 110, and if “01”, a three-dimensional image (left-eye image, right-eye image Image).
[0100]
The plane three-dimensional flag may be arranged in a data part other than PES # private # data that can be used freely by the user.
[0101]
Also, as another example,
On the compression side, in the compressed transmission or recording stream, which stream is a plane image only, a stereoscopic image only, a stereoscopic image and a planar image, a stereoscopic image and a high-resolution planar image, a stereoscopic image and a high-resolution additional image, A flag is inserted to indicate whether or not a combination is made.
[0102]
For example, it may be arranged in PES # private # data in PES # packet. Of the 128 bits, in the 4 bits (B8 to B11) after the stereoscopic video ID,
3D resolution flag
0000: Normal plane image
0001: 3D image only
0010: 3D image and normal 2D image
0011: 3D image and high resolution planar image
0100: 3D image and high-resolution additional image
0101-1111: pending
And so on.
[0103]
On the decompression side, by detecting the planar stereoscopic resolution flag, the stream is only a planar image, only a stereoscopic image, a stereoscopic image and a planar image, a stereoscopic image and a high-resolution planar image, and a stereoscopic image and a high-resolution planar additional image. Which image is composed is detected, decompressed and post-processed according to the detection result.
[0104]
FIG. 2 shows the case of the stereoscopic image and the high-resolution plane additional image.
[0105]
On the compression side, the stereoscopic image preprocessing unit 204 is bypassed for a planar image and the stereoscopic image preprocessing unit 204 is bypassed for a stereoscopic image including a high-resolution additional image (left-eye image, right-eye image, and high-resolution additional image). After passing through, the data is compressed by the MPEG compression unit 208, and specified data is written in the first 3 bits (plane three-dimensional resolution flag) of PES # private # data and output to the transmission / recording unit 209.
[0106]
On the decompression side, the data output from the reception / reproduction unit 210 is decompressed by the MPEG decompression unit 211, and the first three bits (plane three-dimensional resolution flag) of PES # private # data are detected. At this time, if the value is “000”, the plane image is output by bypassing the three-dimensional image post-processing unit 212. Outputs stereo images (left-eye image, right-eye image, and high-resolution additional image).
[0107]
The plane three-dimensional resolution flag may be arranged in a data part other than PES # private # data that can be used freely by the user.
[0108]
Further, as another example,
On the compression side, a flag indicating the multiplexing method of the stereoscopic image is inserted into the compressed transmission or recording stream.
[0109]
For example, it may be arranged in PES # private # data in PES # packet. Of the 128 bits, in the stereoscopic video ID and 4 bits (B12 to B15) after the plane stereoscopic resolution flag,
Stereo multiplexing flag
0000: pending
0001: A method in which a left-eye image is arranged on odd lines and a right-eye image is arranged on even lines, or a right-eye image is arranged on odd lines and a left-eye image is arranged on even lines, and one image is used as an interlaced image.
0010: one of the left-eye image, the right-eye image, and the high-resolution additional image is fixedly arranged on a part of one image, and the other one is solidified and arranged on another part of the one image, A method of arranging another one by fixing it to another part of one image
[0011] The left-eye image and the left-eye high-resolution additional image are multiplexed into one first image, the right-eye image and the dummy image are multiplexed into one second image, or the right-eye image and the right-eye high-resolution image The additional image is multiplexed into one first image, the left-eye image and the dummy image are multiplexed into one second image, and the first image and the second image are combined into one single image. Method to use third interlaced image
0100: One of the left-eye image, the right-eye image, and the high-resolution additional image is arranged on the odd-numbered line of the (3n + 1) th frame and the even-numbered line of the (3n + 2) th frame (n is an integer of 0 or more), and the other one is the even-numbered line of the (3n + 1) th frame. And the 3n + 3 frame are arranged on odd lines, the other is arranged on 3n + 2 frame odd lines and 3n + 3 frame even lines, and the image arranged on odd lines and the image arranged on even lines of the same frame are Interlaced image format
0101-1111: pending
And so on.
[0110]
On the decompression side, by detecting the three-dimensional compression method flag, the type of multiplexing of the stream is detected, and decompression and post-processing are performed according to the detection result.
[0111]
Whether the high-resolution plane image or the high-resolution additional image corresponds to the left-eye image or the right-eye image in the stereo multiplexing method flag described above, and what kind of data is arranged in an odd field and an even field when an interlaced image is used. Alternatively, data of a continuous data arrangement method and a dummy data configuration method may be included.
[0112]
For example, it may be arranged in PES # private # data in PES # packet. Among the 128 bits, the leading 4 bits (B12 to B15) of the 16 bits (B12 to B27) after the stereoscopic video ID and the plane stereoscopic resolution flag are used as flags indicating the above-described stereoscopic image multiplexing method.
[0113]
The next two bits (B16, B17) are used as a high resolution added image left / right flag.
[0114]
High resolution additional image left / right flag
00: High resolution additional image corresponds to left eye image
01: High resolution additional image corresponds to right eye image
10: High resolution additional image supports both left eye image and right eye image
11: pending
The next two bits (B18, B19) are used as an interlace structure flag.
[0115]
Interlace structure flag
00: Left eye image is Top Field, right eye image is Bottom Field
Top Field is where the high-resolution additional image exists, and Bottom Field is where the dummy image exists.
01: Right-eye image is Top Field, left-eye image is Bottom Field
The top field is where the dummy image exists, and the bottom field is where the high-resolution additional image exists.
10, 11: pending
The next two bits (B20 to B24) are used as a continuous data arrangement flag.
[0116]
Continuous data placement flag
00000: From left to right, left-eye image, right-eye image,
Or left-eye image, right-eye image, additional high-resolution image
00001: From left to right, right-eye image, left-eye image,
Or left eye image, high resolution additional image, right eye image
00010: right-eye image, left-eye image, high-resolution additional image from left to right
00011: From left to right, right-eye image, high-resolution additional image, left-eye image
00100: High-resolution additional image, left-eye image, right-eye image from left to right
00101: From left to right, high-resolution additional image, right-eye image, left-eye image
00110,00111 pending
01000: From top to bottom, left eye image, right eye image,
Or left-eye image, right-eye image, additional high-resolution image
01001: From top to bottom, right-eye image, left-eye image,
Or left eye image, high resolution additional image, right eye image
01010: From top to bottom, right-eye image, left-eye image, high-resolution additional image
01011: From right to left, right-eye image, high-resolution additional image, left-eye image
01100: High-resolution additional image, left-eye image, right-eye image from top to bottom
01101: High-resolution additional image, right-eye image, left-eye image from top to bottom
01110, 01111: pending
10000: From left to right, left-eye image, high-resolution additional image, right-eye image
10001: From left to right, right-eye image, left-eye image, and high-resolution additional image
10010: Right-eye image, high-resolution additional image, left-eye image from left to right
10011: From left to right, left-eye image, right-eye image, and high-resolution additional image
10100-11111: pending
The next three bits (B25 to B27) are used as a dummy data configuration flag.
[0117]
Dummy data configuration flag
000: Even column is dummy data, "0" is inserted
001: Odd column is dummy data, "0" inserted
010: Even columns are dummy data, interpolated from left pixel
011: Odd column is dummy data, interpolated from right pixel
100: Even columns are dummy data, average value interpolation from left and right pixels, rightmost column is "0" inserted
101: Odd-numbered columns are dummy data, average value interpolation from left and right pixels, and leftmost column is "0" inserted
110: Even column is dummy data and average value interpolation from left and right pixels, rightmost column is interpolation from left pixel
111: Odd column is dummy data, average value interpolation from left and right pixels, leftmost column is interpolation from right pixel
Although the arrangement of the flags has been described above, the bit positions and the number of bits need not be as illustrated.
[0118]
Also, the order of flag arrangement may not be as illustrated.
[0119]
【The invention's effect】
The present invention
A stereoscopic image preprocessing unit in which the left-eye image is arranged on odd lines and the right-eye image is arranged on even lines, or the right-eye image is arranged on odd lines and the left-eye image is arranged on even lines, and one image is an interlaced image;
One image processed by the stereoscopic image preprocessing unit is divided into a block composed of data of odd lines and even lines, and a block composed of data of only odd lines or only even lines.
Of these, when selecting the optimal block, further, when selecting the optimal prediction method from frame prediction from the previous and next frames, or field prediction from the previous and next fields,
A block and a prediction method in which the error between the block data predicted from the block data and the block data is orthogonally transformed, quantized, and variable-length coded, and as a result, the code amount of the block data is minimized.
An image compression unit that compresses by selecting
Alternatively, one of the left-eye image, the right-eye image, and the high-resolution additional image is one of an odd line of the (3n + 1) th frame (n is an integer of 0 or more), an even line of the (3n + 1) th frame, an odd line of the (3n + 2) th frame, an even line of the (3n + 2) th frame, The odd-numbered line of the 3n + 3 frame and the even-numbered line of the 3n + 3 frame,
The other one of the 3n + 1th frame odd line, the 3n + 1th frame even line, the 3n + 2 frame odd line, the 3n + 2 frame even line, the 3n + 3 frame odd line, and the 3n + 3 frame even line Place them in two,
Still another is not arranged among the 3n + 1th frame odd line, the 3n + 1th frame even line, the 3n + 2 frame odd line, the 3n + 2 frame even line, the 3n + 3 frame odd line, and the 3n + 3 frame even line. Place it on the other two,
A stereoscopic image preprocessing unit that uses an image arranged on an odd line and an image arranged on an even line of the same frame as an interlaced image, an image compression unit that compresses one image processed by the stereoscopic image preprocessing unit,
Thereby, the left-eye image, the right-eye image, and the high-resolution additional image can be efficiently compressed by using the respective correlations.
This has the effect.
[Brief description of the drawings]
FIG. 1 is a block diagram of a first stereoscopic image compression / decompression device that implements the present invention.
FIG. 2 is a block diagram showing a second stereoscopic image compression / decompression device for realizing the present invention.
FIG. 3 is a diagram showing a first embodiment.
FIG. 4 is a diagram illustrating an example of a GOP (Group Of Pictures) configuration in the MPEG compression of FIG. 3;
FIG. 5 is a diagram showing a second embodiment.
6 is a diagram illustrating an example of a GOP (Group Of Pictures) configuration in the MPEG compression of FIG. 5;
FIG. 7 is a diagram showing another example of the arrangement method of FIG. 5;
8 is a diagram showing another example of the arrangement method of FIG.
FIG. 9 is a diagram showing a third embodiment.
FIG. 10 is a diagram illustrating a state where “0” is inserted for each pixel.
FIG. 11 is a diagram illustrating a state where the same value as a previous pixel (left pixel) is inserted for each pixel.
FIG. 12 is a diagram illustrating a state where an average value of a front pixel (left pixel) and a rear pixel (right pixel) is inserted for each pixel.
FIG. 13 is a diagram illustrating a state where an average value of a front pixel (left pixel) and a rear pixel (right pixel) is inserted for each pixel.
14 is a diagram illustrating a configuration example of a GOP (Group Of Pictures) in the MPEG compression of FIG. 9;
FIG. 15 is a diagram showing a fourth embodiment.
16 is a diagram illustrating an example of a GOP (Group Of Pictures) configuration in the MPEG compression of FIG. 15;
17 is a diagram illustrating a configuration example of a GOP (Group Of Pictures) in the MPEG compression of FIG. 15;
18 is a diagram illustrating an example of a GOP (Group Of Pictures) configuration in the MPEG compression of FIG. 15;
19 is a diagram illustrating a configuration example of a GOP (Group Of Pictures) in the MPEG compression of FIG. 15;
FIG. 20 is a diagram showing a conventional example.
[Explanation of symbols]
101 Left eye image signal input
102 Right eye image signal input
103 stereo image preprocessing unit
104 Pixel Configuration Converter
105 Frame / field converter
106 MPEG compression unit
107 Transmission / Recording Unit
108 Receiving / playback unit
109 MPEG decompression unit
110 stereo image post-processing unit
111 Field / frame converter
112 Pixel Configuration Converter
113 Left eye image signal output
114 Right eye image signal output
201 Left eye image signal input
202 Right eye image signal input
203 High resolution additional image signal input
204 stereoscopic image preprocessing unit
205 Pixel Configuration Converter
206 frame / field converter
207 Selector
208 MPEG compression unit
209 Transmission / recording unit
210 Receiving / Playing Unit
211 MPEG decompression unit
212 stereoscopic image post-processing unit
213 Branch
214 Field / frame converter
215 Pixel Configuration Converter
216 Left eye image signal output
217 Right eye image signal output
218 High resolution additional image signal output
301 3D image
302 Left eye image (○)
303 Right eye image (×)
304 first converted image
305 Second converted image (Top Field)
306 Second converted image (Bottom Field)
307 MPEG compression
308 Transmission / Record
309 Reception and playback
310 MPEG decompression
311 Output 3D image
401 picture number
402 Picture Type
403 Top Field / Bottom Field display
404 Picture contents
405 picture contents
406 Relationship between prediction source picture and prediction destination picture
501 3D image
502 Left eye image (○)
503 Right eye image (×)
504 High resolution planar image
505 High resolution additional image (●)
506 Left eye image of first picture (O)
507 Right eye image of first picture (x)
508 High resolution additional image of the first picture (●)
509 Left eye image of second picture (O)
510 Right eye image of second picture (x)
511 High-resolution additional image of the second picture (●)
512 First pixel of 528 pixels x 288 lines
513 528 pixels x 288 lines of second picture
514 MPEG compression
515 Transmission / Record
516 Reception and playback
517 MPEG Decompression
518 Output stereo image
519 Output high resolution planar image
601 picture number
602 picture type
603 Arrangement of left eye image (○), high resolution additional image (●), right eye image (×)
604 Relationship between source picture and destination picture
701 Left eye image (○)
702 Right eye image (×)
703 High resolution additional image (●)
801 Left eye image (○) and high resolution additional image (●)
802 Right eye image (×)
901 3D image
902 Left eye image (○)
903 Right eye image (×)
904 High resolution planar image
905 High-resolution additional image (●)
906 Interlaced image
907 352 pixels x 288 lines Top Field image
908 Bottom Field image of 352 pixels x 288 lines
909 MPEG compression
910 Transmission / Record
911 Reception and playback
912 MPEG Decompression
The same stereoscopic image as 913 901
Same high resolution planar image as 914 904
1001 Insert "0"
1101 Same data insertion as previous pixel
1201 Average value insertion of previous pixel (left pixel) and rear pixel (right pixel)
1202 Insert "0"
1301 Insert average value of previous pixel (left pixel) and rear pixel (right pixel)
1302 Same data insertion as left pixel
1401 Picture number
1402 Picture type
1403 Top Field / Bottom Field display
1404 Picture contents (Top Field)
1405 Picture Field (Bottom Field)
1406 Relationship between source picture and destination picture
1501 3D image
1502 Left eye image (○)
1503 Right eye image (×)
1504 High resolution planar image
1505 High resolution additional image (●)
1506 First Field Top Field Image
1507 Bottom Field image of first frame
1508 2nd frame Top Field image
1509 Bottom Field image of second frame
1510 3rd frame Top Field image
1511 Bottom Field image of third frame
1512 176 pixels x 288 lines Top Field image
1513 Bottom Field image of 176 pixels x 288 lines
1514 Top Field image of 176 pixels × 288 lines
1515 Bottom Field image of 176 pixels x 288 lines
1516 176 pixels x 288 lines Top Field image
1517 Bottom Field image of 176 pixels x 288 lines
1518 MPEG compression
1519 Transmission / recording
1520 Reception and playback
1521 MPEG extension
1522 Same stereo image as 1501
Same high resolution planar image as 1523 1504
1601 Picture number
1602 Picture type
1603 Top Field / Bottom Field display
1604 Picture contents (Top Field)
1605 Picture contents (Bottom Field)
1606 Relationship between source picture and destination picture
1701 Picture number
1702 Picture type
1703 Top Field / Bottom Field display
1704 Picture contents (Top Field)
1705 Picture contents (Bottom Field)
1706 Relationship between prediction source picture and prediction destination picture
1801 Picture number
1802 Picture type
1803 Top Field / Bottom Field display
1804 Picture contents (Top Field)
1805 Picture contents (Bottom Field)
1806 Relationship between source picture and target picture
1901 Picture number
1902 Picture type
1903 Top Field / Bottom Field display
1904 Picture contents (Top Field)
1905 Picture Field (Bottom Field)
1906 Relationship between prediction source picture and prediction destination picture
2001 3D image
2002 Left eye image (○)
2003 Right eye image (×)
2004 converted image
2005 MPEG compression
2006 Transmission / Record
2007 reception and playback
2008 MPEG decompression
2009 Same stereoscopic image as 2001

Claims (20)

A stereoscopic image preprocessing unit that multiplexes the left-eye image and the right-eye image into one image;
A stereoscopic image compression device comprising: an image compression unit that compresses the one image processed by the stereoscopic image preprocessing unit; or
An image decompression unit that decompresses the one image compressed by the image compression unit;
A three-dimensional image decompression device, comprising: a three-dimensional image post-processing unit that separates a left-eye image and a right-eye image from the one image decompressed by the image decompression unit. A left-eye image is arranged on an odd-numbered line and a right-eye image is arranged on an even-numbered line, or a right-eye image is arranged on an odd-numbered line and the left-eye image is arranged on an even-numbered line.
From the block consisting of odd and even lines and the block consisting of only odd lines or even lines only, select the best block, and then predict the frame from the previous and next frames, or the previous and next fields When choosing the best prediction method from field predictions from,
As a result of orthogonal transformation, quantization, and variable length encoding of an error between the block data predicted from the block data and the block data, a block and a prediction method that minimize the code amount of the block data are selected. The stereoscopic image compression or decompression device according to claim 1. A stereoscopic image preprocessing unit that multiplexes a left-eye image, a right-eye image, and a high-resolution additional image for increasing the resolution of the left-eye image or the right-eye image into one image;
A stereoscopic image compression device comprising: an image compression unit that compresses the one image processed by the stereoscopic image preprocessing unit; or
An image decompression unit that decompresses the one image compressed by the image compression unit;
A stereoscopic image post-processing unit that separates a left-eye image, a right-eye image, and a high-resolution additional image for increasing the resolution of the left-eye image or the right-eye image from the one image expanded by the image expansion unit. A stereoscopic image decompression device characterized by the above-mentioned. One of the left-eye image, the right-eye image, and the high-resolution additional image is arranged in the left-third area of one image, the other is arranged in the center of the one image, and The three-dimensional image compression or decompression device according to claim 3, wherein one is arranged in a right one-third area of the one image. One of the left-eye image, the right-eye image, and the high-resolution additional image is arranged in the upper third of one image, the other is arranged in the center of the one image, and The three-dimensional image compression or decompression device according to claim 3, wherein one is arranged in a lower third area of the one image. When the high-resolution plane image is a left-eye image, a high-resolution plane image composed of a left-eye image and a high-resolution additional image for increasing the resolution of the left-eye image, or one of the right-eye images is set to the left 2 of one image.な い し to １ ／ of the area, and the other one is arranged to the right of 右 to ／ of the one image,
When the high-resolution planar image is a right-eye image, a high-resolution planar image composed of a right-eye image and a high-resolution additional image for increasing the resolution of the right-eye image, or one of the left-eye images to the left 2 of one image The three-dimensional image according to claim 3, wherein the three-dimensional image is arranged in an area of ３ to ３, and the other is arranged in an area of 右 to 右 of the one image. Compression or expansion device. When the high-resolution plane image is the left-eye image, the left-eye image and the high-resolution additional image for increasing the resolution of the left-eye image are multiplexed into one first image, and the right-eye image and the dummy image are multiplexed into one image. And the second image of
When the high-resolution planar image is a right-eye image, the right-eye image and the high-resolution additional image for making the right-eye image have high resolution are multiplexed into one first image, and the left-eye image and the dummy image are multiplexed into one image. And the second image of
Further, a three-dimensional image preprocessing unit that combines the first image and the second image into one third image,
A three-dimensional image compression device, comprising: an image compression unit that compresses the one third image processed by the three-dimensional image preprocessing unit; or
An image decompression unit for decompressing the one third image compressed by the image compression unit;
From the one third image decompressed by the image decompression unit,
When the high-resolution image is a left-eye image, one first image composed of a left-eye image and a high-resolution additional image is separated from one second image composed of a right-eye image and a dummy image. Separating a left-eye image and a high-resolution additional image from the first image, and a right-eye image from the one second image,
When the high-resolution image is a right-eye image, one first image consisting of a right-eye image and a high-resolution additional image is separated from one second image consisting of a left-eye image and a dummy image. A three-dimensional image decompression device, comprising: a three-dimensional image post-processing unit having a function of separating a right-eye image and a high-resolution additional image from the first image and a left-eye image from the one second image. The first image is arranged on odd lines and the second image is arranged on even lines, or the second image is arranged on odd lines and the first image is arranged on even lines, and the third image is formed as an interlaced image.
The stereoscopic image compression or decompression device according to claim 7, wherein the dummy image is "0" data. The first image is arranged on odd lines and the second image is arranged on even lines, or the second image is arranged on odd lines and the first image is arranged on even lines, and the third image is formed as an interlaced image.
8. The stereoscopic image compression or decompression device according to claim 7, wherein the dummy image has the same value as the left pixel data or the right pixel data. The first image is arranged on odd lines and the second image is arranged on even lines, or the second image is arranged on odd lines and the first image is arranged on even lines, and the third image is formed as an interlaced image.
The stereoscopic image compression or decompression device according to claim 7, wherein the dummy image is an average value of the left pixel data and the right pixel data. The stereoscopic image compression or decompression device according to claim 10, wherein the dummy image in the leftmost one column or the rightmost one column on the screen is "0" data. The leftmost column of the dummy image on the screen has the same value as the pixel data on the right side thereof, or the rightmost column of the dummy image on the screen has the same value as the pixel data on the left side thereof. The stereoscopic image compression or decompression device according to claim 10. One of the left-eye image, the right-eye image, and the high-resolution additional image is an odd line of the (3n + 1) th frame (n is an integer of 0 or more), an even line of the (3n + 1) th frame, an odd line of the (3n + 2) th frame, an even line of the (3n + 2) th frame, and an (3n + 3) th line. Frame odd lines, 3n + 3 frame even lines, and
The other one of the 3n + 1th frame odd line, the 3n + 1th frame even line, the 3n + 2 frame odd line, the 3n + 2 frame even line, the 3n + 3 frame odd line, and the 3n + 3 frame even line Place them in two,
Still another is not arranged among the 3n + 1th frame odd line, the 3n + 1th frame even line, the 3n + 2 frame odd line, the 3n + 2 frame even line, the 3n + 3 frame odd line, and the 3n + 3 frame even line. Place it on the other two,
The stereoscopic image compression or decompression device according to claim 3, wherein an image arranged on an odd line and an image arranged on an even line of the same frame are interlaced images. One of the left-eye image, the right-eye image, and the high-resolution additional image is arranged on the odd line of the (3n + 1) th frame and the even line of the (3n + 2) th frame (n is an integer of 0 or more), and the other is arranged on the even line of the (3n + 1) th frame. The 3n + 3 frame is arranged on the odd line, the other is arranged on the 3n + 2 frame odd line and the 3n + 3 frame even line, and the image arranged on the odd line and the image arranged on the even line of the same frame are interlaced. 14. The three-dimensional image compression or decompression device according to claim 13, wherein the device is an image. On the compression side, a flag indicating a stereoscopic image stream is inserted into the compressed transmission or recording stream, and on the decompression side, the flag is detected to detect that the image is a stereoscopic image stream. A three-dimensional image compression or decompression device characterized by performing decompression and post-processing. On the compression side, in the compressed transmission or recording stream, which stream is a planar image only, a stereoscopic image only, a stereoscopic image and a planar image, a stereoscopic image and a high-resolution planar image, a stereoscopic image and a high-resolution planar additional image, and By inserting a flag indicating whether the stream is composed of images, on the decompression side, by detecting the flag, the stream is only a planar image, only a stereoscopic image, a stereoscopic image and a planar image, a stereoscopic image and a high-resolution planar image, A three-dimensional image compression or decompression device, which detects which one of a three-dimensional image and a high-resolution additional image is composed, performs decompression and post-processing according to the detection result. On the compression side, a flag indicating all and part of the multiplexing method of the plane image, the stereoscopic image, the high-resolution plane image, and the high-resolution additional image in the stream is inserted into the compressed transmission or recording stream. Then, by detecting the flag, the plane image, the stereoscopic image, the high-resolution plane image, and all or a part of the multiplexing method of the high-resolution additional image in the stream are detected, and decompression and post-processing are performed according to the detection result. A stereoscopic image compression or decompression device, characterized in that: Depending on the multiplexing method, whether the high-resolution plane image or high-resolution additional image corresponds to one or both of the left-eye image and the right-eye image, and what kind of data should be placed in odd and even fields when making an interlaced image 18. The three-dimensional image compression or decompression device according to claim 1, wherein all or a part of a method of arranging continuous data and a method of forming dummy data are included. 19. The apparatus for compressing or decompressing a three-dimensional image according to claim 1, wherein the compression method is an MPEG (Moving Picture Expert Group) method. The compression method is an MPEG method, a difference between a left-eye image and a right-eye image is detected in macroblock units, and a field DCT is applied when the difference is larger than a specified value, and a frame DCT is applied when the difference is smaller than the specified value. The stereoscopic image compression or decompression device according to any one of Items 1 to 3, to 7 to 18.

Images