JP2010283512A - Image data output device and image display system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently output a plurality of parallax image data, and perform high flexible image display based on the plurality of parallax image data that are efficiently output. <P>SOLUTION: In an image compressor 21, one specified parallax image of n pieces of parallax images in one frame period of three-dimensional images is compressed in a frame as a base image (without use of interframe correlation processing). For each of the other (n-1) pieces of parallax images, the difference between each of the parallax images and the specified parallax image that is the base image is calculated to obtain differential images, and then interframe compression is performed for each parallax image. Because of three-dimensional image features, n pieces of parallax images have a high correlation each other, and the difference between each of the parallax images and the specified parallax image is calculated and then interframe compression is performed for the other (n-1) pieces of parallax images, so that because of three-dimensional image features, the data are efficiently compressed. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image data output device that outputs data of a plurality of parallax images, and an image display system that performs image display based on the data of a plurality of parallax images.

  Conventionally, glasses-type stereoscopic display that realizes stereoscopic viewing by wearing special glasses for stereoscopic viewing using light polarization and liquid crystal shutters to show separate images with parallax to both eyes of the observer The device is known. In addition, parallax barrier type and lenticular type stereoscopic display devices are known as one of the stereoscopic display methods that do not require wearing special glasses and can be stereoscopically viewed with the naked eye. In the parallax barrier system, a structure called a parallax barrier in which a plurality of striped slit portions that transmit light are provided on the front surface of a two-dimensional display panel is arranged to face each other. In the lenticular method, a lenticular lens is disposed opposite to the front surface of a two-dimensional display panel. The lenticular lens is a lens in which a large number of cylindrical lenses (cylindrical lenses) having refractive power only in the one-dimensional direction are arranged in parallel in the one-dimensional direction.

  In order to realize stereoscopic viewing, it is necessary to show different parallax images for the left eye and the right eye, so at least two parallax images are required. When three or more parallax images are used, multi-view viewing can be realized. As the number of parallax images increases, stereoscopic viewing according to changes in the viewpoint position of the observer can be realized. That is, motion parallax is obtained, and a more realistic stereoscopic effect is obtained.

JP 2000-11206 A Japanese Patent Laid-Open No. 7-23422 JP 2004-80727 A

  In the stereoscopic display device, a combined image for stereoscopic vision generated based on a plurality of parallax images corresponding to the number of parallaxes is displayed on the two-dimensional display panel. The composite image is generated by spatially dividing a plurality of parallax images and combining them in one screen with a predetermined display pattern. In the case of the parallax barrier system, stereoscopic vision is realized by optically separating the synthesized image into parallax images by the parallax barrier and presenting them to the viewer.

  Here, when supplying image data for stereoscopic display to the stereoscopic display device, a combined image for stereoscopic viewing that matches the display specifications of the stereoscopic display device is generated on the image data output device side different from the stereoscopic display device Then, it is conceivable to output it directly to the stereoscopic display device via the transmission means. Alternatively, it is conceivable to record once on a recording medium and supply a composite image from the recording medium to the stereoscopic display device. However, such a method of supplying a composite image has a problem that it can be displayed correctly only on a stereoscopic display device according to the specifications of the composite image (number of divisions and division widths of each parallax image). Lack of sex.

  In order to avoid this problem and deal with stereoscopic display devices of various specifications, there is a method of sending two pieces of image data as shown in FIG. 12 as stereoscopic display image data. The two image data includes a single base image 201 and a depth image (gray image) 202 indicating depth information of the base image 201. For example, when generating a multi-view composite image for multi-view viewing, on the stereoscopic display device side, a composite image is generated by image processing from the base image 201 and the depth image 202 according to the specifications of the stereoscopic display device. . However, since the depth information is given only to one base image 201 in the multi-view synthesized image by this method, there is a problem that an image (occlusion) surrounding the base image 201 cannot be seen. There are problems that occur.

  Therefore, in order to avoid the above problem, the stereoscopic display device is supplied with a plurality of parallax images before synthesis via a transmission unit or a recording medium, and the stereoscopic display device side of the stereoscopic display device A method of generating and displaying an appropriate composite image according to the specification is conceivable. However, in the case of this method, in order to support multi-view vision, a large amount of parallax images are supplied, so that the amount of image data increases and the transmission data rate increases. Or, there is a problem that the data size to be recorded on the recording medium becomes large. Therefore, when this method is used, it is necessary to develop a method for efficiently transmitting data by compressing the data.

  Patent Document 1 discloses a method of compressing three-dimensional moving image data by converting three-dimensional moving image data into reference frame data and a compressed data group formed by compressing a plurality of frames following the reference frame data. The invention is disclosed. In this 3D moving image data compression method, 3D moving image data composed of a plurality of frames composed of 3D shape data formed by connecting a plurality of vertices on 3D coordinates is handled. The frame data of the three-dimensional moving image data is three-dimensional coordinate data (so-called polygon coordinate data), and the vertex data of the three-dimensional shape data is used for compression. That is, a method using the characteristics of polygon coordinate data is used. However, this compression method cannot deal with the case where the 3D moving image data is normal image data (image pixel data) that does not use 3D coordinate data, and is not versatile.

  Patent Document 2 discloses an invention of a method for compressing moving image data using motion compensation between frames. However, Patent Document 2 does not disclose compression of image data for stereoscopic display. That is, it is not disclosed at all what kind of compression is performed on image data of a stereoscopic image in which a plurality of parallax images are included in time sequence within one frame period. In particular, it does not disclose what kind of compression is performed on a plurality of parallax images within one frame period.

  In Patent Document 3, a left-eye image and a right-eye image for stereoscopic display are converted and compressed into a two-screen linked video that is linked in units of frames on a computer. An invention of a video system configured to be output separately on a screen is disclosed. Patent Document 3 discloses a method of simply connecting and compressing two images on a screen, but such a method cannot sufficiently cope with an increase in parallax images for multi-view viewing.

  The present invention has been made in view of such a problem, and an object of the present invention is to provide an image data output device capable of efficiently outputting data of a plurality of parallax images, and a plurality of efficiently output plural data. An object of the present invention is to provide an image display system that can perform image display with a high degree of freedom of display based on parallax image data.

  An image data output apparatus according to the present invention includes an image compression unit that compresses and outputs image data of a stereoscopic image in which n (n is an integer of 2 or more) parallax images are sequentially included in one frame period, The image compression unit compresses one specific parallax image as a base image as it is as a base image out of n parallax images within one frame period of the stereoscopic image, and the other (n−1) parallax images. With regard to, the difference between each parallax image and a specific parallax image as a base image is taken to be a difference image, and then compressed in each frame for each parallax image.

  An image display system according to the present invention includes the image data output device according to the present invention and a display device. The display device includes a display unit for displaying an image and a signal processing unit including an image decompression unit. The signal processing unit decompresses the compressed image data in the image decompression unit, and converts the decompressed image data into the decompressed image data. Based on this, a display image to be displayed on the display unit is generated.

  In the image data output device according to the present invention, one specific parallax image out of n parallax images within one frame period of a stereoscopic image is compressed as a base image as it is (without using inter-frame correlation processing). The The other (n−1) parallax images are subjected to intra-frame compression for each parallax image after taking the difference between each parallax image and a specific parallax image as a base image to obtain a difference image. Since the n parallax images are images that are highly correlated with each other due to the characteristics of the stereoscopic image, the (n−1) parallax images are subjected to intra-frame compression after taking a difference from the specific parallax image. Due to the characteristics of the stereoscopic image, the data is efficiently compressed.

  In the image display system according to the present invention, the image data output from the image data output device is decompressed by the image decompression unit of the display device. The signal processing unit generates a display image to be displayed on the display unit based on the decompressed image data. In this image display system, since all the n parallax images are output from the image data output device to the display device and the display image is generated on the display device side, an appropriate display image in accordance with the specifications of the display device can be obtained. .

  According to the image data output device of the present invention, one specific parallax image is intra-frame compressed as a base image, and for each of the other (n−1) parallax images, the difference between each parallax image and the base image Since the intra-frame compression is performed for each parallax image after taking the image, the image data can be efficiently compressed due to the characteristics of the stereoscopic image. Thereby, data of a plurality of parallax images can be output efficiently.

  According to the image display system of the present invention, the image data output device efficiently compresses the image data of the stereoscopic image to output all n parallax images, and decompresses the image data on the display device side to display the display image. Since the image is generated, an appropriate display image that matches the specifications of the display device can be obtained. In addition, since the n parallax images are independently compressed, it is possible to easily decompress only the necessary parallax images and use them as display images. As a result, it is possible to display an image with a high degree of freedom of display based on data of a plurality of parallax images that are efficiently output.

1 is a block diagram illustrating a configuration example of an image display system according to a first embodiment of the present invention. It is a block diagram which shows the structural example of the image compression part in the image display system which concerns on 1st Embodiment. It is explanatory drawing which shows an example of the data structure output from an image compression part. It is explanatory drawing which shows typically the concept of the image compression in an image compression part. It is explanatory drawing which shows the structural example of the stream data output from an image transmission part. It is a block diagram which shows the structural example of the image decompression | decompression part in the image display system which concerns on 1st Embodiment. FIG. 10 is an explanatory diagram illustrating a process of generating a multi-viewpoint composite image from a plurality of parallax images in the stereoscopic display device according to the first embodiment. It is a block diagram which shows the example which comprised the three-dimensional display part by the parallax barrier system. It is a block diagram which shows the example which comprised the three-dimensional display part by the lenticular system. It is a block diagram which shows the structural example of the image compression part in the image display system which concerns on 2nd Embodiment. It is a block diagram which shows the structural example of the image data output device in the image display system which concerns on 4th Embodiment. It is explanatory drawing which shows the structural example of the data for the conventional multi-view vision.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<First Embodiment>
[Overall configuration of image display system]
FIG. 1 shows a configuration example of an image display system according to the first embodiment of the present invention. This image display system includes an image data output device 4, a stereoscopic display device 5, a transmission line 23, and a data storage 24. The image data output device 4 compresses and outputs stereoscopic image data in which n parallax images are included in time sequence within one frame period. The transmission line 23 transmits the image data output from the image data output device 4 to the stereoscopic display device 5. The data storage 24 is a recording medium capable of recording and reproducing the image data output from the image data output device 4, and is, for example, an HDD (Hard Disk Drive) or an optical disk. The stereoscopic display device 5 decompresses the image data output from the image data output device 4 via the transmission line 23 or the data storage 24, and generates a display image for stereoscopic display based on the decompressed image data. A three-dimensional display is performed.

  Note that the image display system of the present invention can also be applied to a case where the image data of a stereoscopic image is composed of two binocular parallax images (n = 2). A case where the image data is visible (n = 3 or more) will be described. In particular, as an example, when n = 4, that is, when the image data of a stereoscopic image is multi-view image data in which the first to fourth four parallax images are included in time sequence within one frame period. Will be described.

[Configuration of image data output device]
The image data output device 4 includes a photographing unit 6, an image compression unit 21, and an image transmission unit 22. The photographing unit 6 includes n cameras, which are first to fourth cameras 61, 62, 63, 64 in this embodiment. The imaging unit 6 captures a subject from the first to fourth positions that are different from each other by the first to fourth cameras 61, 62, 63, and 64, thereby having parallax as image data of a stereoscopic image. First to fourth parallax images for multi-view viewing are generated. The imaging unit 6 outputs the generated first to fourth parallax images to the image compression unit 21. The image compression unit 21 arranges the input first to fourth parallax images in order, performs a predetermined image compression process, arranges the compressed images in order, forms a stream, and outputs the stream to the image transmission unit 22. Yes. The image transmission unit 22 performs processing necessary for transmission (processing according to a predetermined communication protocol, etc.) on a video stream composed of compressed stereoscopic image data, and then outputs it to the transmission line 23 or the data storage 24. The process to do is to be performed.

  FIG. 2 shows a configuration example of the image compression unit 21. FIG. 3 shows an example of the data structure output from the image compression unit 21. FIG. 4 schematically shows the concept of image compression processing in the image compression unit 21. FIG. 5 schematically shows the structure of a video stream output from the image transmission unit 22.

  As shown in FIG. 2, the image compression unit 21 includes a frame memory 41, a difference unit 42, a stream encoder 43, and a header creation unit 44. The image compressing unit 21 performs intra-frame compression as it is using one specific parallax image as a base image among n parallax images within one frame period of a stereoscopic image. For the other (n−1) parallax images, the difference between each parallax image and a specific parallax image as a base image is taken to obtain a differential image, and then the intra-frame compression is performed for each parallax image. (See FIG. 4). In the present embodiment, the first parallax image in time within one frame period of the stereoscopic image is processed as the base image. Specifically, the first image data sent from the photographing unit 6, that is, the first parallax image sent from the first camera 61 is processed as a base image. By performing such compression processing for each frame of a stereoscopic image and sequentially outputting from the image compression unit 21 to the image transmission unit 22, a video stream as shown in FIG. 5 is obtained. That is, a video stream in which data including a base image and subsequent compressed difference images is sequentially obtained is obtained.

  The frame memory 41 stores image data for one frame, and stores a base image for each frame of a stereoscopic image. The differentiator 42 outputs the difference between the base image stored in the frame memory 41 and the other (n−1) parallax images to the stream encoder 43. The stream encoder 43 individually compresses image data with a predetermined codec for each of one base image and (n−1) difference images. A plurality of parallax images within one frame period of a stereoscopic image have a difference due to parallax, but most of the images are similar and have a strong correlation between images. For this reason, it can compress efficiently, after compressing after taking a difference with a base image.

  The stream encoder 43 performs compression processing by intra-frame compression using a codec having no inter-frame correlation. As a video codec having no correlation between frames, for example, a JPEG2000 codec can be used. In this case, the stream encoder 43 is configured by a JPEG2000 encoder. In the JPEG2000 codec, color space coordinate conversion is performed on an input image signal as necessary, wavelet conversion (DWT) is performed for each color of the signal, and the coefficients are encoded independently. DWT includes an integer type filter that has a relatively simple circuit configuration capable of reversible conversion and a real number type filter that cannot perform reversible conversion but has better image quality than the integer type. In this embodiment, either irreversible compression (lossy) mode or lossless compression (lossless) mode may be used. In the JPEG 2000 codec, the DWT transform coefficient is decomposed into bit planes and divided into encoded blocks of a certain size (for example, 64 × 64) in each subband, and then binary arithmetic coding is performed. The transform coefficient value in each coding block is divided into sine bits (positive: 0, negative: 1) and absolute value information, and the absolute value is expressed in a natural binary system and encoded sequentially from the upper bit plane. When the whole of each subband of the image becomes close to black, all zero bit planes are continuous from the upper bit plane, so that encoding is efficient. Specifically, the number of all zero bit planes from the upper plane excluding the sign bit plane is stored in the encoding header, and the encoding of all zero bit planes is skipped. Therefore, the amount of recording / reproducing data required per unit time for an image close to black is reduced. In this embodiment, the characteristic that the amount of data decreases is used.

  The header creator 44 adds header information to each of the base image and the difference image compressed by the stream encoder 43. FIG. 3 shows an example of the data structure after adding the header information. As shown in FIG. 3, the data output from the image compression unit 21 has a data structure including header information (Header) followed by actual data part (Camera DATA). As the header information, at least information capable of discriminating between the base image and the difference image is added. In the example of FIG. 3, the header information includes information of “Header = 0xfffffffn” for recognizing the camera image number n, information of the compressed data byte size “Length = xx” [Byte] following the header information, and It consists of “Header = 0xfffffffn” indicates which of the first to fourth cameras 61, 62, 63, 64 of the photographing unit 6 is the output image data. For example, “Header = 0xfffffff1” indicates that the parallax image data is output from the first camera 61. Note that, as described above, in the present embodiment, the first parallax image sent from the first camera 61 is processed as the base image, so that the base image is determined from the information “Header = 0xfffffffn”. It can be determined whether or not.

[Configuration of stereoscopic display device]
The stereoscopic display device 5 includes an image receiving unit 30, a signal processing unit 3, and a stereoscopic display unit 100. The signal processing unit 3 includes an image decompression unit 31, a composite image generation unit 32, and a drive processing unit 33.
In the present embodiment, the stereoscopic display device 4 corresponds to a specific example of “display device” in the present invention. The three-dimensional display unit 5 corresponds to a specific example of “display unit” in the present invention.

  The stereoscopic display unit 100 can perform stereoscopic display based on a stereoscopic display image including a plurality of parallax images, and is, for example, a parallax barrier type or lenticular type display device. FIG. 8 shows an example in which the stereoscopic display unit 100 is configured by a parallax barrier method. The stereoscopic display unit 100 includes a two-dimensional display panel 2 and a parallax barrier 1 arranged so as to face the display surface side of the two-dimensional display panel 2.

  The two-dimensional display panel 2 is a two-dimensional image display in which a plurality of pixels are two-dimensionally arranged. For example, the two-dimensional display panel 2 includes an organic EL (Electro-Luminescence) display or a liquid crystal display. The two-dimensional display panel 2 displays a multi-viewpoint composite image output from the signal processing unit 3.

  The parallax barrier 1 is for optically separating the parallax images included in the multi-viewpoint composite image displayed on the two-dimensional display panel 2 so as to enable multiview viewing. In the parallax barrier 1, shielding portions 11 that shield display image light from the two-dimensional display panel 2 and stripe-shaped slit portions 12 that transmit display image light from the two-dimensional display panel 2 are alternately arranged in the horizontal direction. It is provided. The width of the slit portion 12 is such that when the observer views the stereoscopic display device from a predetermined position and a predetermined direction, different parallax image lights are separately incident on the left and right eyes 10L and 10R of the observer. It is set to an appropriate size.

  FIG. 9 shows a stereoscopic display unit 100A configured by a lenticular method. The stereoscopic display unit 100 </ b> A has a lenticular lens 1 </ b> A arranged on the display surface side of the two-dimensional display panel 2 instead of the parallax barrier 1 of FIG. 9. The lenticular lens 1A is formed by, for example, arranging a large number of cylindrical lenses (cylindrical lenses) 13 extending in the vertical direction and having refractive power only in the horizontal direction in the horizontal direction. The lens pitch of the cylindrical lens 13 is appropriately set according to the number of parallaxes to be displayed in multi-view. For example, when performing multi-view viewing with four parallaxes, four divided images with different parallaxes are set to correspond to one cylindrical lens 13. By setting in this way, the display image light from the two-dimensional display panel 2 is deflected and emitted in different directions for each parallax image.

  The image receiving unit 30 receives the video stream output from the image transmitting unit 22 of the image data output device 4 via the transmission line 23 or the data storage 24 and outputs it to the image decompressing unit 31 of the signal processing unit 3. is there.

  The signal processing unit 3 extracts a parallax image corresponding to the number of parallaxes required for the stereoscopic display on the stereoscopic display unit 100 from the video stream, and combines the images, so that the stereoscopic display on the stereoscopic display unit 100 is performed. A display image for stereoscopic viewing that can be generated is generated.

  As shown in FIG. 6, the image decompression unit 31 includes a header analyzer 51, a stream decoder 52, a frame memory 53, and an adder 54. The image decompressing unit 31 decompresses the compressed stereoscopic image data constituting the video stream. The header analyzer 51 analyzes the header information of the transmitted image data to analyze whether or not it is a base image. The stream decoder 52 decompresses image data using a codec corresponding to the codec of the stream encoder 43 of the image compression unit 21. For example, if the stream encoder 43 is composed of a JPEG2000 encoder, the stream decoder 52 is composed of a JPEG2000 decoder.

  The image decompression unit 31 decompresses a base image compressed in a frame out of n parallax images in one frame period of a stereoscopic image as it is by the stream decoder 52 to restore a specific parallax image, thereby generating a composite image generation unit 32 is output. In addition, the restored base image is stored in the frame memory 53. The image decompression unit 31 also decompresses the difference image compressed in the frame by the stream decoder 52 for each of the other (n−1) parallax images. After that, the restored base image stored in the frame memory 53 is added to the decompressed difference image by the adder 54 to restore (n−1) parallax images, thereby generating the composite image generation unit 32. To output.

  The composite image generation unit 32 generates a multi-viewpoint composite image based on each parallax image for multiview viewing output from the image decompression unit 31. The drive processing unit 33 generates a drive signal for the two-dimensional display panel 2 of the stereoscopic display unit 100 based on the multi-view combined image output from the combined image generation unit 32 and displays the multi-view combined image on the two-dimensional display panel 2. Is displayed.

  FIG. 7 schematically shows a process until the composite image generation unit 32 generates a multi-viewpoint composite image. The composite image generation unit 32 performs a process of cutting out a plurality of stripe-shaped divided images extending in the vertical direction from each parallax image for multi-view viewing, and alternately outputs the divided images in the horizontal direction for each parallax image. To generate a multi-view composite image. FIG. 5 shows an example using the first to fourth four parallax images. First to fourth parallax images as shown in the upper part of FIG. 7 are input to the composite image generation unit 32. As shown in the middle part of FIG. 7, the composite image generation unit 32 cuts out striped divided images from the first to fourth parallax images. Next, the divided images are alternately arranged in the horizontal direction for each parallax image to generate a multi-viewpoint composite image as shown in the lower part of FIG. As a result, in the multi-view synthesized image, the stripe-shaped divided images cut out from the first to fourth parallax images are alternately arranged in the horizontal direction in the order of the first, second, third, and fourth parallax images. This is a composite image arranged in a circular pattern.

  By displaying the multi-viewpoint composite image shown in FIG. 7 on the two-dimensional display panel 2 and observing through the parallax barrier 1 or the lenticular lens 1A, multi-view stereoscopic display can be performed. For example, by observing through the parallax barrier 1, when viewed from a specific viewpoint position, only a specific parallax image in the left eye 10L and the right eye 10R can be seen, so that a stereoscopic image is perceived. Is done. For example, when viewed from a specific first viewpoint position, only the light from the third parallax image is recognized by the left eye 10L, and only the light from the second parallax image is recognized by the right eye 10R. By becoming a situation, a stereoscopic image based on the third parallax image and the second parallax image is perceived. When viewed from another second viewpoint position, for example, the left eye 10L recognizes only light from the first parallax image, and the right eye 10R recognizes only light from the fourth parallax image. By becoming a situation, a stereoscopic image based on the first parallax image and the fourth parallax image is perceived. Thus, motion parallax can be obtained by viewing different parallax images when viewed from different viewpoint positions.

[Operation of image display system]
The overall flow of image processing in this image display system is described below.
(A1) In the photographing unit 6, the first to fourth cameras 61, 62, 63, 64 are synchronized with each other from the first to fourth positions different from each other, so that the subject is simultaneously photographed. The first to fourth parallax images for multi-view viewing are created for one frame. The first to fourth parallax images are set as image data for one frame of a stereoscopic image.
(A2) The process (A1) is repeated to obtain a continuous frame of a stereoscopic image.
(A3) The obtained continuous frames are compressed by the image compression unit 21 according to a predetermined codec. The image compression unit 21 sequentially arranges the compressed images into a stream and outputs the stream to the image transmission unit 22.
(A4) Video stream data composed of compressed stereoscopic image data is processed by the image transmission unit 22 and then output to the transmission line 23 or the data storage 24.
(A5) The video stream data output from the image transmission unit 22 is received by the image reception unit 3 of the stereoscopic display device 5 via the transmission line 23 or the data storage 24. The image receiving unit 3 outputs the received video stream data to the image decompressing unit 31.
(A6) The compressed video stream data is decompressed by the image decompression unit 31 according to a predetermined codec to obtain restored video stream data.
(A7) One frame of data (first to fourth parallax images) as a stereoscopic image is extracted from the restored video stream data, and a multi-view multi-view combined image (see FIG. 7).
(A8) Based on the generated multi-view combined image, the drive processing unit 33 generates a drive signal for the 2D display panel 2 of the stereoscopic display unit 100 and displays the multi-view combined image on the 2D display panel 2. .

[Operation of Image Compression Unit 21]
The flow of image processing in the image compression unit 21 (FIG. 6) will be described below.
(B1) Data of the camera image 1 (first parallax image) captured by the first camera 61 of the imaging unit 6 is sent to the frame memory 41 and stored as a base image.
(B2) At the same time, the data of the camera image 1 is sent to the stream encoder 43. The data of the camera image 1 sent to the stream encoder 43 is compressed in the frame according to a predetermined video codec (JPEG2000 or the like).
(B3) The compressed data of the camera image 1 is sent to the image transmission unit 22 at the next stage.
(B4) Next, the frame of the data of the camera image 2 (second parallax image) captured by the second camera 62 of the imaging unit 6 and the base image (data of the camera image 1) in the frame memory 41 is displayed. The difference for each pixel is calculated, and the difference image data is output to the stream encoder 43.
(B5) The difference image data related to the camera image 2 sent to the stream encoder 43 is compressed in the frame in accordance with a predetermined video codec (JPEG2000 or the like).
(B6) The difference image data relating to the compressed camera image 2 is sent to the image transmission unit 22 at the next stage.
(B7) For the data of the camera image 3 (third parallax image) captured by the third camera 63 and the data of the camera image 4 (fourth parallax image) captured by the fourth camera 64 However, the same processing as the processing (B4) to the processing (B6) is performed.
(B8) In the process (B4) and the process (B6), when sending the compressed camera image to the image transmission unit 22 at the next stage, the header generator 44 precedes the data of each compressed camera image. Send header information. As header information, information of “Header = 0xfffffffn” for recognizing camera image number n and information of compressed data byte size “Length = xx” [Byte] following the header information are added (see FIG. 3).
(B9) Thereafter, the processing (B1) to the processing (B8) are repeated, and each frame data of the stereoscopic image is processed one after another to output video stream data as shown in FIG.

[Operation of Image Decompression Unit 31]
The flow of image processing in the image decompression unit 31 (FIG. 10) will be described below.
(C1) The header analyzer 51 analyzes the header information output from the image receiving unit 30 and input to the image decompressing unit 31. From the information of “Header = 0xfffffffn” and the information of “Length = xx” constituting the header information, information on the camera image number n and the byte size of the actual data of the compressed camera image corresponding to the image number is obtained.
(C2) Confirm that camera image number n is n = 1. Thereby, it is confirmed that the portion of the actual data following the header information is a base image from which no image difference is taken.
(C3) The actual data portion following the header information, that is, the data of “Length = xx” [Byte] obtained in the processing (C1) is decoded by the stream decoder 52 and the compressed camera image 1 Get decompressed data.
(C4) The decompressed camera image 1 data is output to the next-stage composite image generation unit 32.
(C5) At the same time, the decompressed camera image 1 data is also output to the frame memory 53.
(C6) The header analyzer 51 analyzes the next header information input to the image decompression unit 31. From the information of “Header = 0xfffffffn” and the information of “Length = xx” constituting the header information, information on the camera image number n and the byte size of the actual data of the compressed camera image corresponding to the image number is obtained.
(C7) Confirm that camera image number n is n = 2. Thereby, it is confirmed that the part of the actual data following the header information is a difference image taking an image difference.
(C8) The portion of actual data following the header information, that is, the data for “Length = xx” [Byte] obtained in the processing (C6) is decoded by the stream decoder 52, and the compressed camera image 2 is related. The decompressed data of the difference image data is obtained.
(C9) The data of the camera image 2 is restored by adding the pixel values of the difference image data related to the decompressed camera image 2 and the base image stored in the frame memory 53, and Are output to the composite image generation unit 32.
(C10) The same processing as the processing (C6) to the processing (C9) is performed for the data whose header information is the camera image number 3 and the camera image number 4.
(C11) Thereafter, the processes (C1) to (C10) are repeated, and each frame data of the stereoscopic image is processed one after another.

[Effects of First Embodiment]
According to the image data output device 4 according to the present embodiment, one specific parallax image is subjected to intra-frame compression as a base image, and for each of the other (n−1) parallax images, each parallax image and the base are compressed. After taking the difference from the image, each parallax image is compressed within the frame and output, so that the image data can be efficiently compressed due to the characteristics of the stereoscopic image. Thereby, data of a plurality of parallax images can be output efficiently. As a result, the data rate when transmitting to the transmission line 23 can be suppressed even though all the n parallax images necessary for displaying the stereoscopic image are sent from the image data output device 4. Further, the data size recorded in the data storage 24 can be suppressed.

  Further, according to the image display system according to the present embodiment, the image data output device 4 efficiently compresses the image data of the stereoscopic image and outputs all n parallax images, and the stereoscopic display device 5 side outputs the image. Since the display image is generated by decompressing the data, an appropriate display image that matches the specification of the stereoscopic display device 5 can be obtained. Since each of the n parallax images has been independently compressed, only the necessary parallax image can be decompressed and used as a display image. As a result, it is possible to display an image with a high degree of freedom of display based on data of a plurality of parallax images that are efficiently output.

  Further, since all the n parallax images necessary for displaying the stereoscopic image are sent from the image data output device 4, when the stereoscopic display is performed on the stereoscopic display device 5 side, as shown in FIG. As compared with a method of sending simple image data, it is possible to suppress the occurrence of a problem that a surrounding image (occlusion) cannot be seen.

<Second Embodiment>
Next, a stereoscopic display device according to a second embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected to the substantially same component as the three-dimensional display apparatus concerning the said 1st Embodiment, and description is abbreviate | omitted suitably.

  Although the overall configuration of the image display system in the present embodiment is the same as that of FIG. 1, an image compression unit 21A having the configuration shown in FIG. 10 is provided instead of the image compression unit 21 having the configuration of FIG. The image compression unit 21A has a stream decoder 45. The stream decoder 45 decompresses image data using the same codec as the stream decoder 52 in the image decompression unit 31 of the stereoscopic display device 5.

  The image compression unit 21 configured as shown in FIG. 2 stores the first parallax image sent from the first camera 61 as it is in the frame memory 41 as a base image. On the other hand, in the image compression unit 21A of FIG. 10, the base image compressed in the frame by the stream encoder 43 is decompressed as it is by the stream decoder 45 to restore the first parallax image, and the restored first image The parallax image is stored in the frame memory 41. Then, the difference between the base image (restored first parallax image) stored in the frame memory 41 and each of the other (n−1) parallax images is output to the stream encoder 43 for compression processing.

  According to the present embodiment, the stream decoder 45 stores the base image in the frame memory 41 after being decompressed by the same codec as the stream decoder 52 in the image decompressing unit 31. Therefore, the base image stored in the frame memory 41 and the base image stored in the frame memory 53 (FIG. 6) in the image decompression unit 31 can be substantially matched. Thereby, the error of the difference process in the image compression part 21A and the image decompression part 31 can be reduced.

<Third Embodiment>
Next, a stereoscopic display device according to the third embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected to the substantially same component as the three-dimensional display apparatus concerning the said 1st or 2nd embodiment, and description is abbreviate | omitted suitably.

  In the first embodiment, the stereoscopic display device 5 has been described as having a stereoscopic display function in multi-view. However, in the present invention, the stereoscopic display device 5 has a twin-lens function. It is also applicable to. In this case, on the stereoscopic display device 5 side, when the parallax image of n = 3 or more is included in the image data of the stereoscopic image, the signal processing unit 3 extracts any two parallax images and extracts them. A display image for binocular stereoscopic viewing may be generated based on the two parallax images thus obtained. Alternatively, a base image and a parallax image that is compressed next to the base image may be extracted, and a binocular stereoscopic display image may be generated based on the two extracted parallax images. .

  The present invention is also applicable when the stereoscopic display device 5 has a switching function between stereoscopic display and two-dimensional display. Furthermore, instead of the stereoscopic display device 5, the present invention can also be applied to a display device dedicated to two-dimensional display without a stereoscopic display function. In this case, on the display device side, any one parallax image may be extracted by the signal processing unit 3 to generate a display image for two-dimensional display. Alternatively, a base image may be extracted to generate a display image for two-dimensional display.

  Further, in order to make it easy to support image display on a binocular stereoscopic display device or a display device dedicated to two-dimensional display, images from the first to fourth cameras 61, 62, 63, 64 of the photographing unit 6 are used. You may make it change the priority of the image processing with respect to data. In other words, in the first embodiment, the camera image output from the first camera 61 is first compressed as a first parallax image (base image), and then the second to fourth cameras 62 are processed. , 63, 64 are sequentially compressed as second to fourth parallax images. This process is performed in the order of cameras.

  Instead, the image compression unit 21 may determine the order of compression processing of each parallax image, for example, according to the shooting position. For example, a camera image obtained by shooting from a central shooting position (for example, the second camera 62) is first compressed as a base image. Next, a camera image obtained by shooting from the shooting position (for example, the third camera 63) having the most suitable parallax for performing binocular stereoscopic display with respect to the center shooting position is obtained as a base image. Is compressed as the first parallax image that takes the difference between the two. Thereafter, camera images obtained by shooting from other shooting positions are sequentially processed. For example, the image by the first camera 61 is processed, and then the image by the fourth camera 64 is processed.

  In this way, instead of simply processing in the order in which the cameras are arranged, an image suitable for binocular stereoscopic display is preferentially processed, so that, for example, in a binocular stereoscopic display device, a base image and a base By extracting the parallax image compressed next to the image, a display image suitable for the binocular stereoscopic display can be easily generated.

<Fourth embodiment>
Next, a stereoscopic display device according to a fourth embodiment of the present invention will be described. Note that components that are substantially the same as those of the stereoscopic display device according to the first to third embodiments are denoted by the same reference numerals, and description thereof is omitted as appropriate.

  FIG. 11 shows a configuration example of the image data output device 4A in the image display system according to the fourth embodiment of the present invention. In the image data output device 4 shown in FIG. 1, the photographing unit 6 has four cameras 61, 62, 63, and 64, and directly generates four parallax images from them. On the other hand, in the image data output device 4A of FIG. 11, the photographing unit 6A is composed of only two cameras 61 and 64. The image data output device 4A includes a parallax image generation unit 25. The parallax image generation unit 25 generates at least one other parallax image by image processing (image interpolation processing) based on the plurality of parallax images output from the imaging unit 6A. In the example of FIG. 11, the parallax image generation unit 25 generates two parallax images from the data of the camera image 1 and the camera image 4 output from the imaging unit 6A.

  For example, the data of the camera image 1 output from the photographing unit 6A is the first parallax image, the data of the camera image 4 is the fourth parallax image, and the two parallax images generated by the parallax image generation unit 25 are the second and second parallax images. 3 parallax images are input to the image compression unit 21. The processing after the image compression unit 21 is the same as that in the first embodiment.

  The imaging unit 6A may be configured with three cameras, and the parallax image generation unit 25 may generate only one parallax image.

  According to the present embodiment, since some of the parallax images are generated by image processing, it is not necessary to prepare as many cameras as the number of parallaxes necessary for multi-view viewing, and the number of cameras is reduced. be able to. Thereby, the system configuration can be simplified.

<Other embodiments>
The present invention is not limited to the above embodiments, and various modifications can be made.
For example, in FIG. 3, as header information, information of “Header = 0xfffffffn” for recognizing the camera image number n, information of the byte size “Length = xx” [Byte] of the compressed data of the actual data portion following the header information, and Although an example of adding is shown, other header information may be added. For example, related information such as camera parallax information and camera sequence number may be added. This facilitates signal processing on the display device side.

  DESCRIPTION OF SYMBOLS 1 ... Parallax barrier, 1A ... Lenticular lens, 2 ... Two-dimensional display panel, 3 ... Signal processing part, 4, 4A ... Image data output device, 5 ... Stereoscopic display device, 6, 6A ... Shooting part, 10L ... Left eye DESCRIPTION OF SYMBOLS 10R ... Right eye, 11 ... Shielding part, 12 ... Slit part, 13 ... Cylindrical lens, 21, 21A ... Image compression part, 22 ... Image transmission part, 23 ... Transmission line, 24 ... Data storage, 25 ... Parallax image generation , 30... Image receiving unit, 31... Image decompressing unit, 32... Composite image generating unit, 33... Drive processing unit, 41 ... frame memory, 42 ... differencer, 43 ... stream encoder, 44 ... header creator, 45. Stream decoder 51 ... Header analyzer 52 ... Stream decoder 53 ... Frame memory 54 ... Adder 61, 62, 63, 64 ... Camera, 100, 100A Three-dimensional display unit.

Claims (11)

Image data that has an image compression unit and that is compressed by the image compression unit and output image data of a stereoscopic image in which n (n is an integer of 2 or more) parallax images are sequentially included in one frame period An output device;
A display unit for displaying an image; and a signal processing unit including an image decompression unit, wherein the signal processing unit decompresses the compressed image data in the image decompression unit, and based on the decompressed image data A display device configured to generate a display image to be displayed on the display unit,
The image compression unit
Among the n parallax images in one frame period of the stereoscopic image, one specific parallax image is directly subjected to intra-frame compression as a base image, and the other (n−1) parallax images are An image display system configured to take a difference between each parallax image and the specific parallax image used as the base image to form a difference image, and then perform intra-frame compression for each parallax image. The image decompression unit decompresses the base image compressed in the frame as it is to restore the specific parallax image, and for each of the other (n−1) parallax images, for each parallax image The image according to claim 1, further comprising: decompressing the intra-frame-compressed differential image, and then restoring the decompressed differential image by adding the restored specific parallax image. Display system. The image compression unit includes an encoder that directly compresses the specific parallax image as a base image, and a decoder that decompresses the intra-frame compressed base image and restores the specific parallax image. A frame memory for storing the specific parallax image,
For each of the other (n−1) parallax images, the difference between each parallax image and the specific parallax image stored in the frame memory is obtained as a difference image, and then each parallax image is obtained. The image display system according to claim 1, wherein the image display system is configured to compress within a frame. The image data of the stereoscopic image that is compressed in the image data output device is multi-view image data in which three or more parallax images are sequentially included in one frame period. The image display system according to any one of claims. The display unit is capable of stereoscopic display based on a stereoscopic display image including a plurality of parallax images.
The signal processing unit
By extracting and synthesizing parallax images corresponding to the number of parallaxes required for stereoscopic display on the display unit from the compressed image data of the stereoscopic image, stereoscopic display on the display unit is achieved. The image display system according to claim 4, wherein a stereoscopic display image that can be generated is generated. The display unit is capable of stereoscopic display based on a binocular stereoscopic display image including two parallax images,
The image data output device captures a subject at the same time from three or more different positions, and outputs three or more parallax images for multi-view viewing having parallax as image data of the stereoscopic image. Further comprising
The image compression unit determines the order of compression processing of each parallax image according to the shooting position, and first compresses a specific parallax image obtained by shooting from the center shooting position as the base image. Next, a difference between the parallax image obtained by shooting from the shooting position that is the most suitable parallax for performing binocular stereoscopic display with respect to the central shooting position and the base image is obtained. Compression processing is performed as the first parallax image,
The signal processing unit extracts a specific parallax image that is the base image and a parallax image that is compressed next to the base image from the compressed image data of the stereoscopic image, and extracts them The image display system according to claim 4, wherein the display image for binocular stereoscopic viewing is generated based on the two parallax images thus obtained. The display unit is capable of two-dimensional image display,
The signal processing unit
5. The image display system according to claim 4, wherein only the base image is extracted from the compressed image data of the stereoscopic image to obtain a display image on the display unit. The image data output device includes:
A shooting unit that outputs a plurality of parallax images having parallax by shooting a subject simultaneously from different positions;
A parallax image generation unit that generates at least one other parallax image based on a plurality of parallax images output from the imaging unit;
The image data of the stereoscopic image to be compressed by the image compression unit includes a plurality of parallax images output from the imaging unit and other parallax images generated by the parallax image generation unit within one frame period. The image display system according to claim 4, wherein the image display system includes multi-view image data that is sequentially included. The image compression unit includes a header creator for adding header information after the intra-frame compression for each of the base image and the difference image.
The image display system according to any one of claims 1 to 3, wherein the image display system is configured as described above. The image display system according to claim 9, wherein the header creator adds at least information capable of discriminating between the base image and the difference image as the header information. An image compression unit that compresses and outputs image data of a stereoscopic image in which n (n is an integer of 2 or more) parallax images are included in time sequence within one frame period;
The image compression unit
Among the n parallax images in one frame period of the stereoscopic image, one specific parallax image is directly subjected to intra-frame compression as a base image, and the other (n−1) parallax images are An image data output device configured to perform intra-frame compression for each parallax image after taking a difference between each parallax image and the specific parallax image as the base image to obtain a difference image.

Images