JP2011002843A - Image generating device for stereoscopic vision, stereoscopic vision video display device, and image generating method for stereoscopic vision - Google Patents

Abstract

PROBLEM TO BE SOLVED: To further improve the reproducibility of the brightness of an entire original image with respect to the brightness of an entire stereoscopic vision video.SOLUTION: A value is set for each synthetic image sub-pixel included in a lenticular lens L1 by obtaining the sum of three fourths of the luminance of a sub-pixel in an original image pixel corresponding to a synthetic image pixel P0 and one fourth of the luminance of a sub-pixel in the original image pixel corresponding to a synthetic image pixel P1. A value is set for each synthetic image sub-pixel included in a lenticular lens L2 by obtaining the sum of two fourths of the luminance of sub-pixels in the original image pixels corresponding to the synthetic image pixels P1 and P2. A value is set for each synthetic image sub-pixel included in a lenticular lens L3 by obtaining the sum of one fourth of the luminance of the sub-pixel in the original image pixel corresponding to the synthetic image pixel P2 and three fourths of the sub-pixel in the original image pixel corresponding to a synthetic image pixel P3.

Description

  The present invention relates to a method for generating a stereoscopic image.

  2. Description of the Related Art Conventionally, in a multi-lens lenticular stereoscopic image display apparatus, stereoscopic viewing is performed by sequentially assigning images 2 (hereinafter referred to as original images) 2 viewed from different viewpoints as shown in FIG. A subpixel interleaver (hereinafter referred to as an interleaver) that generates an image for use (an image displayed as a stereoscopic image through a lenticular plate) (hereinafter referred to as a composite image) is known. In general, images viewed from a plurality of different viewpoints are different from each other. However, in FIG. 2, the images viewed from all viewpoints are the same for the sake of simplicity. FIG. 2 shows an original image 2 in the case of a four-lens lenticular method (hereinafter referred to as a four-lens method). The original image 2 includes four original images (far left original image 2-0, left An original image 2-1, a right original image 2-2, and a far right original image 2-3) are included. The interleaver generates a composite image from these four original images. In the case of the four-lens system, the entire composite image is generated by interleaving four pixels at a time and repeating this. In FIG. 2, only four pixels are shown as the original image 2 for simplicity. . In the following description, only 4 pixels will be described for simplicity.

  Conventionally, as a method of generating a composite image from a plurality of original images, a subpixel of each original image corresponding to each subpixel of the composite image (hereinafter referred to as a composite image subpixel) is selected as shown in FIG. There has been a method (hereinafter referred to as direct sampling) in which the luminance of the selected sub-pixel is used as the luminance of the composite image sub-pixel. FIG. 8 shows a case where the far right original image 2-3, the right original image 2-2, the left original image 2-1, and the far left original image 2-0 are directly sampled.

More specifically, the direct sampling will be described. Each of the composite image subpixels r ₀ , g ₀ , b ₀ ,..., R ₃ , g ₃ , b ₃ included in the four composite image pixels P0, P1, P2, and P3. Are determined in turn based on the luminance of the sub-pixels of the far right original image 2-3, the right original image 2-2, the left original image 2-1, and the far left original image 2-0. That is, the brightness of the composite image subpixel r ₀ is determined from the far right original image 2-3, the brightness of the composite image subpixel g ₀ is determined from the right original image 2-2, and the brightness of the composite image subpixel b ₀ is The luminance of the composite image subpixel r ₁ is determined from the far left original image 2-0. Then, the luminance of the composite image sub-pixel g ₁ is determined again from the far-right original image 2-3.

Since the composite image sub-pixel r ₀ is a sub-pixel of the leftmost composite image pixel (composite image pixel P 0), the far right original image 2-3 pixel (hereinafter referred to as “far”) is compared with the composite image sub-pixel r ₀ . that right original image pixels.) of the subpixels r ₃₀ of the leftmost pixel P30 is selected. Similarly, the composite image subpixels g ₀ against the right original image 2-2 pixels of the (hereinafter. Referred right original image pixels), the sub-pixel g ₂₀ of the leftmost pixel P20 is selected and synthesized image sub For the pixel b ₀ , the sub-pixel b ₁₀ of the leftmost pixel P ₁₀ is selected from the pixels of the left original image 2-1 (hereinafter referred to as the left original image pixel). Moreover, the composite image subpixels r ₁ are the sub-pixel of the second synthesized image pixels from the left (the composite image pixels P1), with respect to the composite image subpixels r _1, the far left original image 2-0 Of the pixels (hereinafter referred to as the far left original image pixel), the sub pixel r ₀₁ of the second pixel P 01 from the left is selected.

Then, a synthesized image displayed as a stereoscopic image is generated through the lenticular plate L by setting the luminance of each synthesized image subpixel to the luminance of each selected original image subpixel. A composite image obtained by sampling and interleaving the original images 2-0, 2-1, 2-2, and 2-3 as shown in FIG. 2 by this direct sampling becomes an image as shown in FIG. That is, the image 3-3 seen from the far right viewpoint is composed of the synthesized image subpixels r ₀ , g ₁ , b ₂ in which the sub pixels of the far right original image 2-3 are selected by the lenses of the lenticular plate L. The image is enlarged and displayed in the width (4 composite image subpixels). The brightness of the composite image sub-pixel r _{0 is} the brightness of the far-right original image sub-pixel r ₃₀ , the brightness of the composite image sub-pixel g _{1 is} the brightness of the far-right original image sub-pixel g ₃₁ , and the composite image sub-pixel b ₂ Is the luminance of the far right original image sub-pixel b ₃₂ . In addition, the sub-pixel enlarged to the lens width is called an apparent display cell.

Similarly, in the stereoscopic video 3-2 viewed from the right viewpoint, the composite image subpixels g ₀ , b ₁ , and r ₃ in which the subpixels of the right original image 2-2 are selected are expanded to the lens width, respectively. The stereoscopic video 3-1 displayed from the left viewpoint is displayed with b ₀ , r ₂ , and g ₃ in which the sub-pixels of the left original image 2-1 are selected enlarged to the lens width. The stereoscopic video 3-0 seen from the far left viewpoint is displayed with r ₁ , g ₂ , and b ₃ in which the sub-pixels of the far left original image 2-0 are selected enlarged to the lens width. .

  In addition, as shown in FIG. 10, a method of selecting a sub-pixel from the original image pixel corresponding to the composite image pixel where the sample point is located (hereinafter referred to as point sampling) using the center of the apparent display cell as the sample point. It was used.

  FIG. 10 is a diagram for explaining point sampling. As shown in FIG. 10, the centers of the lenticular lenses L1, L2, and L3 of the lenticular plate L, that is, the center positions of the apparent display cells are set as sample points S1, S2, and S3, respectively. Then, subpixels are selected from the original image pixels corresponding to the composite image pixel where the sample point of each apparent display cell is located.

For example, the center of the lenticular lens L1, that is, the sample point S1, is located at the composite image pixel P0. Therefore, for the composite image subpixels r ₀ , g ₀ , b ₀ , r ₁ included in the leftmost apparent display cell (enlarged by the lenticular lens L1 when displayed as a stereoscopic image), respectively. A subpixel is selected from the original image pixels corresponding to the composite image pixel P0. That is, since the composite image pixel P0 is the leftmost composite image pixel, the subpixels of the leftmost original image pixels P00, P10, P20, and P30 of each original image are selected.

Accordingly, the far right original image subpixels r ₃₀ with respect to the display sub-pixels r ₀ is selected, the right original image sub-pixel g ₂₀ is selected for the composite image subpixels g _0, with respect to the composite image subpixels b ₀ left original image sub-pixel b ₁₀ is selected, the composite image subpixels r far left original image subpixels r ₀₀ against ₁ is selected Te. The luminance of the original image subpixel selected for each composite image subpixel is set as the luminance of each display subpixel.

In addition, since the sample point S2, which is the center of the lenticular lens L2, is located between the composite image pixel P1 and the composite image pixel P2, the composite image pixel is not determined by the position of the sample point S2. Therefore, the synthesized image subpixels g ₁ , b ₁ , r ₂ , and g ₂ enlarged by the lenticular lens L2 are selected from the synthesized image pixel P1 and the original image pixel corresponding to the synthesized image pixel P2. That is, two original image subpixels are selected for one composite image subpixel. Then, the average value of the luminances of the two selected original image subpixels is set as the luminance of the synthesized image subpixel. For example, with respect to the synthetic image subpixels g _1, far right original of the sub-pixels of the image 2-3, g ₃₁ and g ₃₂ are selected, the composite image subpixel average value of the luminance of the two subpixels the brightness of g _1.

For the synthesized image sub-pixels b ₂ , r ₃ , g ₃ , and b ₃ that are enlarged by the lenticular lens L3, the sample point S3 is located at the rightmost synthesized image pixel P3, and therefore the rightmost original image of each original image. The original image subpixels of the pixels P03, P13, P23, and P33 are selected. Then, the luminance of the selected original image subpixel is set as the luminance of each composite image subpixel.

International Publication No. 00/10332

  However, when a composite image is generated by sampling and interleaving by direct sampling from an original image 2 in which the same black and white pattern is repeated for all viewpoints as shown in FIG. The viewing video is a stereoscopic video (far right stereoscopic video 3-3, right stereoscopic video 3-2, left stereoscopic video 3-1, far left stereoscopic video 3-0) as shown in FIG. . As shown in FIG. 2, the far-right original image 2-3 is dark at the center and bright at both ends, but the far-right stereoscopic image 3-3 shown in FIG. The outline information is different from that of the original image 2-3. Similarly, the far left stereoscopic image 3-0 also has different outline information from the far left original image 2-0. As described above, there is a problem that the contour information of the original image is not reproduced in the stereoscopic video. Here, the contour information means the contour (shape) of the display object indicated by the luminance value (brightness / darkness) of each sub-pixel. The far-right original image 2-3 and the far-right stereoscopic video image 3-3 are different in shape from the original image in the bright part and the part, so that the contour information is not reproduced.

  In addition, there is a problem that the brightness of the entire stereoscopic image when displaying a composite image generated by direct sampling based on the same theory is different from the brightness of the entire original image. For example, in each original image shown in FIG. 2, the luminance of half of the subpixels in the entire original image is “1”, and the luminance of the remaining half of the subpixels is “0”. That is, it is half the brightness when all the subpixels of the original image are set to “1” (the brightest case). However, in the stereoscopic video shown in FIG. 9, in the far left stereoscopic video 3-0 and the far right stereoscopic video 3-3, the luminance of all sub-pixels in the entire stereoscopic video is set to “1”. In the left stereoscopic video 3-1 and the right stereoscopic video 3-2, the brightness is 2/3 when all the luminances of the sub-pixels of the stereoscopic video are “1”. Thus, the brightness of the entire original image is different.

  When a synthesized image is generated from the original image as shown in FIG. 2 by sampling by point sampling and interleaving, a stereoscopic video as shown in FIG. 11 is displayed. That is, the information of the original image pixels at both ends (white portion in the original image) is enlarged and displayed in the width of the lenticular lens, but the information for the center two pixels (black portion in the original image) is one lens width. Will be displayed in a reduced size. For example, in the case of an original image in which the center two pixels are black (dark) and the pixels at both ends are white (bright) as shown in FIG. 2, the black (dark) portion is reduced in the stereoscopic video. For this reason, there is a problem that the entire stereoscopic image becomes brighter than the entire original image.

  Also, for example, in the original image as shown in FIG. 2, the black and white pattern moves in the horizontal direction (the ratio between white and black does not change, only the position changes. That is, the brightness of the entire original image does not change. .) When displaying a moving image as a stereoscopic image, the brightness of the entire stereoscopic image generated and displayed by point sampling changes even though the brightness of the entire original image does not change (white and black). Each time the position of the image changes, the white part is reduced or the black part is reduced, so that the brightness of the entire stereoscopic image becomes brighter or darker than the brightness of the entire original image). There are problems such as eye strain and discomfort to the observer. Moreover, such a problem became more prominent as the number of viewpoints increased.

  An object of the present invention is to further improve the reproducibility of the brightness of the entire original image with respect to the brightness of the entire stereoscopic video.

The first invention for solving the above-described problem is that each of the sub-pixels of the stereoscopic video (for example, the composition shown in FIG. 3) is selected from a plurality of original images having different viewpoints (for example, the original image 2 shown in FIG. 2). Sub-pixels corresponding to the image sub-pixels r ₀ , g ₀ , b ₀ ,..., R ₃ , g ₃ , b ₃ ) (for example, the original image sub-pixels r ₀₀ , g ₀₀ , b ₀₀ , ..., R ₃₃ , g ₃₃ , or b ₃₃ ) are selected, and a stereoscopic image (composite image in the embodiment) generated by interleaving the selected sub-pixels is converted into a multi-lens lenticular method. An interleaver (for example, the interleaver 10 illustrated in FIG. 1) that is output to a stereoscopic video display device (for example, the display unit 30 illustrated in FIG. 1), and includes a display cell as a unit and is included in the display cell. , Pixel region of the stereoscopic image Based on the ratio, a interleaver comprising luminance setting means for setting the luminance of each sub-pixel of the stereoscopic image.

  A fifth invention is a stereoscopic video generation method for generating a stereoscopic video to be displayed on a multi-lens lenticular stereoscopic video display device, wherein a stereoscopic video is generated from a plurality of original images having different viewpoints. When selecting a subpixel corresponding to each subpixel and interleaving the selected subpixel, the luminance of each of the subpixels is expressed in units of display cells and included in the display cell. This is a stereoscopic video generation method that is determined based on the ratio of pixel regions.

  Here, the display cell refers to a sub-pixel (apparent display cell) of a stereoscopic image magnified by one lens of the lenticular plate. That is, the width of the display cell is equal to the width of the lenticular lens.

  According to the first or fifth invention, since the luminance of each sub-pixel of the stereoscopic video is set based on the ratio of the pixel area of the stereoscopic video included in the display cell, the original corresponding to each display cell is set. Since the ratio of each pixel of the image can be reflected on each subpixel of the stereoscopic video, the reproducibility of the brightness of the entire original image in the brightness of the stereoscopic video can be improved.

  According to a second aspect of the present invention, a stereoscopic video generated by selecting subpixels corresponding to each subpixel of a stereoscopic video image from a plurality of original images having different viewpoints and interleaving the selected subpixels is multiviewed. An interleaver for outputting to a lenticular stereoscopic video display device, comprising: a luminance setting means for setting the luminance of each subpixel of the stereoscopic video as a luminance average value of each subpixel of the original image Interleaver.

  A sixth invention is a stereoscopic video generation method for generating a stereoscopic video to be displayed on a multi-lens lenticular stereoscopic video display device, wherein a stereoscopic video is generated from a plurality of original images having different viewpoints. A stereoscopic video generation method that selects a subpixel corresponding to each subpixel and determines the luminance of each subpixel as a luminance average value of each subpixel of the original image when interleaving the selected subpixel. is there.

  According to the second or sixth invention, since the luminance of each subpixel of the stereoscopic image is set as the luminance average value of each subpixel of the original image, the luminance of the subpixel of the stereoscopic image is set to the luminance of each subpixel of the original image. The brightness of the pixel can be reflected. Therefore, the brightness of the entire stereoscopic video image can be made equal to the brightness of the entire original image.

  A third invention is an interleaver that generates a stereoscopic video for display on a multi-lens lenticular stereoscopic video display device based on a plurality of original images with different viewpoints, and can be set arbitrarily. Sampling method determining means for determining the selection and mixing ratio of subpixels corresponding to each subpixel of the stereoscopic video in the plurality of original images based on a coefficient (for example, matrix K shown in FIG. 1) Is an interleaver.

  A seventh invention is a stereoscopic video generation method for generating a stereoscopic video to be displayed on a multi-lens lenticular stereoscopic video display device based on a plurality of original images having different viewpoints, the original image The sub-pixel corresponding to each sub-pixel of the stereoscopic video is selected based on a sampling coefficient that can be arbitrarily set, and when the selected sub-pixel is interleaved, the luminance of each of the sub-pixels is selected. This is a stereoscopic video generation method that is determined according to a mixing ratio based on a sampling coefficient.

  According to the third or seventh invention, since the sampling method can be easily changed by changing the sampling coefficient, the sampling method can be selectively used, and the image quality can be adjusted more easily. For example, in the case of an image where brightness reproducibility is important, a stereoscopic video is generated from the original image by a sampling method with higher brightness reproducibility, and in the case of an image where contour information is important, A stereoscopic video based on the original image can be generated by a sampling method with high reproducibility of contour information. Further, for example, when the processing speed is important, a stereoscopic video can be generated by a sampling method capable of higher speed processing. Here, the sampling method refers to a method including sampling (selection of a subpixel corresponding to each subpixel of a stereoscopic video image) and interleaving.

  As a fourth invention, a stereoscopic video display apparatus including the interleaver according to any one of the first to third inventions may be configured.

  According to the fourth aspect of the invention, it is possible to provide a stereoscopic video display device that generates and displays a stereoscopic video with improved reproducibility of the brightness of the entire original image. As a result, the flicker of the video can be improved.

It is a figure which shows schematic structure of the stereoscopic vision video display apparatus in this Embodiment. It is a figure which shows an example of an original image. It is a figure explaining the principle of area sampling. It is a figure which shows an example of the stereoscopic vision image displayed when an image is synthesize | combined by area sampling. It is a figure explaining the principle of a total averaging sampling. It is a figure which shows an example of the stereoscopic vision image displayed when an image is synthesize | combined by total averaging sampling. It is a flowchart which shows an example of the operation | movement which concerns on the sampling and interleaving in this Embodiment. It is a figure explaining the principle of direct sampling. It is a figure which shows an example of the stereoscopic vision image displayed when an image is synthesize | combined by direct sampling. It is a figure explaining the principle of point sampling. It is a figure which shows an example of the stereoscopic vision image displayed when an image is synthesize | combined by point sampling.

  Preferred embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram illustrating a schematic configuration of a stereoscopic video display apparatus 100 including an interleaver to which the present invention is applied. In the figure, a stereoscopic video display apparatus 100 includes an interleaver 10 and a display unit 30. The interleaver 10 is a stereoscopic image generating device that samples and interleaves the original image 2 based on a matrix K input from the outside and displays the original image 2 on the display unit 30. In addition, the image processing apparatus includes a RAM that stores an image generation program, a CPU that generates a stereoscopic video by executing the program, and the like. Further, the interleaver 10 may be configured as a circuit using a dedicated image generation IC.

  The display unit 30 is a stereoscopic display that includes a liquid crystal display device or the like and includes a lenticular plate. The display unit 30 displays a stereoscopic image via the lenticular plate by displaying the composite image interleaved by the interleaver 10.

  FIG. 1 shows a four-eye stereoscopic image display device 100. The original image 2 includes a far left original image 2-0, a left original image 2-1, a right original image 2-2, and a far right original. Four original images of the image 2-3 are sampled and interleaved by the interleaver 10. In FIG. 1, the original image 2 is input from the outside of the stereoscopic video display device 100, but is stored in advance in the stereoscopic video display device 100 or generated in the stereoscopic video display device 100. It is also good. Specifically, for example, a case where the stereoscopic video display device 100 is a game device and a game image is generated and displayed can be considered.

  Although details will be described later, the matrix K is a coefficient that determines a sampling method when the interleaver 10 performs sampling and interleaving. In FIG. 1, the matrix K is also input from the outside of the stereoscopic video display device 100 as with the original image 2, but may be stored in the stereoscopic video display device 100.

  FIG. 2 is a diagram illustrating an example of the original image 2. In the figure, the four original images 2-0, 2-1, 2-2, and 2-3 (in four viewpoints) as the original image 2 are all the same image, and the four pixels are white from the left ( Assume that the values of r, g, and b (luminance) are all “1”), black (the values of r, g, and b are all “0”), black, and white. Throughout this specification, the luminance of each sub-pixel (r, g, b) is “1” when it is brightest and “0” when it is darkest.

FIG. 3 is a diagram illustrating an example of sampling and interleaving by the interleaver 10 in the present embodiment. Hereinafter, the sampling and interleaving method shown in FIG. 3 is referred to as area sampling. FIG. 3A is a diagram illustrating a ratio of the composite image pixels included in the region by dividing the composite image by a region according to the width of each lenticular lens of the lenticular plate L. For example, three composite image subpixels (composite image subpixels r ₀ , g ₀ , b ₀ , r ₁ ) delimited by the lenticular lens L1 belong to the composite image pixel P0 (composite image subpixels r ₀ , g ₁ ). ₀ , b ₀ ) and one that belongs to the composite image pixel P1 (composite image sub-pixel r ₁ ).

Therefore, as shown in FIG. 3B, the composite image subpixels r ₀ , g ₀ , b ₀ , and r _{1 in the} region delimited by the lenticular lens L1 correspond to the position of the composite image pixel P0. The luminance of the subpixels P30, P20, P10 and P00 of the original image to be multiplied by 3/4 and the luminance of the subpixels of the original image pixels P31, P21, P11 and P01 corresponding to the composite image pixel P1 are 1 Set a value obtained by adding / 4 times.

For example, the brightness of the composite image sub-pixel r ₀ is obtained by multiplying the brightness of the far-right original image sub-pixel r ₃₀ by 3/4 and the brightness of the far-right original image sub-pixel r ₃₁ by 1/4. Set to the added value.

Further, as shown in FIG. 3A, the composite image subpixels (composite image subpixels g ₁ , b ₁ , r ₂ , g ₂ ) in the region delimited by the lenticular lens L2 belong to the composite image pixel P1. Are two (synthesized image subpixels g ₁ and b ₁ ) and two belong to the synthesized image pixel P 2 (synthesized image subpixels r ₂ and g ₂ ).

Therefore, as shown in FIG. 3B, for the composite image subpixels g ₁ , b ₁ , r ₂ , and g _{2 in the} region delimited by the lenticular lens L2, each corresponds to the position of the composite image pixel P1. The luminance of the subpixels of the original image pixels P31, P21, P11, and P01 to be multiplied by 2/4 (= 1/2) and the subpixels of the original image pixels P32, P22, P12, and P02 corresponding to the composite image pixel P2 A value obtained by adding the luminance of the pixel multiplied by 2/4 (= 1/2) is set.

For example, the luminance of the composite image sub-pixel g ₁ is obtained by halving the luminance of the far-right original image sub-pixel g ₃₁ and halving the luminance of the far-right original image sub-pixel g _32. Set to the added value.

  Similarly, for the composite image subpixel in the region delimited by the lenticular lens L3, the luminance of the subpixels of the original image pixels P32, P22, P12 and P02 corresponding to the position of the composite image pixel P2 is multiplied by 1/4. And a value obtained by adding 3/4 times the luminance of the sub-pixels of the original image pixels P33, P23, P13, and P03 corresponding to the position of the composite image pixel P3.

  FIG. 4 is an original image as shown in FIG. 2, that is, white, black, black, white patterns, and when the original images at all viewpoints are the same, each interleaver 10 generates each area by sampling. It is a figure which shows the stereoscopic vision image of a viewpoint. As shown in FIG. 2, the brightness of the entire image of the original image is half that of the case where the entire image is white (brightest). Further, the brightness of the entire stereoscopic video generated by the area sampling shown in FIG. 4 is also half the brightness when the entire stereoscopic video is white.

  That is, when a stereoscopic video is generated by area sampling, if the luminance of the subpixels (r, g, b) in each pixel is the same as in the original image shown in FIG. The overall brightness can be reproduced as the brightness of the entire stereoscopic video image by the composite image generated by the interleaver 10.

Next, the relationship between the subpixel value (luminance) of each original image and the value of each subpixel after interleaving will be described. Here, the case where the number of viewpoints is set to n (n is a natural number other than a multiple of 3) is described without being limited to the case of the four-lens system (that is, the number of viewpoints is 4). Equation (1) is an equation showing the relationship between the sub-pixel value of each original image and each sub-pixel value sI _j after interleaving.

Here, (sI ₀ , sI ₁ , sI ₂ ,..., SI _3n-1 ) = (r ₀ , g ₀ , b ₀ ,..., R _n−1 , g _n−1 , b _{n− 1} ). In addition, v = (n-1- (j mod n), c = j mod 3, j = 0, 1, 2,..., 3n-1, where mod is an operator indicating a remainder. It is.

  v indicates a number corresponding to the original image of each viewpoint (for example, in the case of a four-lens system, v = 3, 2, 1, 0). C represents r, g, and b of the original image. (For example, c = 0 for r, c = 1 for g, and c = 2 for b).

  j indicates the number of the composite image sub-pixel. For example, in the case of an n-eye system, the number of pixels is n (sampling and interleaving are performed every n pixels). In addition, since 3 subpixels are included in one pixel, the number of composite image subpixels is 3n. Therefore, the value of j is “0” to “3n−1”.

  In addition, i indicates a pixel number (for example, in the case of an n-eye system, i takes a value from “0” to “n−1” for sampling and interleaving every n pixels).

That is, s _{v, 3i + c} represents each subpixel of each original image before sampling. k _{j, i} indicates a ratio of mixing the sub-pixel values, and Equation (2) holds for all j (= 0, 1, 2,..., 3n−1).

Further, the vector k _j = (k _{j, 0} , k _{j, 1} ,..., K _{j, n−1} ), the vector s _j = (s _{v, c} , s _{v, 3 + c} ,. s _{v, 3 (n-1) + c} ), Equation (1) is expressed as Equation (3).

sI _j = k _j · s _j (3)
(J = 0, 1, ..., 3n-1)

Further, when the vector k _j for each value of _j is represented by a matrix K, the matrix K is expressed by Equation (4).

  For example, when performing area sampling in a four-lens system, the matrix K is expressed by Expression (5).

As described above, if sampling and interleaving are performed according to Equation (1) _{, the} sampling method can be changed by changing the matrix K, that is, k _{j, i} in Equation (1).

  For example, Expression (6) is an expression that represents a matrix K when direct sampling is performed in a four-eye system.

  Equation (7) is an equation representing a matrix K when performing point sampling in the four-lens method.

  In addition to the above three samplings, for example, sampling and interleaving may be performed using a matrix K as shown in Equation (8).

FIG. 5 is a diagram for explaining sampling in a four-eye system (hereinafter referred to as total averaging sampling) when the matrix K shown in Expression (8) is used. As shown in the figure, for each composite image sub-pixel, the corresponding sub-pixel of all the pixels of the corresponding original image (for example, far right original image 2-3 when the composite image pixel is r ₀ ). For example, when the composite image subpixel is r ₀ , an average value of r subpixels (r ₃₀ , r ₃₁ , r ₃₂ , r ₃₃ ) of the far right original image 2-3 is set.

  For example, when the original image shown in FIG. 2 is subjected to total averaging sampling and interleaved, a stereoscopic video as shown in FIG. 6 is obtained. That is, the outline information of the stereoscopic video generated by the interleaver 10 is different from the original image, but the brightness of the entire image of the original image is reproduced as the brightness of the entire stereoscopic video by the composite image generated by the interleaver 10. can do.

In this way, by changing the matrix K (for example, in the case of a four-lens system, as in Expressions (5) to (8)), that is, by changing k _{j, i} in Expression (1). The sampling method can be changed.

  For example, if the contour information is more important, point sampling, color information, and overall brightness information are more important. When both are important, the image quality can be adjusted by properly using sampling such as area sampling. That is, the generated stereoscopic image can have higher image quality.

  In addition, for example, when priority is given to high-speed processing over image quality, direct sampling can be used properly.

  Next, operations related to sampling and interleaving in the present embodiment will be described based on the flowchart shown in FIG. Note that the operation shown in FIG. 7 is an operation related to a composite image generation process of a still image (one frame image in a moving image).

  First, the interleaver 10 determines the sampling method, that is, determines the matrix K (step S1). Next, n (n = 4 in the case of a four-lens type) pixel is acquired from the original image of each viewpoint (step S2), and sampling and sampling are performed according to equation (1) based on the matrix K determined in step S1. Interleaving is performed (step S3).

  Then, it is determined whether or not all the pixels of the original image have been completed (step S4). If not, the process returns to step S2 to acquire the next n pixels and the processes of steps S3 to S4 are performed. repeat. When the interleaving is completed for all the pixels, the synthesized image is displayed according to the luminance of the synthesized image pixel after the interleaving (step S5), and the process is terminated.

  Note that it is also possible to generate and display a stereoscopic video for a moving image by repeating the processing of step S2 to step S5 and sequentially generating and displaying a composite image of each frame.

  As described above, according to the present invention, the reproducibility in stereoscopic video of the brightness of the entire original image can be further improved by area sampling. Therefore, for example, screen flickering (change in screen brightness) that occurs when a moving image in which a monochrome pattern image as shown in FIG. 2 moves in the horizontal direction is displayed as a stereoscopic image is reduced. Can do.

  In addition, since the sampling can be changed by changing the matrix K, the sampling can be easily changed according to the image, and a higher quality stereoscopic image can be displayed.

  The present invention is not limited to the one described in the above embodiment, and various modifications can be made. For example, in the above embodiment, the original image is white, black, black, and white pattern as shown in FIG. 2 and the original images for all viewpoints are equal. The images for each viewpoint may be different from each other.

  For example, when the values of r, g, and b are different for each pixel of the original image, the brightness of the entire original image and the brightness of the entire stereoscopic video image may be different when the composite image is generated by area sampling. . However, since the brightness of the composite image sub-pixel is set based on the ratio of the composite image pixel included in the region divided by each lenticular lens, the composite image pixel included in the region can be reflected. Therefore, the brightness of the original image can be reflected on the brightness of the stereoscopic video image, compared to the conventional point sampling or direct sampling.

2 Original image 100 Stereoscopic image display device 10 Interleaver 11 Display unit

Claims (4)

A stereoscopic image that generates the composite image by interleaving original image sub-pixels corresponding to each of the composite image sub-pixels constituting the composite image displayed as a stereoscopic image from a plurality of original images having different viewpoints A generating device,
In each of the plurality of original images, an original image subpixel included in the original image pixel corresponding to each of the plurality of combined image pixels related to the display cell, and the original image subpixel having the same color as the combined image subpixel of the luminance setting target Is generated based on the area ratio between the combined image pixels of the display cell and the brightness of the combined image subpixel is set to generate the combined image.   The stereoscopic image generating apparatus according to claim 1, further comprising original image generating means for generating the plurality of original images having different viewpoints.   A multi-lens lenticular stereoscopic image display device comprising the stereoscopic image generating device according to claim 1. A stereoscopic image that generates the composite image by interleaving original image sub-pixels corresponding to each of the composite image sub-pixels constituting the composite image displayed as a stereoscopic image from a plurality of original images having different viewpoints A generation method,
In each of the plurality of original images, an original image subpixel included in the original image pixel corresponding to each of the plurality of combined image pixels related to the display cell, and the original image subpixel having the same color as the combined image subpixel of the luminance setting target A stereoscopic image generation method including generating the composite image by combining the brightness of the composite image based on a region ratio between the composite image pixels of the display cell and setting the brightness of the composite image sub-pixel.

Images