WO2012121222A2

WO2012121222A2 - Intermediate image generation method, intermediate image generation device, three-dimensional image generation method, three-dimensional image generation device, and three-dimensional image generation system

Info

Publication number: WO2012121222A2
Application number: PCT/JP2012/055591
Authority: WO
Inventors: 吉田　健治
Original assignee: Yoshida Kenji
Priority date: 2011-03-07
Filing date: 2012-03-05
Publication date: 2012-09-13
Also published as: JP2012186739A; JP5881959B2; WO2012121222A3

Abstract

In conventional three-dimensional image generation systems, in order to suppress image quality degradation as much as possible, complicated interpolation calculations are carried out in real-time from multiple-viewpoint original images, it is necessary to generate three-dimensional images, resource consumption is high and extremely specialised and expensive computers, etcetera, are required. Also, the inability to output or send multiple-viewpoint original images in a standard format represents a great problem for the spread of three-dimensional image display technology. In this completely new three-dimensional image generation system, by generating in advance an intermediate image which has the same resolution as a three-dimensional image and in which the pixels for each viewpoint have been compiled, a three-dimensional image can be generated by converting pixel arrangement and without using high-speed specialised computers or the like, and by using an intermediate image in which multiple-viewpoint images are arranged in a tile-shape, an intermediate image which is outputted or sent in a standard format by a standard image output device or image delivery server is used to generate a three-dimensional image by means of a simple and low-cost three-dimensional image generation device

Description

Intermediate image generation method, intermediate image generation device, stereoscopic image generation method, stereoscopic image generation device, stereoscopic image generation system

The present invention relates to a method and apparatus for converting a plurality of viewpoint images taken from a plurality of viewpoints to generate an intermediate image and a stereoscopic image, a system for generating a stereoscopic image using them, and an apparatus for displaying the stereoscopic image And about.

Conventionally, a display screen is configured by arranging images taken from a plurality of viewpoints obliquely in units of pixels or sub-pixels for each viewpoint, and a stereoscopic image is visually recognized by providing a barrier in front of the display screen. Is widely used (for example, Patent Document 1).

In addition, as an existing technique for converting the pixel arrangement of an image into a pixel arrangement corresponding to a stereoscopic image display device, Patent Document 2 discloses that the pixel arrangement of an image is converted into a pixel arrangement corresponding to a parallax barrier type stereoscopic image display device. This technique is disclosed in paragraph 0088 and FIG. Paragraph 0088 and FIG. 1 disclose a stereoscopic image display apparatus provided with an image composition circuit as a technical solution for converting the pixel arrangement of an image into a stereoscopic display in the apparatus.

Further, Non-Patent Document 1 discloses software for converting a pixel arrangement of an image for stereoscopic display to be executed on a personal computer.

In such a stereoscopic image generation process, generally a large-scale stereoscopic image display system, that is, a special and expensive graphic board capable of executing complex arithmetic processing for generating a stereoscopic image from a captured image at high speed is installed. A computer is an essential element.

Patent Documents 3 to 5 disclose processing methods for converting three primary color signals composed of RGB into signals of four primary colors such as RGBY or five primary colors.

JP 2004-191570 A JP-A-2004-179806 JP 2001-306023 A Special table 2004-529396 JP 2001-209047 A

However, as described above, providing a stereoscopic image requires a very special and expensive computer with high resource consumption, such as a high-speed CPU, a large-capacity memory, and a high-performance graphic board. It has become a big issue in the spread of.

In addition, when an image file that can be stereoscopically viewed is compressed, the images taken for each viewpoint are arranged consecutively for each pixel or sub-pixel, so they interfere with each other and are decompressed and played back. However, there is a problem that the image is distorted to such an extent that it can no longer be stereoscopically viewed, and a stereoscopic image cannot be compressed. Therefore, images for each viewpoint are compressed and transmitted, or stored in a storage medium, and images for each viewpoint are decompressed when generating stereoscopic images, and complex interpolation calculations are performed at high speed to generate stereoscopic images. There was a need to do.

Furthermore, an image in which the images of the respective viewpoints are arranged in a tile shape often has a resolution and an aspect ratio different from that of a stereoscopic image, that is, the resolution of an autostereoscopic display. In particular, in order to bring the image quality when a stereoscopic image is generated as close as possible to the highest quality, it is necessary to have a certain resolution or higher for each viewpoint image. Become bigger. For this reason, a high-speed transmission network is required to compress and transmit, and a larger-capacity storage medium is required. Of course, even if the images for each viewpoint are not arranged in a tile shape and are independent images, the total problem is the same.
Therefore, the biggest problem is that it is not possible to provide images for each viewpoint for generating stereoscopic images in a standard format for storing and transmitting video on Blu-ray or STB. is there.

Therefore, in order to solve the above-described problem, the present invention generates a high-speed special computer or the like by generating in advance an intermediate image having the same resolution as that of the stereoscopic image that is the final output image and collecting pixels for each viewpoint. It is possible to generate a stereoscopic image by simply changing the arrangement of pixels without using it, and using an intermediate image in which the images of each viewpoint are arranged in a tile shape, standard image output devices such as Blu-ray and STB and image distribution It is a technical object to realize a stereoscopic image generation system that generates a stereoscopic image from the intermediate image output or transmitted by a server in a standard format by a simple and inexpensive stereoscopic image generation apparatus (converter).

The problem solving means of the present invention will be described below. The present application includes inventions according to a plurality of independent claims, and the technical ideas intended by the inventor are the following five.

The first invention of the present application is an intermediate image that generates a plurality of intermediate images used to generate a stereoscopic image converted from a plurality of viewpoint images photographed and / or drawn from a plurality of viewpoints from one viewpoint to N viewpoints. An RGBY staircase in which RGBY staircase-arranged pixel units in which subpixels made of RGBY are arranged in four rows diagonally at a corner in an oblique direction are connected in a horizontal direction from the first viewpoint to the N viewpoint. In order to generate the stereoscopic image by repeatedly arranging the arrangement pixel blocks, the R, G, B, and Y values of the sub-pixels constituting the RGBY step arrangement pixel unit are converted into the RGBY step arrangement pixel unit. Constituting at least one or more pixel units arranged in the vicinity of the corresponding positions in the plurality of viewpoint images corresponding to the arrangement positions of the sub-pixels constituting An arrangement in which RGBY parallel arrangement pixel units obtained by interpolating from RGBY values of subpixels and arranging the subpixels constituting the RGBY staircase arrangement pixel unit in the order of RGBY in the horizontal direction are arranged together for each of the plurality of viewpoints. By arranging according to the rules and generating the intermediate image for each of the plurality of viewpoints, the total number of the RGBY staircase arranged pixel units of the stereoscopic image and the RGBY parallel arranged pixel units of the plurality of intermediate images, or each of them An intermediate image generation method characterized in that the total number of sub-pixels constituting the same number is the same.

The second invention of the present application is an intermediate image that generates a plurality of intermediate images used to generate a stereoscopic image converted from a plurality of viewpoint images photographed and / or drawn from a plurality of viewpoints from one viewpoint to N viewpoints. In the generation method, the N viewpoints are 3n + 1 (n is a natural number) viewpoints, and RGB staircase-arranged pixel units in which RGB subpixels are arranged in three rows in a diagonal direction in contact with corners are arranged in the horizontal direction. In order to generate the stereoscopic image by repeatedly arranging the RGB staircase-arranged pixel blocks arranged in a sequence from 1 viewpoint to N viewpoints, the R value and G value of each of the sub-pixels constituting the RGB staircase-arranged pixel unit are generated. , B values corresponding to the arrangement positions of the sub-pixels constituting the RGB staircase arrangement pixel unit, at least arranged in the vicinity of the corresponding positions in the plurality of viewpoint images An RGB parallel arrangement pixel unit obtained by interpolating RGB values of subpixels constituting one or more pixel units and arranging the subpixels constituting the RGB staircase arrangement pixel unit in the horizontal direction in the order of R, G, B The RGB staircase-arranged pixel unit of the stereoscopic image and the plurality of intermediate images are generated by arranging according to an arrangement rule for collectively arranging the plurality of viewpoints and generating the intermediate image for each of the plurality of viewpoints. The total number of RGB parallel-arranged pixel units or the total number of sub-pixels constituting each is the same, and the plurality of intermediate images are arranged in the form of a plurality of belts divided by the number of viewpoints in the horizontal direction as image frames. An intermediate image generation method characterized by:

The third invention of the present application is an intermediate image that generates a plurality of intermediate images used to generate a stereoscopic image converted from a plurality of viewpoint images photographed and / or drawn from a plurality of viewpoints from one viewpoint to N viewpoints. In the generation method, the N viewpoints are 4n + 1 (n is a natural number) viewpoints, and RGBY staircase-arranged pixel units in which RGBY sub-pixels are arranged in four rows in contact with diagonal corners are arranged in the horizontal direction. In order to generate the stereoscopic image by repeatedly arranging the RGBY staircase-arranged pixel blocks arranged in a concatenation from 1 viewpoint to N viewpoints, the R value and G value of each of the sub-pixels constituting the RGBY staircase-arranged pixel unit are generated. , B value, and Y value are arranged in the vicinity of the corresponding positions in the plurality of viewpoint images corresponding to the arrangement positions of the sub-pixels constituting the RGBY staircase arrangement pixel unit. Obtained by interpolating from the RGBY values of the sub-pixels constituting at least one pixel unit, and arranging the sub-pixels constituting the RGBY stepped pixel unit in the horizontal direction in the order of R, G, B, Y. An RGBY parallel arrangement pixel unit is arranged according to an arrangement rule for collectively arranging the plurality of viewpoints, and the intermediate image for each of the plurality of viewpoints is generated, whereby the RGBY staircase arrangement pixel unit of the stereoscopic image and The total number of the RGBY parallel arrangement pixel units of the plurality of intermediate images, or the total number of sub-pixels constituting each of the plurality of intermediate images is the same, and the plurality of intermediate images are divided into the number of viewpoints in the horizontal direction as image frames. An intermediate image generation method characterized by being arranged in a plurality of belts.

The fourth invention of the present application is an intermediate image that generates a plurality of intermediate images used to generate a stereoscopic image converted from a plurality of viewpoint images photographed and / or drawn from a plurality of viewpoints from one viewpoint to N viewpoints. In the generation method, the N viewpoints are (3 + m) n + 1 (n and m are natural numbers) viewpoints, and RGB and other m sub-pixels are arranged in 3 + m rows diagonally in contact with corners. The staircase-arranged pixel unit is configured to generate the stereoscopic image by repeatedly arranging the staircase-arranged pixel blocks in which the staircase-arranged pixel units arranged in the horizontal direction from the 1st viewpoint to the Nth viewpoint are arranged repeatedly. The R value, G value, B value, and other color values of each of the subpixels are assigned to the plurality of viewpoint images corresponding to the arrangement positions of the subpixels constituting the staircase arrangement pixel unit. R, G, B, and other colors in the horizontal direction are obtained by interpolating from the values of subpixels constituting at least one or more pixel units arranged in the vicinity of the corresponding position. The staircase arrangement of the three-dimensional image is generated by arranging the parallel arrangement pixel units arranged in the order of the plurality of viewpoints according to an arrangement rule for collectively arranging the plurality of viewpoints and generating the intermediate image for each of the plurality of viewpoints. The total number of pixel units and the total number of pixel units arranged in parallel in the plurality of intermediate images, or the total number of sub-pixels constituting each pixel unit, and the plurality of intermediate images are divided into the number of viewpoints in the horizontal direction as image frames. An intermediate image generation method, characterized by being arranged in a plurality of belt shapes.

The fifth invention of the present application is an intermediate image for generating a plurality of intermediate images used for generating a stereoscopic image converted from a plurality of viewpoint images photographed and / or drawn from a plurality of viewpoints from one viewpoint to N viewpoints. A method of generating a pixel unit in which sub-pixels composed of RGB or RGB plus at least one of C, M, Y, W or other colors are connected in the vertical direction. In order to generate the three-dimensional image by repeatedly arranging pixel blocks connected in a direction from the 1 viewpoint to the N viewpoint, the values of the sub-pixels constituting the pixel unit are configured as the pixel unit. And interpolating from the values of the sub-pixels constituting at least one or more pixel units arranged in the vicinity of the corresponding positions in the plurality of viewpoint images corresponding to the arrangement positions of the sub-pixels. The parallel arrangement pixel units in which the sub-pixels constituting the pixel unit are arranged in a horizontal direction are arranged in accordance with an arrangement rule for collectively arranging the plurality of viewpoints, and the intermediate images for the plurality of viewpoints are arranged. An intermediate image generation method, characterized in that, by generating, the total number of pixel units of the stereoscopic image and the parallel arrangement pixel units of the plurality of intermediate images, or the total number of sub-pixels constituting each, is the same. .

According to the present invention, it is possible to generate a compressible intermediate image with the minimum necessary resolution so that the total number of pixels constituting the image for each viewpoint is the same as the number of pixels of the stereoscopic image, and to configure such an intermediate image. 3D images can be generated at high speed without using a special computer by only rearranging (mapping) the subpixels to be used, and images of each viewpoint are arranged in tiles according to the present invention. By doing so, the resolution and aspect ratio of the intermediate image and the autostereoscopic display (stereoscopic image) become the same, and the intermediate image is output or transmitted in a standard format by a standard image output device such as Blu-ray or STB or an image distribution server. Provides a practical 3D image display system that can easily generate 3D images with a 3D image generation device (converter) at a very low price. Theft is possible.

It is a block diagram which shows roughly the structure of an intermediate | middle image generation apparatus, a stereo image generation apparatus, and a stereo image generation system. It is a figure which shows the flowchart of the information processing performed by an intermediate image generation apparatus and a stereo image generation system. It is a figure explaining embodiment about the production | generation method of an intermediate image. It is a figure explaining embodiment about the production | generation method of an intermediate image. It is a figure explaining embodiment about the production | generation method of an intermediate image. It is a figure explaining embodiment about the production | generation method of an intermediate image. It is a figure explaining an intermediate image generation table. It is a figure which shows the example of arrangement | positioning of the image frame of an intermediate image. It is a figure explaining the difference of an image frame. It is a figure which shows the example of the image frame which consists of a some intermediate | middle image. It is a figure which shows the image frame of a 5 viewpoint format. It is a figure which shows the example of the image frame of a belt-like format. It is a figure which shows the example of the image frame of a belt-like format. It is a figure which shows the example of the image frame of a belt-like format. It is a figure which shows the example of the image frame of a belt-like format. It is a figure which shows the example of arrangement | positioning of RGB staircase arrangement pixel units. It is an external view which shows an example of embodiment of a stereo image production | generation apparatus. It is an external view which shows an example of other embodiment of a stereo image production | generation apparatus. It is a figure which shows the example of the image of a several viewpoint. It is a figure which shows the example of the moving image of multiple viewpoints. It is a figure which shows the example of the moving image of multiple viewpoints. It is a figure which shows the 1st Example of the intermediate image of a several viewpoint. It is a figure which shows the 2nd Example of the intermediate image of a several viewpoint. It is a figure which shows the 3rd Example of the intermediate image of a several viewpoint. It is a figure which shows the 4th Example of the intermediate image of a several viewpoint. It is a figure explaining what kind of information the image information actually means. It is a figure explaining what kind of information the image information actually means. It is a figure which shows the flowchart which showed the method of discriminating a planar image and the intermediate image of several viewpoints. It is a figure explaining the appropriate value of lateral width Sh of a visible light transmission part. It is a figure explaining the appropriate value of lateral width Sh of a visible light transmission part. It is a figure explaining the example which calculates | requires height Sv of a visible light transmission part. It is a figure explaining the example which calculates | requires height Sv of a visible light transmission part. It is a figure explaining the example which calculates | requires the space | interval Hh of a some visible light transmissive part. It is a figure explaining the example which calculates | requires the space | interval Hh of a some visible light transmissive part. It is a figure which shows the example which calculates | requires the space | interval Hh of a some visible light transmissive part. It is a figure which shows the example which calculates | requires the space | interval Hh of a some visible light transmissive part. It is a figure explaining a moire. It is a figure explaining the relative relationship of L2, L2n, and L2f. It is a figure explaining the method of calculating | requiring the value of the space | interval Hv of a visible light transmissive part. It is a figure explaining the relationship between a visible light transmissive part and a pixel. It is a figure explaining the relationship between the distance K between gazing points, and L3. It is a figure explaining the method of calculating | requiring the value of the space | interval Hv of a visible light transmissive part. It is a figure explaining the method of calculating | requiring the value of the space | interval Hv of a visible light transmissive part. It is a figure explaining the number of visible light transmission parts. It is a figure explaining the relationship between [Hv × (Mv−1)] and [(Jr−1 / β) × Pv]. It is a figure which shows the example which calculates | requires the value of L3n and L3f. It is a figure which shows the example which calculates | requires the value of L3n and L3f. It is a figure which shows the example which calculates | requires the value of L3n and L3f. It is a figure explaining an example of the shape of the slit of a parallax barrier. It is a figure explaining an example of the shape of the slit of a parallax barrier. It is a figure explaining an example of the shape of the slit of a parallax barrier. It is a figure explaining an example of the shape of the slit of a parallax barrier. It is a figure explaining an example of the shape of the slit of a parallax barrier. It is a figure explaining an example of the shape of the slit of a parallax barrier. It is a figure explaining a moire appropriate cancellation area. It is a figure which shows the example of the shape of the sub pixel in an autostereoscopic image display apparatus. It is a figure which shows the stereo image which arrange | positions a RGB sub pixel in the orthogonal | vertical direction. It is a figure which shows the stereo image which arrange | positions a RGB sub pixel in the orthogonal | vertical direction. It is a figure which shows the stereo image which arrange | positions a RGB sub pixel in a horizontal direction. It is a figure which shows the appearance in the horizontal direction of the stereo image which has arrange | positioned RGB sub pixel in the horizontal direction. It is a figure which shows the appearance in the horizontal direction of the stereo image which has arrange | positioned RGB sub pixel in the horizontal direction. It is a figure explaining a stereo image production | generation table. It is a figure explaining a stereo image production | generation table.

Embodiments of the present invention will be described.

FIG. 1 is a block diagram schematically showing the configuration of an intermediate image generation device 101, the configuration of a stereoscopic image generation device 201, and the configuration of a stereoscopic image generation system 301 according to the present invention.

<Configuration of intermediate image generation apparatus>
The intermediate image generation apparatus 101 in FIG. 1A includes a central processing unit (part 1) 1011 and a storage unit (part 1) 1012.

<Storage device (part 1) 1012>
The storage device (part 1) 1012 stores a plurality of viewpoint images 601 taken and / or drawn from a plurality of viewpoints from one viewpoint to N viewpoints. The generation of the viewpoint image 601 includes capturing the object 8012 from different viewpoints using a plurality of cameras 801 and drawing using computer graphics.

<Central processing unit (1) 1011>
The central processing unit (part 1) 1011 performs a plurality of arithmetic processes from a plurality of viewpoint images 601 stored in the storage unit (part 1) 1012 to generate an intermediate image 401.

<Configuration of stereoscopic image generation apparatus>
1B includes a central processing unit (part 2) 2011 and a storage device (part 2) 2012.

<Central processing unit (2) 2011>
The central processing unit (part 2) 2011 stores the plurality of input intermediate images 401 in the storage unit (part 2) 2012 (frame buffer), converts the pixel arrangement, and generates a stereoscopic image 501.

<Configuration of stereoscopic image generation system>
1C includes a first information processing apparatus 3011 and a second information processing apparatus 3012. The first information processing apparatus 3011 includes a central processing unit (part 3) 30111, The second information processing device 3012 includes a central processing unit (part 4) 30121, a storage unit (part 4) 30122, a decompression unit 30123, and a receiving unit. 30124.

<Compression device 30113>
The compression device 30113 performs irreversible compression of the plurality of intermediate images 401 by a predetermined method. As a compression method, JPEG is used for still images, and MPEG-2, MPEG-4, etc. are used for moving images.

<Transmitter 30114>
The transmission device 30114 transmits the plurality of intermediate images 401 compressed by the compression device 30113 to the second information processing device 3012. As a transmission method, in addition to wired transmission via a USB port, wireless transmission such as optical communication, BLUETOOTH (registered trademark), and wireless LAN can be considered.

<Reception device 30124>
The reception device 30124 receives a plurality of intermediate images 401 transmitted by the transmission device 30114.

<Defroster 30123>
The decompressing device 30123 decompresses the plurality of intermediate images 401 compressed by the compressing device 30113.

<Intermediate Image Generation Process in Intermediate Image Generation Device>
A generation process of the intermediate image 401 executed by the intermediate image generation apparatus 101 will be described.

FIG. 2A is a flowchart of information processing executed by the intermediate image generation apparatus 101 in FIG.

<Step S201>
In FIG. 2A, first, a central processing unit (part 1) 1011 provided in the intermediate image generation apparatus 101 is operated by a user (an object 8012 by a camera 801 from a plurality of viewpoints from one viewpoint to N viewpoints). Storage device (part 1) provided in the intermediate image generation apparatus 101 with a plurality of viewpoint images 601 input in response to the shooting of the image or the drawing by computer graphics from a plurality of viewpoints from one viewpoint to N viewpoints) 1012 (step S201).

<Step S202>
Next, the central processing unit (part 1) 1011 determines whether or not there is an input of control information (step S202). The control information means a scanning method such as NTSC or PAL, a transmission method such as interlace or progressive, the number of viewpoints, resolution, pixel arrangement method, and the like. The control information is input by a user operation using a keyboard, a mouse, or the like further provided in the intermediate image generating apparatus 101. As a result, the format is determined.

<Step S203>
As a result of the determination in step S202, when control information is input, the central processing unit (part 1) 1011 generates a stereoscopic image 501 based on the control information (step S203). Here, the stereoscopic image 501 refers to an image having a stereoscopically visible subpixel arrangement that is finally presented to the user.

In the case of autostereoscopic viewing using a parallax barrier or the like, the RGB staircase-arranged pixel units 5011 arranged in three rows with the sub-pixels in contact with the corners in the diagonal direction from the first viewpoint to the Nth It is desirable to configure the stereoscopic image 501 by repeatedly arranging RGB staircase-arranged pixel blocks 5012 arranged in a continuous manner up to the viewpoint.

Here, it is a well-known technique that subpixels constituting one pixel may include C (cyan), M (magenta), Y (yellow), W (white), or other colors in addition to RGB. However, in this specification, the term “RGB” includes “RGBY”, “RGBW”, “RGBYW”, “RGBCMY”, and the like.

As a typical arrangement of the sub-pixels constituting the RGB staircase arrangement pixel unit 5011, for example, the one shown in FIG. The generation of the stereoscopic image 501 based on the control information is, for example, when the resolution of the display on which the stereoscopic image 501 is scheduled to be displayed is 1980 × 1080, that is, the stereoscopic image 501 having a resolution suitable for output, that is, A stereoscopic image 501 having a resolution of 1980 × 1080 is generated. The generation of the stereoscopic image 501 performed in step S203 will be described later.

<Step S204>
Next, the central processing unit (part 1) 1011 generates a plurality of intermediate images 401 from the generated stereoscopic image 501 (step S204). The intermediate image 401 is an image used to generate the stereoscopic image 501, and each of the plurality of intermediate images 401 includes R, G, B, and R subpixels that constitute the RGB staircase arranged pixel unit 5011 in the horizontal direction. In addition, RGB parallel arrangement pixel units 4011 arranged in order of other sub-pixels if necessary are arranged together for each of a plurality of viewpoints. In the present invention, a stereoscopic image 501 is generated or assumed in advance from a plurality of viewpoint images 601 photographed or drawn from a plurality of viewpoints, and an intermediate image 401 is generated based on the stereoscopic image 501. The generation of the intermediate image 401 performed in step S204 will be described in detail below as in step S203.

<Step S205>
Next, the central processing unit (part 1) 1011 stores the intermediate image 401 generated in step S204 in the storage unit (part 1) 1012 (step S205).

When the intermediate image 401 is stored in step S205, this process ends.

<Supplement of intermediate image generation device>
You may return to step S203 again after a process is complete | finished. For example, when the stereoscopic image 501 and the intermediate image 401 are repeatedly generated based on the same control information (when the viewpoint image 601 is input to the intermediate image generation apparatus 101 one after another), the control information of each time No input is required, and usability can be expected to improve. In this case, in order to cause the intermediate image generating apparatus 101 to recognize that the stereoscopic image 501 and the intermediate image 401 are continuously generated, the mode is changed by a specific operation such as a keyboard, and the stereoscopic image 501 and the intermediate image 401 are repeated. You may link with the process which produces | generates.

Note that, as illustrated in FIG. 2B, in step S204, the following processing may be performed assuming the stereoscopic image 501 without actually generating the stereoscopic image 501.

<Stereoscopic Image Generation Process in Stereoscopic Image Generation System>
A generation process of the stereoscopic image 501 executed by the stereoscopic image generation system 301 will be described.

FIG. 2C is a flowchart of information processing executed by the stereoscopic image generation system 301 in FIG.

2 (c) is basically the same as the processing up to step S204 in FIG. 2 (a), and redundant description of the same components is omitted, and only different parts are described below.

The first information processing apparatus 3011 and the second information processing apparatus 3012 include a compression apparatus 30113, a transmission apparatus 30114, a decompression apparatus 30123, and a reception apparatus 30124, respectively, in the intermediate image generation apparatus 101 used in FIG. An information processing apparatus further provided.

<Step S205>
First, the central processing unit (No. 3) 30111 of the first information processing device 3011 compresses the plurality of intermediate images 401 generated in step S204 by the compression device 30113 (step S205).

<Step S206>
Next, the plurality of intermediate images 401 compressed in step S205 are stored in the storage device (part 3) 30112 (step S206).

<Step S207>
Next, (Part 3) 30111 transmits a plurality of intermediate images 401 that have been compressed and stored from the transmission device 30114 to the second information processing device 3012 (step S207).

<Step S208>
The central processing unit (part 4) 30121 of the second information processing device 3012 receives the plurality of intermediate images 401 transmitted from the first information processing device 3011 in step S207 by the reception device 30124 (step S208).

<Step S209>
Next, the central processing unit (part 4) 30121 decompresses the received plurality of intermediate images 401 by the decompression device 30123 (step S209).

<Step S210>
Next, the central processing unit (part 4) 30121 generates a stereoscopic image 501 that is finally output to the user from the plurality of decompressed intermediate images 401 (step S210). The stereoscopic image 501 in step S210 and the stereoscopic image 501 in step S203 are the same. The generation of the intermediate image 401 performed in step S210 will be described in detail below in the same manner as in steps S203 and S204.

When the three-dimensional image 501 is generated in step S210, this process ends.

<Supplement of stereoscopic image generation system>
After the processing is completed, the generated intermediate image 401 may be output to the stereoscopic image display device 701. Further, when the intermediate image 401 is continuously transmitted from the first information processing apparatus 3011, the processing from step S208 to step S210 may be performed continuously. According to the system in the illustrated example (c), for example, a plurality of cameras 801 are installed at any point and shooting is performed continuously, and a plurality of images generated one after another by the first information processing apparatus 3011 are performed. An intermediate image 401 in which images for each viewpoint are arranged in a tile shape can be distributed all over the world in an existing format so that many users can view the stereoscopic image 501 simultaneously and in real time.

That is, the first information processing apparatus 3011 is equipped with an expensive graphic board, a high-speed CPU, or the like in order to generate a plurality of intermediate images 401 in real time, and a plurality of second information used by the user. If a relatively low-speed CPU is installed in the processing device 3012, an unprecedented stereoscopic image viewing environment utilizing the characteristics of a plurality of intermediate images 401 capable of generating a stereoscopic image 501 only by changing the arrangement of pixels. This can be realized with existing formats. That is, the generation of a compressible intermediate image 401 and the formation of such an intermediate image 401 with the minimum necessary resolution in which the total number of pixels constituting the image for each viewpoint is the same as the number of pixels of the stereoscopic image 501. A stereoscopic image 501 that can be stereoscopically viewed only by subpixel rearrangement (mapping) can be generated at high speed without using a special computer.

Embodiments of a method for generating a plurality of intermediate images 401 according to the present invention will be described with reference to FIGS.

FIG. 3 shows an example in which a certain object 8012 is photographed by a camera 801 from a plurality of different viewpoints, and a viewpoint image 601 for each viewpoint is generated. Since the gazing point 8011 is captured from the six viewpoints using the camera 801, six viewpoint images 601 are obtained. Note that the resolution of the viewpoint image 601 at this time is arbitrary. Note that the cameras 801 may be arranged horizontally so that the optical axes of the respective cameras 801 are directed toward the gazing point 8011.

In general, the standard of image signals that are widely used expresses colors with three primary colors of RGB. However, when the three-dimensional image display device 701 expresses colors with the four primary colors of RGBY, or when expressed with five or more primary colors, Japanese Patent Laid-Open No. 2001-2001 306023 (Seiko Epson Corporation), Special Table 2004-529396 (Genoa Technologies Limited, etc.), Japanese Patent Application Laid-Open No. 2001-209047 (Sharp Corporation), etc. Performs conversion to a signal of more than four primary colors.

FIG. 4 shows an example of generating a stereoscopic image 501 having a sub-pixel arrangement for finally outputting on the display from the plurality of viewpoint images 601 taken in FIG. In the illustrated example, an RGB value of the RGB staircase-arranged pixel unit 5011 is obtained by interpolating from the RGB values of sub-pixels constituting at least one or more pixel units arranged in the vicinity of the corresponding position.

Hereinafter, the generation of the stereoscopic image performed in step S203 of FIG. 2 will be described in detail with reference to FIG.

When generating the stereoscopic image 501, the central processing unit (part 1) 1011 described above first determines the sub-pixel arrangement of the stereoscopic image 501 according to the resolution of the display that performs the final output input in step S202 of FIG. For example, the sub-pixel arrangement of the stereoscopic image 501 is the one shown in FIG. In the illustrated example, when the sub-pixel is a vertically long rectangle with a ratio of 1: 3, three sub-pixels that are most appropriately stereoscopically viewable are arranged in 3 rows and 1 column in contact with the corner in an oblique direction. A stereoscopic image 501 having a subpixel arrangement is assumed.

Next, an RGB staircase arranged pixel block 5012 corresponding to the number of viewpoints as shown in FIG. The RGB staircase-arranged pixel block 5012 is an RGB staircase-arranged pixel unit 5011 in which subpixels are arranged in 3 rows with diagonal corners in contact with corners and arranged in a horizontal direction from 1 viewpoint to N viewpoints. Point to. In the example (a) illustrated in FIG. 5, since the object 8012 is photographed from six viewpoints in FIG. 3, a group of pixels composed of 18 subpixels is an RGB staircase arranged pixel block 5012.

Next, the RGB staircase arrangement pixel block 5012 is repeatedly arranged to form the stereoscopic image 501, and the RGB values of the sub-pixels constituting the RGB staircase arrangement pixel unit 5011 are acquired. The acquisition of RGB values is desirably performed based on any of the RGB staircase-arranged pixel units 5011 in the RGB staircase-arranged pixel block 5012, and the coordinate values on the stereoscopic image 501 of the subpixels constituting the RGB staircase-arranged pixel unit 5011. Are obtained from the RGB values of the sub-pixels arranged at the coordinate values on the viewpoint image 601 corresponding thereto. As shown in FIG. 5 (a), the coordinate values on the stereoscopic image 501 have a horizontal axis on the stereoscopic image 501 as U and a vertical axis as V in the sub-pixel coordinate system. The subpixel arranged at the uppermost stage of the first viewpoint can be expressed as (U, V) as shown in the figure.

Next, the corresponding coordinate value of the viewpoint image 601 taken from the first viewpoint is calculated.

The coordinate values on the viewpoint image 601 are x as the horizontal axis and y as the vertical axis in the pixel coordinate system, and the coordinate values of the pixels on the viewpoint image 601 of the first viewpoint as a reference in this embodiment are as shown in the figure. (X, y).

In general, the stereoscopic image 501 and the plurality of viewpoint images 601 have different numbers of subpixels, and the stereoscopic image 501 uses a subpixel coordinate system and the viewpoint image 601 uses a pixel coordinate system. The following conversion formula is required.

Therefore, the total number of sub-pixels constituting the stereoscopic image 501 in the horizontal direction is W, the total number in the vertical direction is H, the total number of pixels constituting the viewpoint image 601 in the first viewpoint in the horizontal direction is a, and the total number in the vertical direction is b. The conversion from the sub-pixel coordinate system to the pixel coordinate system is

Can be obtained.

By representing the coordinate values on the stereoscopic image 501 in the sub-pixel coordinate system, RGB values can be obtained for each sub-pixel unit, and a higher-definition stereoscopic image 501 is generated than when the RGB value is obtained for each pixel unit. can do.

At this time, it is desirable to set a and b so that W: H = a: b. Of course, this is not the case when it is desired to display the viewpoint image 601 as much as possible even if it is deformed.

However, as shown in FIG. 5C, since only the center point of each pixel is defined in the pixel coordinate system, for example, when (x, y) is at the position shown in the figure, The RGB value of the subpixel cannot be obtained directly. Therefore, it is obtained by interpolation from the pixel unit arranged in the vicinity of the pixel to which the (x, y) belongs.
First, α = x−x1, β = x2−x, γ = y−y1, δ = y3−y,
If the RGB values of P1 to P4 that are the center points of the pixels of the viewpoint image 601 are C1 to C4, respectively, the RGB values of P are expressed by the following equation by linear interpolation.

Here, it is only necessary to obtain RGB values of only one of R, G, and B indicated by the subpixels of the stereoscopic image 501 to be calculated. If α + β = γ + δ = 1,

It becomes.

Next, the RGB values of other sub-pixels constituting the same RGB staircase-arranged pixel unit 5011 are similarly obtained by interpolation from neighboring pixel units. There are various interpolation methods, and any interpolation method may be used as long as it is an appropriate interpolation method.

Next, the RGB values of the sub-pixels constituting the RGB staircase arranged pixel unit 5011 other than the RGB staircase arranged pixel unit 5011 of the first viewpoint as a reference are calculated. In this case, it is desirable to calculate based on the coordinate values of the sub-pixels constituting the RGB staircase arrangement pixel unit 5011 as a reference. That is, for example, when generating a stereoscopic image consisting of 6 viewpoints using a parallax barrier, images from multiple viewpoints that are viewed from the same slit or hole by the image presentation target person need to correspond to each other. The RGB values are calculated from the pixels of the other viewpoint image 601 at the same position as the coordinate values on the viewpoint image 601 corresponding to the coordinate values of the sub-pixels constituting the RGB staircase arranged pixel unit 5011. The correspondence between the plurality of viewpoint images 601 and the stereoscopic image 501 is, for example, as illustrated in FIG.

As described above, the RGB values of the sub-pixels constituting the stereoscopic image 501 are acquired by interpolation calculation, and the stereoscopic image 501 is generated or assumed. When the stereoscopic image 501 is assumed, since the intermediate image 401 is directly generated (step S204) without generating the stereoscopic image 501 from the plurality of viewpoint images 601, interpolation calculation for acquiring RGB values, It is necessary to rearrange the sub-pixels constituting the RGB staircase arrangement pixel unit 5011 of the stereoscopic image 501 obtained by the interpolation calculation to obtain the RGB parallel arrangement pixel unit 4011 of the plurality of intermediate images 401.

In this way, by obtaining the RGB values of other subpixels based on any of the subpixels constituting the RGB staircase-arranged pixel unit 5011, a sharp stereoscopic image 501 based on reality can be expressed. In addition, when the RGB values of all the subpixels are obtained without determining the representative points, a stereoscopic image 501 with a smooth change of viewpoint can be obtained. It is desirable to change these appropriately according to the situation and purpose of presenting the stereoscopic image 501.

Next, generation of the intermediate image 401 performed in step S204 will be described in detail.

FIG. 6 is a diagram illustrating an example of generating a plurality of intermediate images 401 from the stereoscopic image 501.

In the illustrated example, a plurality of RGB parallel arrangement pixel units 4011 in which subpixels constituting the RGB staircase arrangement pixel unit 5011 of the stereoscopic image 501 generated in FIG. 4 are arranged in the horizontal order in the order of R, G, B are provided. The six intermediate images 401 are generated by arranging them for each viewpoint.

The RGB parallel arrangement pixel unit 4011 arranges the sub-pixels peeking from the slits arranged in a staircase shape or the holes arranged in a staircase shape as shown in the figure. This is performed for all the RGB staircase-arranged pixel units 5011. It should be noted that the generation of the plurality of intermediate images 401 from the stereoscopic image 501 is desirably performed using the intermediate image generation table 4013.

FIG. 7 is a diagram illustrating the intermediate image generation table 4013.

FIG. 7A shows which viewpoint image the subpixel on the stereoscopic image 501 displays, which of R, G, and B is displayed, and the subpixel coordinates on the stereoscopic image 501. This is a table showing coordinate values in the system. For example, the sub-pixel at the upper left corner is (1I1) because it is located in the first column of the first row counting from the upper left corner.

FIG. 7B shows at which point the subpixels constituting the intermediate image 401 correspond to the subpixels arranged in the subpixel coordinate system on the stereoscopic image 501. For example, the subpixel at the upper left corner of the first viewpoint corresponds to the subpixels of 1, (1I1), R located at the upper left corner of the stereoscopic image 501, and the subpixel is on the intermediate image 401. Will be placed. Similarly, the sub-pixel at the upper left corner of the intermediate image 401 at the second viewpoint corresponds to the sub-pixel at the position (2C1) on the stereoscopic image 501, and one column in the second row on the stereoscopic image 501. Sub-pixels arranged in the eyes are arranged on the intermediate image 401. In addition, since the RGB staircase-arranged pixel unit 5011 having the sub-pixel does not have a sub-pixel having a B value, the RGB parallel-arranged pixel unit constituting the intermediate image 401 disposed at the upper left corner of the second viewpoint Similarly, a sub-pixel having a B value is not arranged in 4011.

In this way, the RGB staircase-arranged pixel unit 5011 from the first viewpoint to the sixth viewpoint is rearranged in the horizontal direction to form the RGB parallel-arranged pixel unit 4011, and the RGB staircase-arranged pixel located at the upper left corner When the arrangement change of the block 5012 is completed, the arrangement of subpixels constituting the adjacent RGB staircase arrangement pixel block 5012 is also changed as shown in the figure.

As described above, the intermediate image that associates the position of the sub-pixel constituting the RGB staircase-arranged pixel unit 5011 of the stereoscopic image 501 with the position of the sub-pixel constituting the RGB parallel-arranged pixel unit 4011 of the intermediate image 401 for each of a plurality of viewpoints. If the generation table 4013 is created in advance, it is possible to generate the intermediate image 401 for each viewpoint by simply changing the arrangement of the subpixels constituting the stereoscopic image 501 without requiring complicated calculation processing for interpolation. it can.

The intermediate image generation table 4013 is preferably stored in the storage device (part 1) 1012 of the intermediate image generation apparatus 101. As a result, when the stereoscopic image 501 is generated using the intermediate image generation apparatus 101, the table can be used as the stereoscopic image generation table 5013 without creating a table again.

<Tile format>
FIG. 8 is a diagram showing an example of the arrangement of the image frames 4012 of the intermediate image 401 that is particularly desirable in the embodiment according to the present invention. In the image frame 4012, an intermediate image 401 for each viewpoint includes, for example, a first viewpoint image in the first column of the first row, a second viewpoint image in the first column of the second row, and a first row of the third row. The third viewpoint image in the column, the fourth viewpoint image in the second column of the first row, the fifth viewpoint image in the second column of the second row, and the sixth viewpoint image in the second column of the third row. Are arranged in tiles.

As a result, the total number of subpixels constituting the stereoscopic image 501 and the total number of subpixels on the image frame 4012 in which the intermediate image 401 is arranged in a tile shape are the same both vertically and horizontally, and there is no useless pixel. Since the pixels for each viewpoint are arranged together, there is no interference between different viewpoints, and irreversible compression can be used. Since the resolution and aspect ratio of the image frame 4012 in which the intermediate image 401 is arranged in a tile shape and the autostereoscopic display (stereoscopic image 501) are the same, a standard video reproduction device such as Blu-ray or STB or video A practical 3D image display system that can easily generate a 3D image 501 by a 3D image generation device (converter) from an intermediate image 401 output or transmitted in a standard format by a distribution server at a very low price. Can be provided.

FIG. 9 is a diagram showing an example in which a plurality of viewpoint images 601 are arranged as they are in a tile-shaped image frame 4012, and is a diagram for comparing the intermediate image 401 with the image frame 4012 described above.

When the resolution of the display that finally outputs is 16: 9, the aspect ratio of the viewpoint image 601 for one viewpoint in FIG. 9A is also 16: 9, and the entire image frame 4012 is 32:27.

On the other hand, when a plurality of intermediate images 401 are arranged in the tile-shaped image frame 4012 (FIG. 9B), in the stereoscopic image 501, three rows of sub-pixels constituting the RGB staircase arrangement pixel unit 5011 are arranged in the horizontal direction. Since the RGB parallel arrangement pixel units 4011 are arranged side by side, the total number of pixels in the vertical direction is 1/3. Furthermore, when the number of viewpoints is 6, for example, the total number of pixels in the horizontal direction is

It becomes. Assuming that the vertical and horizontal directions of the stereoscopic image 501 are H and W, respectively, when the intermediate image 401 is arranged in an image frame 4012 having tiles of 3 rows × 2 columns as in the example shown in FIG.

In the horizontal direction,

Thus, the image frame 4012 having the same resolution in both the vertical and horizontal directions as the stereoscopic image 501 can be generated. If a plurality of viewpoint images 601 are arranged as they are in a tile-shaped image frame 4012 as in the prior art so that the aspect ratio and resolution of the stereoscopic image 501 are the same, paste margins are provided on both sides of the image for each viewpoint. In addition, the aspect ratio must be the same, and the resolution of the images at each viewpoint must be reduced so that the resolutions are the same when arranged in tiles. As a result, the viewpoint image 601 for generating the stereoscopic image 501 has a low resolution, and the image quality of the stereoscopic image 501 is significantly reduced. On the other hand, in the present invention, since the intermediate image 401 is generated from the high-quality viewpoint images 601, it is possible to completely maintain the resolution that is indispensable for generating a stereoscopic image.

FIG. 10 is a diagram illustrating an example of an image frame 4012 including a plurality of intermediate images 401. An image frame 4012 when a stereoscopic image 501 is generated from two viewpoints to five viewpoints and seven viewpoints to eleven viewpoints is shown, and if the pixels of each viewpoint are arranged in a tile shape in the frame, the aspect ratio with the stereoscopic image 501 is shown. Can create the same image file.

That is, when the stereoscopic image 501 is composed of two viewpoints, 2/3 of the one-view intermediate image 401 is displayed on the first row tile, and one viewpoint intermediate image 401 is displayed on the second row first tile. 1/3 of the two-viewpoint intermediate image 401 is placed on the second tile connected to 1/3, and 2/3 of the two-viewpoint intermediate image 401 is placed on the third row tile. Further, when the stereoscopic image 501 is composed of three viewpoints, the intermediate image 401 of each viewpoint is arranged on each row tile. Also, when the stereoscopic image 501 is composed of 4 to 6 viewpoints, the 1st to 3rd viewpoint intermediate images 401 and the 1st to 3rd viewpoint intermediate images 401 are connected to the first tile of each line from the first line to the 2nd viewpoint. The intermediate images 401 of the remaining viewpoints are arranged on the tiles arranged in three rows. When the stereoscopic image 501 is composed of 7 to 9 viewpoints, the first to third rows connected to the first to third viewpoint intermediate images 401 and the first to third viewpoint intermediate images 401 on the first tile of each row. The intermediate images 401 with 4 to 6 viewpoints are arranged on the tiles No. 4 and the intermediate images 401 with the remaining viewpoints are arranged on the tiles arranged in the first to third rows connected to the intermediate images 401 with the 4 to 6 viewpoints. Similarly, even when the stereoscopic image 501 is composed of 10 or more viewpoints, a part or all of the intermediate image 401 is arranged on the tiles in each row in order from one viewpoint.

Thereby, it is possible to generate an image frame 4012 having the same resolution as that of the stereoscopic image 501 in both the vertical and horizontal directions.

<Belt format>
In addition to the tile format, the intermediate image 401 is divided and arranged in the horizontal direction by the number of viewpoints. A belt-like format may be used.

A particularly preferable combination in the relationship between the sub-pixels constituting one pixel of the stereoscopic image display device 701 and the number of viewpoints of the intermediate image 401 in the belt-like format will be described.

<In the case of RGB>
When the sub-pixels constituting one pixel are composed of three colors of R, G, and B, the number of viewpoints of the intermediate image 401 is 4 viewpoints (see FIG. 12), 7 viewpoints (see FIG. 13), 10 viewpoints, 12 viewpoints, etc. , “3n + 1 (n is a natural number)” viewpoint is preferable.

By setting the number of viewpoints to 3n + 1, the effect on the pixels due to irreversible compression occurs only at both ends of each viewpoint image, so that the stereoscopic image 501 is displayed more clearly.

<RGB + 1>
The sub-pixels constituting one pixel are composed of four colors including R, G and B plus one other color (C (cyan), M (magenta), Y (yellow), W (white), etc.)) In this case, the number of viewpoints of the intermediate image 401 is preferably “4n + 1 (n is a natural number)” viewpoints such as 5 viewpoints (see FIG. 14), 9 viewpoints (see FIG. 15), and 13 viewpoints.

By setting the number of viewpoints to 4n + 1, the effect on the pixels due to irreversible compression occurs only at both ends of each viewpoint image, so that the stereoscopic image 501 is displayed more clearly.

<In the case of RGB + m>
When the sub-pixels constituting one pixel are composed of 3 + m colors obtained by adding the other m colors to R, G, and B, the number of viewpoints of the intermediate image 401 is “(3 + m) n + 1 (n is a natural number). It is preferable to take a viewpoint.

By setting the number of viewpoints to (3 + m) n + 1, the irreversible compression affects the pixels only at both ends of each viewpoint image, so that the stereoscopic image 501 is displayed more clearly.

<RGB staircase arrangement pixel unit>
FIG. 16 is a diagram illustrating an arrangement example of the RGB staircase arrangement pixel unit 5011.

FIG. 16A is a diagram showing an example of an RGB staircase-arranged pixel unit 5011 in which subpixels constituting a pixel unit of a viewpoint image 601 from 1 viewpoint to 6 viewpoints are arranged in a staircase in contact with a corner in an oblique direction. It is composed of 3 subpixels in 3 rows and 1 column. When the stereoscopic image 501 having the RGB staircase-arranged pixel unit 5011 is converted into the intermediate image 401, the RGB parallel-arranged pixel unit 4011 is configured by arranging the sub-pixels constituting the RGB staircase-arranged pixel unit 5011 in the horizontal direction. Next, the RGB staircase-arranged pixel unit 5011 of the same viewpoint arranged in a staircase shape in contact with the RGB staircase-arranged pixel unit 5011 at the corner is also arranged in the horizontal direction as shown in FIG. Then, the intermediate image 401 is configured by arranging the RGB parallel arranged pixel units 4011 in a stepwise manner in contact with each other at a corner in an oblique direction.

FIG. 16B is a diagram illustrating an example of an RGB staircase-arranged pixel unit 5011 configured by 6 subpixels in 3 rows and 2 columns. When the stereoscopic image 501 having the RGB staircase-arranged pixel unit 5011 is converted into the intermediate image 401, a collection of subpixels in the first column constituting the RGB staircase-arranged pixel unit 5011 is first arranged in the horizontal direction, and the RGB parallel-arranged pixel unit 4011 is first arranged. And the group of subpixels in the second column is arranged so as to be further connected to the RGB parallel arrangement pixel unit 4011 in the horizontal direction. Next, as for the RGB staircase-arranged pixel unit 5011 of the same viewpoint arranged in a staircase in contact with the RGB staircase-arranged pixel unit 5011 at the corner, as shown in FIG. The RGB parallel-arranged pixel unit 4011 is arranged side by side, and a collection of subpixels in the second column is arranged so as to be further connected to the RGB parallel-arranged pixel unit 4011 in the horizontal direction.

FIG. 16C is a diagram illustrating an example of an RGB staircase-arranged pixel unit 5011 configured by nine subpixels in three rows and three columns. Similarly, when converting the stereoscopic image 501 having the RGB staircase-arranged pixel unit 5011 into the intermediate image 401, the collection of subpixels in the first column constituting the RGB staircase-arranged pixel unit 5011 is first arranged in the horizontal direction in parallel with RGB. An arrangement pixel unit 4011 is configured, and a group of subpixels constituting the RGB staircase arrangement pixel unit 5011 in the second column is arranged so as to be connected to the RGB parallel arrangement pixel unit 4011 in the horizontal direction. A group of sub-pixels constituting the RGB staircase-arranged pixel unit 5011 in the third column is arranged so as to be connected to the parallel-arranged pixel unit 4011 in the horizontal direction.

In the arrangement example of the RGB staircase arrangement pixel unit 5011 shown in FIG. 16, it is desirable to acquire RGB values from a plurality of viewpoint images 601 for each subpixel as shown in FIG. This prevents a decrease in resolution (for example, in FIG. 16B, the horizontal resolution is halved compared to FIG. 16A) due to the calculation and acquisition of RGB values in pixel units. And a clear stereoscopic image 501 can be provided to the user. The subpixel is usually a vertically long rectangle at a ratio of 1: 3, but the shape of the subpixel is a circle, a circle, a V shape, or a shape obtained by rotating W by 90 degrees (see FIG. There are various shapes such as those shown in Fig. 56). Depending on the shape, the RGB staircase-arranged pixel unit 5011 in which three sub-pixels are arranged in a row may not be easily visible. In that case, a mask with a wide slit and hole width of the parallax barrier 704 for peeking at the stereoscopic image 501 and an arrangement interval thereof is created, and accordingly, three subpixels are displayed so that a stereoscopic image can be displayed appropriately. It is desirable to generate RGB staircase-arranged pixel units 5011 arranged in two rows as shown in (b) or three rows as shown in (c).

However, if the original resolution is high, a sufficient resolution for visually recognizing the stereoscopic image 501 is ensured even if the resolution is reduced. It is good also as arrange | positioning in a direction (FIG. 59). In this case, the slit of the parallax barrier 704 is provided in the vertical direction.

Also preferably, RGB subpixels are arranged in the vertical direction.

FIG. 60 is a diagram illustrating how the stereoscopic image 501 in which RGB subpixels are arranged in the horizontal direction is viewed in the horizontal direction.

From the first viewpoint 708, the GB sub-pixel is visually recognized through the parallax barrier 704 having a vertical slit, and the R sub-pixel is lost, so that the stereoscopic image 501 is not properly viewed.

A set of RGB sub-pixels is visually recognized from the second viewpoint 709 located farther from the stereoscopic image display device 701 than the first viewpoint 708. Therefore, the stereoscopic image 501 is properly viewed from the second viewpoint 709.

In addition to the set of RGB subpixels, the R subpixel is further visually recognized from the third viewpoint 710 located farther from the stereoscopic image display device 701 than the second viewpoint. Therefore, the stereoscopic image 501 viewed from the third viewpoint 710 is visually recognized as reddish, and is not properly viewed.

In this regard, as shown in FIG. 61, if a set of RGB sub-pixels is arranged in the vertical direction, at least one set of R, G, B sub-pixels is always visible from the horizontal direction without omission. The three-dimensional image 501 is properly viewed from the point.

Next, the generation of the stereoscopic image 501 that is finally output from the intermediate image 401 for each of a plurality of viewpoints performed in step S210 in FIG. 2 will be described in detail.

The stereoscopic image 501 is generated by subpixels constituting the RGB parallel arrangement pixel unit 4011 of the intermediate image 401 transmitted from the optical disk such as BLU-RAY DISC (registered trademark), the server, or the first information processing apparatus described above. This is performed by changing the arrangement of the above to the arrangement for stereoscopic viewing. That is, the subpixels constituting the RGB parallel arrangement pixel unit 4011 are rearranged in a staircase shape again to constitute the RGB staircase arrangement pixel unit 5011. In this case, a stereoscopic image relating the positions of these subpixels is performed. It is desirable to use the generation table 5013. Since the intermediate image generation table 4013 shown in FIG. 7 associates the positions of the subpixels constituting the RGB parallel arrangement pixel unit 4011 with the positions of the subpixels constituting the RGB staircase arrangement pixel unit 5011, the three-dimensional image generation is performed. The table 5013 can be used.

That is, the RGB staircase arrangement pixel unit 5011 can be generated again in accordance with the reverse order of the arrangement rules shown in the intermediate image generation table 4013 in FIG.

As another example, when generating the stereoscopic image 501 from the intermediate image 401 in the tile format, a stereoscopic image generation table 5013 shown in FIG. 62 may be used.

62 will be described in detail. The “8-viewpoint tile format” in the upper part of FIG. 62 is the intermediate image 401, and the “8-viewpoint parallax stereoscopic image” indicates the stereoscopic image 501. The “R coordinate conversion table”, “G coordinate conversion table”, “B coordinate conversion table”, and “Y coordinate conversion table” in the lower part of FIG.

When generating the stereoscopic image 501, the R value of the pixel coordinates (for example, (x _i , y _j ) of the first viewpoint) of the intermediate image 401 is acquired with reference to the stereoscopic image generation table 5013, Rendering is performed as an R value of the pixel coordinates (X _m , Y _n ) of the associated stereoscopic image 501. By performing this process for all the coordinates of each viewpoint, drawing of the R subpixels on the stereoscopic image 501 is completed. Similar processing is performed for the G subpixel, the B subpixel, and the Y subpixel, and the generation of the stereoscopic image 501 is completed.

When the stereoscopic image 501 is generated from the intermediate image 401 in the belt-like format, a stereoscopic image generation table 5013 shown in FIG. 63 is used.

63 will be described in detail. The “4-view belt format” in the upper part of FIG. 62 is the intermediate image 401, and the “4-view parallax stereoscopic image” indicates the stereoscopic image 501. The “R coordinate conversion table”, “G coordinate conversion table”, and “B coordinate conversion table” in the lower part of FIG.

The method of generating the stereoscopic image 501 is the same as that in the case of the 8-view tile format shown in FIG. That is, the method for generating the stereoscopic image 501 using the stereoscopic image generation table 5013 does not depend on the number of viewpoints and the format of the intermediate image 401, and can be implemented with appropriate changes.

The stereoscopic image generation table 5013 shown in FIGS. 62 and 63 is characterized in that it is created separately for each of R, G, B, and (Y) sub-pixels, and the memory access size can be reduced. By mapping all R values in a table and using them sequentially with G, B, (Y), it becomes easier to expand each table on the main memory, reducing the capacity of the main memory, and improving the processing speed. Can do.

<Stereoscopic image generation device>
FIG. 17 is an external view showing an example of an embodiment of a stereoscopic image generating apparatus 201 according to the present invention.

As shown in the figure, a stereoscopic image generating apparatus 201 according to this embodiment is electrically connected by a video cable 702 between a general image output apparatus 1001 and a general stereoscopic image display apparatus 701 (display). Connected to and used to receive images of a plurality of viewpoints transmitted as image signals (image input signals) from the image output apparatus 1001, and control information (scanning method, number of viewpoints, resolution, pixel arrangement method, etc.) set in advance Information) is converted into a pixel arrangement for stereoscopic image display, and the image after the pixel arrangement conversion is transmitted to the stereoscopic image display device 701 as an image signal (image output signal).

Here, the video cable 702 that electrically connects the stereoscopic image generation apparatus 201, the image output apparatus 1001, and the stereoscopic image display apparatus 701 is specifically the 3D image output apparatus 1001 and the stereoscopic output of the standard such as VDI and HMVI. As a cable for electrically connecting the image display device 701 and transmitting an image signal, a cable that has been widely used can be used.

FIG. 18 is an external view showing an example of another embodiment of the stereoscopic image generating apparatus 201 according to the present invention.

As shown in the figure, the stereoscopic image generating apparatus 201 receives a control signal by further electrically connecting either one or both of the image output apparatus 1001 and the stereoscopic image display apparatus 701 with a control cable 703. May be. The control signal means a signal that gives control information other than an image such as a scanning method, a resolution, the number of viewpoints, and a pixel arrangement method to the stereoscopic image generating apparatus 201.

Here, specifically, the control cable 703 is a widely used control cable 703 that electrically connects the image output device 1001 of the standard such as i / LINK or serial and the stereoscopic image display device 701. be able to.

However, for convenience of explanation, the video cable 702 and the control cable 703 have been described as separate cables in the figure, but these cables may be bundled to form one cable.

The transmission of the intermediate image 401 (image signal) and the transmission of the control signal are those that have been widely used as standard wireless communication means such as wireless LAN, Bluetooth (registered trademark), UWB instead of the cable. It may be used.

In view of the gist of the present invention, it is most preferable that the image output apparatus 1001 uses the existing image output technology as it is. In other words, the image output apparatus 1001 is an existing set-top box that acquires moving images by terrestrial, satellite broadcasting, streaming from the Internet, or download, or a stand-alone DVD player or Blu-ray (registered trademark) player. It is preferable to use a playback device such as a device having a recording function.

Further, it is most preferable in view of the gist of the present invention that the stereoscopic image display device 701 (display) be used as it is without any improvement on the existing stereoscopic image display device. That is, the stereoscopic image display device 701 is an autostereoscopic image display device such as a liquid crystal display, a plasma display, or an organic EL display that employs an existing parallax barrier method or lenticular method, or a two-viewpoint image at high speed. It is preferable that they are displayed alternately and viewed through shutter-type glasses. However, it goes without saying that the stereoscopic image generating apparatus 201 according to the present invention can be used in addition to the above-described stereoscopic image display apparatus.

FIG. 19 is a diagram showing an example of a multi-viewpoint image received as an image signal by the stereoscopic image generating apparatus 201 according to the present invention. This is a recommended standardized tile format for multi-viewpoint images according to the present invention, in which only pixels to be read out from multi-viewpoint images are arranged for conversion to a pixel arrangement for stereoscopic image display. Usually, a predetermined compressed file is generated after being arranged in a tile format. The resolution is arbitrary, and usually the compression standard MPG2, which is lossy compression, is often used. Although not shown, an intermediate image 401 may be generated in a tile format corresponding to an arbitrary number of viewpoints based on the present invention to form a multi-viewpoint image. In particular, in a pixel arrangement method in which three sub-pixels constituting RGB of one pixel are arranged diagonally in 3 rows and 3 columns, an image having an arbitrary resolution with a 16: 9 aspect ratio is set to horizontal 960 pixels and each vertical 360 pixels. It is desirable to create an intermediate image 401 of the viewpoint and convert it to a pixel arrangement for stereoscopic image display. As a result, the resolution of the six-view tile image is 1920 × 1080, and the tile image can be received as a high-definition image and converted into a stereoscopic image with the least image quality loss.

FIG. 20 is a diagram showing an example of a moving image of a plurality of viewpoints received as a moving image signal by the stereoscopic image generating apparatus 201 according to the present invention. Such a moving image is well known and can be created by multi-streaming of the compression standard MPG4. A plurality of moving images synchronized with one file can be recorded. The received intermediate image 401 from one viewpoint to N viewpoints is stored in a storage unit in a predetermined arrangement, and converted into a pixel arrangement for stereoscopic image display.

FIG. 21 is a diagram showing an example of a moving image of a plurality of viewpoints received as a moving image signal by the stereoscopic image generating apparatus 201 according to the present invention. The moving images of multiple viewpoints are repeatedly formed in the time direction by assigning the moving images of the respective viewpoints to each successive frame. The received intermediate images 401 from one viewpoint to N viewpoints are sequentially stored in the storage means and converted into a pixel arrangement for stereoscopic image display.

FIG. 22 is a diagram showing a first example of a multi-viewpoint intermediate image 401 in which pixel information 4014 is embedded, which is received as an image signal by the stereoscopic image generating apparatus 201 according to the present invention. The pixel information is a predetermined encryption and defines information such as 2D image distinction, resolution, and the number of viewpoints. It is embedded in order to notify a stereoscopic image generation device or a converter (stereoscopic image generation device) whether a stereoscopic image or a normal image.

(A) in the figure is a diagram showing an embedding position of the pixel information 4014 on the image. According to FIG. 5A, the pixel information 4014 is embedded in the upper left corner of the image. However, since the embedding position of the pixel information 4014 is based on a predefined arrangement pattern, it is not always necessary to be at the upper left end, but the end of the image is an image output device connected to the stereoscopic image generating device 201. Since it is a portion that overlaps the monitor frame 1001 and is invisible to the user, there is an advantage that even if the pixel information 4014 is embedded, the display of the stereoscopic image to the user is not affected.

FIG. 4B is an enlarged view showing embedded pixel information 4014. According to FIG. 5B, the pixel information 4014 is embedded in the horizontal line without any gap. However, although not shown, it may be embedded at a predetermined interval.

FIG. 23 is a diagram showing a second example of the multi-viewpoint intermediate image 401 in which the pixel information 4014 is embedded, which is received by the stereoscopic image generating apparatus 201 according to the present invention as an image signal.

(A) in the figure is a diagram showing an embedding position of the pixel information 4014 on the image.

FIG. 5B is an enlarged view of a portion where pixel information 4014 is embedded. In the second embodiment, a pixel matrix 4015 in which a plurality of pixel information 4014 defined as the same image information is continuously arranged in the XY direction is embedded.

FIG. 10C is an enlarged view showing one of the pixel matrices 4015 in FIG. A 3 × 3 matrix surrounded by a central thick frame is a pixel matrix 4015, and nine pieces of pixel information Cm · n in which the same image information is defined are arranged. In the stereoscopic image generating apparatus 201 according to the present invention, image information is analyzed from pixel information 4014 at the center of the pixel matrix 4015 indicated by a circle. Note that it is appropriate to specify the position of the pixel information 4014 at the center of the pixel matrix 4015 by specifying the XY coordinates of the pixel information 4014 based on a predefined arrangement pattern. However, the image information may be obtained from an average value of a plurality of pieces of pixel information 4014 in the pixel matrix 4015.

FIG. 24 is a diagram showing a third embodiment of a multi-viewpoint intermediate image 401 in which pixel information 4014 is embedded, which is received as an image signal by the stereoscopic image generating apparatus 201 according to the present invention.

FIG. 5B is an enlarged view of a portion where pixel information 4014 is embedded.

FIG. 10C is an enlarged view showing one of the pixel matrices 4015 in FIG. In the third embodiment, pixel information 4014 defined as image information is arranged at the center of the pixel matrix 4015, and the pixel matrix 4015 has an intermediate portion between the pixel adjacent to the pixel matrix 4015 and the pixel information 4014. Value pixel information 4014 is embedded.

FIG. 25 is a diagram showing a fourth example of the multi-viewpoint intermediate image 401 embedded with the pixel information 4014 received by the stereoscopic image generating apparatus 201 according to the present invention as an image signal.

FIG. 5B is an enlarged view of a portion where pixel information 4014 is embedded. In the fourth embodiment, the pixel matrix 4015 is 2 × 3, and compared with the pixel matrix 4015 of the third embodiment, the pixel information 4014 in the upper row is removed and arranged at the upper end of the image. Yes. This embodiment is suitable because the influence on the image is reduced when the area occupied by the pixel matrix 4015 is reduced.

FIG. 10C is an enlarged view showing one of the pixel matrices 4015 in FIG. The pixel information 4014 defining the image information is arranged at the center of the row above the pixel matrix 4015. The pixel information 4014 is an intermediate value between the pixel adjacent to the pixel matrix 4015 and the pixel information 4014 on the outer periphery of the pixel matrix 4015. Pixels that are interpolated with predetermined weights are arranged on both pixels.

Here, weighting means multiplying the value of the pixel information 4014 by a predetermined number when obtaining an intermediate value in order to analyze the image information defined by the pixel information 4014 more reliably. According to FIG. 5C, the weighting is doubled the value of the pixel information 4014, but may be tripled or quadrupled as necessary. It should be noted that weighting is possible in the third embodiment.

FIG. 26 and FIG. 27 are diagrams for explaining what kind of information the image information actually means in the above embodiment. According to FIG. 26, codes C ₀ to C ₂₃ are used as determination codes (headers). The combination of the RGB values of the determination code is a combination that is not possible in the natural world, so that the central processing unit (part 2) 2011 indicates that the pixel information is embedded as defining image information. Be able to recognize.

Codes C ₂₄ to C ₂₉ are used for parity check as shown in FIG. Codes _C30 to _C89 mean control information as specifically shown in FIG. Codes C ₉₀ to C ₉₅ are used for parity check as shown in FIG.

As described above, by using the pixel information 4014, it is possible to determine whether an image received as a video signal is a normal planar image or a plurality of intermediate images 401. This is because when the image input to the stereoscopic image generating apparatus 201 is a normal planar image, it is necessary to output the image directly to the stereoscopic image display apparatus 701 without performing processing such as pixel arrangement conversion. . Note that since the RGB value of the pixel information 4014 changes due to irreversible compression as described above, it is preferable to analyze the image information with reference to only the upper bits of a predetermined number of digits. The central processing unit (part 2) 2011 identifies the position where the pixel information 4014 is embedded based on a predetermined arrangement pattern, determines the presence or absence of a header for collating image information, and if there is a header, the image information Is analyzed.

FIG. 28 is a flowchart showing a method of discriminating between a normal planar image and a multi-viewpoint intermediate image 401 by always embedding pixel information 4014 defined as image information in the multi-viewpoint intermediate image 401.

However, in order to prevent the influence from the time direction due to irreversible compression, the frames before and after the frame at the moment when the intermediate image 401 of the multiple viewpoints starts and the frames before and after the frame at the time of the end of the intermediate image 401 of the multiple viewpoints In addition, pixel information 4014 defined as the same image information may be embedded.

Referring to the figure, when receiving an image, the central processing unit (part 2) 2011 analyzes the presence / absence of a header at a predetermined position for each frame based on a predetermined pixel arrangement pattern defined in advance. (1) When there is a header, the frame is an intermediate image 401 frame of a plurality of viewpoints, and image information 4013 is defined in the frame. Therefore, the central processing unit (part 2) 2011 analyzes the image information 4013. To do. (2) When there is no header, the frame is a normal planar image frame, and image information is not defined in the frame, so the central processing unit (part 2) 2011 does not analyze the image information 4013. When the above analysis ends, the central processing unit (part 2) 2011 proceeds to analysis of the next frame.

<Autostereoscopic image display device>
Next, an autostereoscopic image display device 701 having a parallax barrier 704 that outputs a stereoscopic image generated by the stereoscopic image generation method according to the present invention will be described with reference to FIGS.

Conventionally used autostereoscopic image display devices adopting a parallax barrier method have different ranges in which an image presentation target person can visually recognize from a visible light transmission part, and pass through each visible light transmission part. There is a difference in the intensity of the light traveling to the side, the light interferes, and stripe-like interference fringes (moire) are visually recognized by the image presentation target person, resulting in a problem that the image quality of the display image is degraded.

However, according to the configuration of the stereoscopic image display device 701 according to the present embodiment, for example, a predetermined optimum stereoscopic visible position and a predetermined oblique moire elimination position are set at positions where the most people can be stored, and from these values, Since it is possible to determine the distance from the image display surface of the display to the parallax barrier 704 and the interval between the one or more visible light transmission portions 7041 in the horizontal direction by reverse calculation, the moire elimination position in a predetermined oblique direction In this case, the image presentation target person can always visually recognize the predetermined position of the pixel displaying the image of the predetermined viewpoint through the visible light transmitting portion 7041 of the parallax barrier 704, and the moire elimination position is completely at the predetermined moire elimination position. Will be eliminated.

Here, the “visible light transmitting portion 7041” is a portion that transmits visible light that is provided on a surface of the parallax barrier 704 that does not transmit visible light. That is, the “visible light transmitting portion 7041” referred to in the present invention is a shape in which the edge of the slit is linear, stepped, zigzag, or a continuous shape of arc or elliptical arc (gum shape). Also good. Further, the shape of the arrangement of the slits may be a sine arc. Further, the visible light transmitting portion 7041 may be a hole type independently disposed on the parallax barrier 704.

It should be noted that “not transmitting visible light” means any one of the optical characteristics of (1) absorbing visible light, (2) diffusely reflecting visible light, and (3) specularly reflecting visible light.

Also, the “optimal stereoscopic view position” is a position where the person who presents the image can obtain the stereoscopic effect particularly effectively. In other words, at the optimum stereoscopic view position, both eyes of the image presentation target person visually recognize the center of the stereoscopic display pixel 707 for the viewpoint to be visually recognized through the visible light transmission unit 7041 of the parallax barrier 704.

Further, the “moire elimination position” refers to a position at which a subject can be made to visually recognize a stereoscopic image effectively in a form in which moiré is completely reduced, and an image is displayed at a predetermined moire elimination position. The presentation target person can visually recognize a predetermined position of the stereoscopic display pixel 707 that always displays an image of a predetermined viewpoint through the visible light transmitting portion 7041 of the parallax barrier 704 with either the left or right eye. In the moiré elimination position, even if the image presentation subject moves to the left or right or up and down in parallel with the autostereoscopic display, the effect of moiré elimination does not change. The concept of the moire elimination position includes an oblique direction moire elimination position and a horizontal direction moire elimination position, which will be described later.

However, a position where the three-dimensional image can be viewed effectively (optimum three-dimensional visible position), a position where the moire in the oblique direction can be eliminated (oblique moire elimination position), and a position where the moire in the horizontal direction can be eliminated (horizontal direction) The moire elimination position) is another concept, and the distance from these positions to the parallax barrier 704 is not necessarily the same.

However, if these predetermined moire elimination positions are the same distance as the optimal stereoscopic viewing position, the stereoscopic effect can be visually recognized most effectively on the entire surface of the display.

In this way, by setting the moiré elimination position and the optimum stereoscopic visible position at different distances, for example, the moiré elimination position at a distance farther from the parallax barrier 704 than the optimum stereoscopic visibility position, the image presentation target person who is far away first. In particular, a 3D image in which moiré has been eliminated can be viewed without causing the image presentation subject to feel the moire stress, thereby drawing the attention of the image presentation subject and approaching the optimal stereoscopic viewing position. It is also conceivable to make a stereoscopic image with a particularly high stereoscopic effect visible.

First, with reference to FIG. 29 and FIG. 30, an appropriate value of the lateral width Sh of the visible light transmitting portion 7041 will be described.

Vh is the width of the effective visible region that is visible with one eye through the visible light transmitting portion 7041 having the width Sh, αPh is the distance between the centers of the stereoscopic display pixels 707 that display the images of the adjacent viewpoints, and Z is the display image. The distance from the display surface to the parallax barrier 704, L1 is the distance from the image presentation target person to the parallax barrier 704 at the optimum stereoscopic view position, W is the distance between the pupils of the left and right eyes of the image presentation target person, and K is The distance between the gazing points of both eyes of the image presentation target person is shown. A one-dot chain line extending from one eye of the image presentation target person toward the display indicates a gaze line of the image presentation target person.

For example, the optimal stereoscopic viewing position may be a position where the image presentation target person wants to visually recognize the autostereoscopic image particularly effectively in consideration of the application and installation location of the stereoscopic image display device. That is, the distance L1 from the optimal stereoscopic view position to the parallax barrier 704 can take an arbitrary value.

The distance W between the eyes of the left and right eyes of the image presentation subject is 60 to 65 mm if the main subject of the stereoscopic image is Western, 65 to 70 mm if Asian, and 50 if children. What is necessary is just to set and calculate to about ~ 60 mm.

Further, the distance αPh between the centers of the stereoscopic display pixels 707 that display the images of the adjacent viewpoints, for example, constitutes one stereoscopic display pixel 707 with three subpixels as illustrated in FIG. The value of αPh when subpixels are arranged in a stepwise manner in an oblique direction is 1Ph.

Next, the value of the width Vh of the effective visible region that is visually recognized with one eye of the image presentation target person through the visible light transmitting portion 7041 of the parallax barrier 704 is determined.

The effective visible region refers to a region on the image display surface that can be visually recognized through the visible light transmitting portion 7041 of the parallax barrier 704 at the optimal stereoscopic viewing position. That is, it is the range of the display intended to be visually recognized by the image presentation target person at the optimum stereoscopic visible position.

The width Vh of the effective visible region occurs when a person moves, when the image changes from one viewpoint to another, and when the left and right eyes view an image of the opposite viewpoint. In order to reduce the jump point at which the position of the object reverses back and forth, the stereoscopic display pixel 707 that displays images of adjacent viewpoints that should be visually recognized by both eyes is mainly used for the right and left stereoscopic display. This is the horizontal width that is necessary for visually recognizing a part of the pixel 707 to generate an appropriate view mix and visually recognizing with one eye of the image display surface.

Therefore, if Vh is large, the viewpoint is changed and the jump point is reduced. However, the stereoscopic display pixel 707 is different from the stereoscopic display pixel 707 that displays an image of an adjacent viewpoint that should be visually recognized by both eyes. Since 707 (especially, the same image is visually recognized with both eyes covered), the stereoscopic effect is poor. On the other hand, when the value of Vh is small, the stereoscopic effect of the image is enhanced and the stereoscopic image is clearly displayed, but the jump point is increased. However, the above effect is greatly different depending on the shape and arrangement of the slit or visible light transmitting portion 7041.

In this way, the effective visible region width is appropriately widened according to the usage of the stereoscopic image, etc., so that the stereoscopic image can be provided more effectively in accordance with the demand and situation of the image presentation target person. be able to.

As can be seen from FIG. 29, at the optimum stereoscopic visible position, the gaze line (FIG. 29, one-dot chain line) of the image presentation target person visually recognizes the center of each of the stereoscopic display pixels 707. The distance K between the gazing points of the eyes is the same value as αPh.

Next, the value of the distance Z from the image display surface of the display to the parallax barrier 704 is obtained based on the determined value of the effective visible region width Vh. Z is calculated by the following formula.

Note that Z is the distance from the display surface to the parallax barrier 704 even when processing such as transfer prevention is performed on the display surface of the stereoscopic image display device or when a transparent sheet such as reflection prevention is pasted. To do.

As can be seen from FIG. 29, there is a relationship represented by the following formula between Z: L1 and αPh: W.

Therefore, the distance Z is expressed by the following mathematical formula.

Next, based on the determined value of the distance Z, the value of the lateral width Sh of the visible light transmitting portion 7041 is obtained.

From the formula <1>, L1 is expressed as the following formula.

Further, as can be seen from FIG. 29, there is a relationship represented by the following formula between S: Vh and L1: (L1 + Z).

Therefore, the height Sh of the visible light transmitting portion 7041 is expressed by the following mathematical formula.

Therefore, when the formula <2> is substituted into the formula <3>, Sh is represented by the following formula.

Thus, the value of Sh can be obtained from the values of W, αPh, and Vh.

Next, referring to FIG. 31, the shape of the edge of the slit, which is the visible light transmitting portion 7041 constituting the parallax barrier 704, is a stepped shape or a shape in which an arc, an elliptical arc, a polygon is continuous, or the parallax When the shape of the visible light transmitting portion 7041 constituting the barrier 704 is a hole shape formed independently, the continuous visible light transmitting portion 7041 of the shape or the visible light transmitting portion of the plurality of holes. A height Sv of 7041 is obtained.

Here, the height Vv of the effective visible region of the parallax barrier 704 is the range of the display that is visually recognized through the visible light transmitting portion 7041 having the height Sv at the optimum stereoscopic viewing position, and the value is set for the autostereoscopic display. It can be set to a predetermined value in accordance with conditions such as a place to perform.

For example, when the aperture ratio of the parallax barrier 704 is suppressed and the illuminance of the display is reduced, the value of the effective visible region may be reduced.

As another method of adjusting the aperture ratio of the parallax barrier 704, one unit of edges of a plurality of continuous slits or a visible light transmitting portion 7041 may be used for one subpixel, or two or more. One continuous visible light transmitting portion 7041 having the shape or the plurality of hole-shaped visible light transmitting portions 7041 may be used for each of the sub-pixels.

Thus, even when the ratio of the number of visible light transmitting portions 7041 to one subpixel is other than 1: 1, the height Vv of the effective visible region is the height of the visible light transmitting portion 7041. This refers to the range of the display that can be seen through.

As can be seen from FIG. 31, there is a relationship represented by the following mathematical formula between Sv: Vv and L1: (L1 + Z).

Therefore, the height Sv of the visible light transmitting portion 7041 is expressed by the following equation.

Thus, the value of the height Sv of the visible light transmitting portion 7041 can be calculated backward by first determining the value of the effective visible region height Vv.

Further, the height Sv of the visible light transmitting portion 7041 can also be obtained by the following formula based on the interval Hv of the visible light transmitting portion 7041.

That is, as shown in FIG. 32, first, after obtaining the interval Hv of the visible light transmitting portion 7041 based on the above equation, the value of λ (1/2 in the drawing) is determined and substituted into the above equation. Thus, the height of the visible light transmitting portion 7041 can be obtained.

Next, referring to FIG. 33, the distance from the predetermined oblique direction moire elimination position to the parallax barrier 704 is based on L2, and the interval between the plurality of visible light transmitting portions 7041 constituting the parallax barrier 704 adjacent in the horizontal direction. Find Hh.

In FIG. 33, the image presentation target person configures the RGB staircase-arranged pixel block 13 at the left end of the display through the visible light transmitting portion 7041 of the parallax barrier 704 with one eye (left eye) at a predetermined oblique direction moire elimination position. 3D display pixels and the 3D display pixels constituting the RGB staircase-arranged pixel block 13 at the right end of the display are visually recognized, and the 3D display pixels viewed by the image presentation target person are images of the same viewpoint. Is displayed.

As described above, if the sub-pixels that are visually recognized through the visible light transmitting portion 7041 of the parallax barrier 704 always display images of the same viewpoint, the image presentation target person does not visually recognize the moire on the screen.

Here, first, from the visible light transmitting portion 7041 of the parallax barrier 704 to the RGB staircase arranged pixel block 13 at the left end of the display at a predetermined oblique direction moire elimination position, the parallax to the RGB staircase arranged pixel block 13 at the right end of the display is set. The number Mh of the visible light transmitting portions 7041 in the horizontal direction between the barrier 704 and the visible light transmitting portion 7041 is expressed by the following formula using the number N of viewpoints for displaying a stereoscopic image and the horizontal resolution Ir. Can do.

That is, 3Ir obtained by multiplying the horizontal resolution Ir by 3 (R, G, B) is the number of sub-pixels in the horizontal direction. For example, as illustrated in FIG. 34, when the number of viewpoints is assumed to be 7, the sub-pixel on the right end of the display does not display the image of the seventh viewpoint, which is the last viewpoint of the viewpoint. This is because there is a case of the first viewpoint, and in this case, it is necessary to calculate by subtracting the number of sub-pixels that display the surplus image of the first viewpoint. In addition, adding 1 at the end is an integer obtained by subtracting 1 from the total number of subpixels even when the subpixel displaying the image of the first viewpoint is not at the right end of the display. This is to compensate for the lack of one Mh value.

In addition, from the center of the visible light transmitting portion 7041 for the stereoscopic display pixels constituting the RGB staircase arranged pixel block 13 at the left end of the display, the stereoscopic display pixels that display an image of the same viewpoint as this, and the RGB at the right end of the display The distance to the center of the visible light transmitting portion 7041 with respect to what constitutes the staircase arranged pixel block 13 is a value obtained by multiplying Hh (the interval between the visible light transmitting portions 7041 in the horizontal direction) by (Mh−1).

Further, in the horizontal direction, from the center of the stereoscopic display pixels constituting the RGB staircase-arranged pixel block 13 at the left end of the display, which is viewed by the image presentation target person through the visible light transmitting unit 7041 of the parallax barrier 704, the same viewpoint as this The distance to the center of the RGB staircase arranged pixel block 13 at the right end of the display, which is a stereoscopic display pixel that displays the image of the image, is adjacent to N as the number of viewpoints of the image for generating a naked-eye stereoscopic image This can be expressed by the following equation using the distance αPh between the centers of the stereoscopic display pixels for displaying the viewpoint image.

As can be seen from FIG. 33, there is a relationship that can be expressed by the following equation between [Hh × (Mh−1)]: [N × (Mh−1) × αPh] and L2: (Z + L2).

Therefore, the value of Hh can be obtained by the following equation.

Thus, based on the distance L2 from the predetermined oblique moire elimination position to the parallax barrier 704, the value of the interval Hh between the plurality of visible light transmitting portions 7041 constituting the parallax barrier 704 adjacent in the horizontal direction is obtained. be able to.

Next, referring to FIG. 35 and FIG. 36, a plurality of the parallax barriers 704 that are adjacent in the horizontal direction based on the distance from the parallax barrier 704 to the point where one moire in the oblique direction occurs. An interval Hh between the visible light transmitting portions 7041 is obtained.

35, the distance from the position where one moire in the oblique direction is generated to the parallax barrier 704, and the parallax barrier 704 is closer to the parallax barrier 704 out of the two kinds of positions. At the predetermined distance L2n to the lux barrier 704, as illustrated in FIG. 37, the person who presents the image can display the RGB staircase-arranged pixel block 13 at the left end of the display in the same manner as the predetermined oblique moire elimination position (L2). 3D are displayed through the visible light transmitting portion 7041 of the parallax barrier 704 while displaying the image for the first viewpoint among the three-dimensional display pixels constituting the image. As the viewpoint shifts to the right, the visible light transmission is performed. Through the unit 7041, the stereoscopic display pixels for the other viewpoints are not visually recognized but the stereoscopic display pixels for the other viewpoints. It becomes Kukoto. Finally, the right edge of the display passes through the visible light transmitting portion 7041 through which visible light is transmitted when viewing the stereoscopic display pixel for the first viewpoint in the RGB staircase arranged pixel block 13 at the right edge of the display at the point L2. When the virtual pixel 705 is assumed on the right side of the three-dimensional display, the stereoscopic display pixel (virtual) for the first viewpoint is visually recognized again. Since such a cycle occurs once, it is considered that moire occurs once in L2n.

When such a value of L2n is set to a predetermined value, an interval Hh between the plurality of visible light transmitting portions 7041 constituting the parallax barrier 704 adjacent in the horizontal direction is obtained based on this value.

That is, as can be seen from FIG. 35, there is a relationship represented by the following equation between [Hh × (M−1)]: [N × M × αPh] and L2n: (Z + L2n).

Therefore, Hh can be obtained by the following equation.

Similarly to the determination of the value of Hh based on L2n, it is the distance from the position where one moire in the oblique direction occurs to the parallax barrier 704, and more The value of Hh can be obtained based on a predetermined distance L2f from a position far from the lux barrier 704 to the parallax barrier 704.

In the point of L2f illustrated in FIG. 36, the person who presents the image displays the 3D display pixel constituting the RGB staircase arrangement pixel block 13 at the left end of the display in the same way as the predetermined moire elimination position (L2) in the diagonal direction. Among them, the one that displays the image for the first viewpoint is visually recognized through the visible light transmission unit 7041 of the parallax barrier 704. As the viewpoint shifts to the right, the image for the first viewpoint is displayed through the visible light transmission unit 7041. Instead of the stereoscopic display pixels, the stereoscopic display pixels for other viewpoints are visually recognized. Finally, the right edge of the display passes through the visible light transmitting portion 7041 through which visible light is transmitted when viewing the stereoscopic display pixel for the first viewpoint in the RGB staircase arranged pixel block 13 at the right edge of the display at the point L2. The stereoscopic display pixel 2 for the first viewpoint in the RGB staircase-arranged pixel block 13 adjacent to the left of the RGB staircase-arranged pixel block 13 is visually recognized. Since such a cycle occurs once, it is considered that moire occurs once in L2f.

In FIG. 38, the relative relationship between L2, L2n, and L2f is illustrated.

When the value of L2f is set to a predetermined value, the distance Hh between the plurality of visible light transmitting portions 7041 constituting the parallax barrier 704 adjacent in the horizontal direction is obtained based on this value.

That is, as can be seen from FIG. 36, there is a relationship represented by the following equation between [Hh × (M−1)]: [N × (M−2) × αPh] and Z: (Z + L2). There is.

Therefore, the value of Hh can be obtained by the following equation.

As described above, the distance Hh between the plurality of visible light transmitting portions 7041 constituting the parallax barrier 704 adjacent in the horizontal direction can be obtained based on the value of the point (L2n · L2f) where one moire occurs. For example, the area from the point L2n to the point L2f can be designated as a moire appropriate elimination region, and a point where the stereoscopic image can be visually recognized particularly effectively can be clearly shown to the image presentation target person. Furthermore, by setting the moiré elimination area in a range where the most people can be collected, it is possible to attract the attention of the image presentation target person.

Next, referring to FIG. 39, the shape of the edge of the slit which is the visible light transmitting portion 7041 constituting the parallax barrier 704 is a stepped shape or a shape in which an arc, an elliptical arc, a polygon is continuous, or the parallax Based on the value of the distance L3 from the predetermined moire elimination position in the horizontal direction to the parallax barrier 704 when the shape of the visible light transmitting portion 7041 constituting the barrier 704 is a hole shape formed independently. A method of determining the value of the interval Hv between the continuous visible light transmitting portions 7041 or the plurality of hole-shaped visible light transmitting portions 7041 connected in the vertical direction of the parallax barrier 704 will be described.

Here, the alternate long and short dash line in FIG. 39 indicates the gaze line of the image presentation target person, and K indicates the distance between the upper and lower gaze points of the image presentation target person.

The value of the distance L3 from the parallax barrier 704 to a predetermined horizontal moiré elimination position is determined depending on what distance from the display the user wants to provide a stereoscopic image in a form in which moiré is particularly eliminated.

In addition, since the image presentation target always gazes at the center of the sub-pixel through the visible light transmitting portion 7041 of the parallax barrier 704 at the horizontal moire elimination position, the distance between the gaze points of the image presentation target person K Is equal to the subpixel height Pv.

Further, β represents the number of visible light transmitting portions 7041 corresponding to one subpixel in the vertical direction. For example, as shown in FIGS. When the visible light transmitting portion 7041 is formed, β is 1. Further, as shown in FIGS. 40B and 40E, when two visible light transmitting portions 7041 are formed for one subpixel, β is two. Furthermore, as shown in FIGS. 40C and 40F, when one visible light transmitting portion 7041 is formed for three subpixels, β is １／.

That is, β is a unit of the continuous visible light transmitting portion 7041 having the shape corresponding to one subpixel or the number of the visible light transmitting portions 7041 having the plurality of hole shapes in the vertical direction.

Note that the number of visible light transmitting portions 7041 provided for one subpixel is desirably an integer. In the case where one visible light transmitting portion 7041 is provided for a plurality of subpixels, it is desirable to provide an integer number of visible light transmitting portions 7041 for one stereoscopic display pixel.

Here, the value of the interval Hv of the continuous unit or the visible light transmitting portion 7041 is obtained.

As can be seen from FIG. 41, the relationship between the upper and lower gaze point distance K (= Pv) and L3: (L3 + Z) in Hv × β: L3 can be expressed by the following equation.

Therefore, Hv is expressed by the following equation.

As described above, at a predetermined horizontal moire elimination position, it is possible to determine a value of Hv that can eliminate moire in particular by calculating backward from the value of L3.

42 and 43, the shape of the edge of the slit which is the visible light transmitting portion 7041 constituting the parallax barrier 704 is a stepped shape or a shape in which an arc, an elliptical arc, a polygon is continuous, or the parallax. When the shape of the visible light transmitting portion 7041 constituting the barrier 704 is a hole shape formed independently, the distance from the position where one horizontal moire occurs to the parallax barrier 704, Based on the value of L3n, a predetermined distance from the position closer to the parallax barrier 704 to the parallax barrier 704 is connected in the vertical direction of the parallax barrier 704 among the two types of the near and far positions. A method for obtaining a value of the interval Hv between the visible light transmitting portions 7041 having the continuous shape or the visible light transmitting portions 7041 having the plurality of holes. It described Te.

In L3n illustrated in FIG. 42, the image presentation target person visually recognizes the subpixel at the lower end of the display through the visible light transmitting portion 7041 of the parallax barrier 704 in the same manner as the predetermined horizontal moire elimination position (L3). However, as the viewpoint shifts upward, not the sub-pixel that should be visually recognized at the point of L3 but the sub-pixel above it is visually recognized through the visible light transmitting portion 7041. Finally, when a virtual subpixel 706 is assumed above the upper end of the display through the visible light transmitting portion 7041 through which visible light is transmitted when the subpixel at the upper end of the display is viewed at the point of L3, the virtual subpixel 706 will be visually recognized. Since such a cycle occurs once, it is considered that moire occurs once in L3n.

First, the continuous visible light transmission of the shape in the vertical direction from the visible light transmission portion 7041 having the shape corresponding to the sub-pixel at the upper end of the display to the visible light transmission portion 7041 having the shape corresponding to the sub-pixel at the lower end of the display. One unit of the portion 7041 or the number Mv of the plurality of hole-shaped visible light transmitting portions 7041 will be described.

As shown in FIG. 44B, Mv is for stereoscopic display in which an image presentation target person displays an image of the same viewpoint on a display at one point in a predetermined horizontal moire elimination position (L3). This is the number of visible light transmitting portions 7041 of the parallax barrier 704 necessary for visually recognizing all of the pixels and obtaining a stereoscopic image effect.

Here, “the number of units of the visible light transmitting portion 7041 having the continuous shape” means, for example, when the shape of the slit that is the visible light transmitting portion 7041 of the parallax barrier 704 is an elliptical arc. Means the number of the three-dimensional display pixels displaying the image of the same viewpoint on the corresponding slits. Also. “The number of the plurality of hole-shaped visible light transmitting portions 7041” means that the number of the hole-shaped visible light transmitting portions 7041 corresponding to the arrangement of the three-dimensional display pixels displaying the same viewpoint image is formed. It means the number. Jr indicates the vertical resolution of the display.

Therefore, Mv can be expressed by the equation Jr × β.

When the value of L3n is a predetermined value, based on this value, the interval Hv between the plurality of visible light transmitting portions 7041 constituting the parallax barrier 704 connected in the vertical direction is obtained.

That is, as can be seen from FIG. 42, the relationship between [Hv (Mv−1)]: [(Jr−1 / β + 1) × Pv] and Z: (Z + L3n) is expressed by the following equation. There is.

Therefore, Hv can be obtained by the following equation.

Next, the shape of the edge of the slit which is the visible light transmitting portion 7041 constituting the parallax barrier 704 is a stepped shape or a shape in which arcs, elliptical arcs, and polygons are continuous, or visible light constituting the parallax barrier 704. When the shape of the transmission part 7041 is a hole shape formed independently, it is the distance from the position where one moire in the horizontal direction occurs to the parallax barrier 704, and the two kinds of the near and near positions Among them, the continuous visible light having the shape connected to the parallax barrier 704 in a vertical direction based on the value of L3f from a position farther from the parallax barrier 704 to the parallax barrier 704. A method of obtaining the value of the interval Hv between the transmission part 7041 or the plurality of hole-shaped visible light transmission parts 7041 will be described.

In the L3f illustrated in FIG. 43, the image presentation target person visually recognizes the subpixel at the lower end of the display through the visible light transmitting portion 7041 of the parallax barrier 704 in the same manner as the predetermined horizontal moire elimination position (L3). However, as the viewpoint shifts upward, not the sub-pixel that should be visually recognized at the point L3 but the sub-pixel below it is visually recognized through the visible light transmitting portion 7041. Finally, the subpixel below the upper end of the display is viewed through the visible light transmitting portion 7041 through which visible light passes when the subpixel at the upper end of the display is viewed at the point L3. Since such a cycle occurs once, it is considered that moire occurs once in L3n.

When such a value of L3f is set to a predetermined value, the interval Hv between the plurality of visible light transmitting portions 7041 constituting the parallax barrier 704 connected in the vertical direction is obtained based on this value.

That is, as can be seen from FIG. 43, a value between [Hv × (Mv−1)]: [(Jr−1 / β−1) × Pv] and Z: (Z + L3f) is expressed by the following equation. There is a relationship.

Therefore, Hv can be obtained by the following equation.

When β = 2, the relationship between [Hv × (Mv−1)] and [(Jr−1 / β) × Pv] is as shown in FIG.

The value of Hv is preferably a value that satisfies the relationship of the equation: Hv = Hpv / β (β is a natural number), where Hpv is the interval between subpixels connected in the vertical direction.

The values of L3n and L3f can be determined based on the value of L3. This will be described with reference to FIGS. 46 to 48. FIG.

46, when the vertical resolution Jr is multiplied by the height Pv of each sub-pixel, the distance from the lower end of the display to the upper end of the display is obtained. Therefore, the distance from the center of the subpixel at the lower end of the display to the center of the virtual subpixel 706 above the upper end of the display can also be expressed as (Pv × Jr).

Further, when Jr is multiplied by the interval Hv of the visible light transmitting portion 7041 connected in the vertical direction, at the moire elimination position, from the center of the visible light transmitting portion 7041 corresponding to the sub pixel at the lower end of the display, the virtual above the upper end of the display This is the distance to the center of the visible light transmitting portion 7041 corresponding to the subpixel 706. (Hv x Jr)
Next, referring to FIG. 47, L3n is obtained.

In L3n, one vertical moiré is visually recognized by the image presentation target person. Therefore, the parallax barrier 704 that is visible when the image presentation target person visually recognizes the stereoscopic image is more visible than the number of subpixels in the vertical direction. Although the number of the light transmission parts 7041 is one, the person who presents the image is viewing all the sub-pixels through the visible light transmission part 7041.

Therefore, it can be said that L3n is a point where a cycle of moire generation in the vertical direction occurs once.

That is, in L3n, from the center of the visible light transmitting portion 7041 of the parallax barrier 704 corresponding to the sub pixel at the lower end of the display to the visible light transmitting portion 7041 of the parallax barrier 704 corresponding to the virtual sub pixel 706 above the upper end of the display. Can be expressed as Hv × (Jr−1).

Here, by substituting Hv in the equation (4) above, it can be expressed as follows.

As can be seen from FIG.
L3n: (L3n + Z)

There is a relationship represented by the following equation.

Therefore, L3n is represented by the following equation.

Next, referring to FIG. 48, the value of L3f is obtained based on the value of L3.

Even in L3f, since one vertical moiré is visually recognized by the image presentation target person, the visible light of the parallax barrier 704 that is transmitted when the image presentation target person visually recognizes the stereoscopic image is determined from the number of vertical subpixels. Although the number of the transmissive portions 7041 is one, the image presentation target person visually recognizes all the sub-pixels through the visible light transmissive portion 7041.

That is, in L3f, from the center of the visible light transmitting portion 7041 of the parallax barrier 704 corresponding to the subpixel at the lower end of the display to the visible light transmitting portion 7041 of the parallax barrier 704 corresponding to the virtual subpixel 706 at the upper end of the display. Can be expressed as Hv × (Jr + 1).

Therefore, L3f can be expressed by the following equation if the same idea as the equation for obtaining L3n is used.

Note that the range from the L3n to the L3f is a moire appropriate elimination region in the vertical direction.

Here, an embodiment in the case of a full-vision 40-inch autostereoscopic display will be described. In this case, the horizontal resolution Ir is 1920 and the vertical resolution Jr is 1080.

The horizontal width Ph of the subpixel is 0.15375 mm, the distance L1 from the parallax barrier 704 to the optimal stereoscopic view position is 2500 mm, the number of viewpoints N is 5, and the distance W between the eyes of the left and right eyes of the image presentation subject is 65 mm. The horizontal resolution Ir is 1920, and the vertical resolution Jr is 1080. Further, the distances L2 and L3 from the parallax barrier 704 to the moire elimination positions in the oblique direction and the horizontal direction are also set to 2500 mm. In this embodiment, L1, L2, and L3 are the same value, but L1, L2, and L3 are not necessarily the same value.

Further, the distance αPh between the centers of the stereoscopic display pixels that display the images of the adjacent viewpoints is 1 Ph, and the effective visible region that is visually recognized with one eye of the image presentation target person through the visible light transmitting portion 7041 of the parallax barrier 704. The width Vh is 1.2 Ph.

Therefore, the values of αPh and Vh are as follows.

Next, the value of the distance Z is obtained by the following formula.

Next, Sh is obtained based on the obtained values of Z and Vh.

Note that how short Sh is with respect to Vh is as follows.

Next, the shape of the edge of the slit which is the visible light transmitting portion 7041 constituting the parallax barrier 704 is a stepped shape or a shape in which arcs, elliptical arcs, and polygons are continuous, or visible light constituting the parallax barrier 704. When the shape of the transmitting portion 7041 is a hole shape formed independently, the value of the height Sv of the visible light transmitting portion 7041 having the continuous shape or the visible light transmitting portion 7041 having the plurality of holes. Ask for.

The value of the height Vv of the effective visible region of the parallax barrier 704 is ε × Pv. Note that ε is a coefficient indicating the range of the sub-pixels visible through Sv, that is, the ratio of the height Vv of the effective visible region to the sub-pixel height Pv. In other words, the aperture ratio of the parallax barrier 704 in the vertical direction. In this embodiment, ε is 0.9.

Further, here, it is assumed that the display is an autostereoscopic display in which one pixel is composed of three RGB sub-pixels, and Pv is 3Ph (= 0.40625) assuming that one pixel is a square.

Also, one unit of the visible light transmitting portion 7041 having the above-described shape corresponding to one subpixel or the number β in the vertical direction of the plurality of hole-shaped visible light transmitting portions 7041 is 1.

Therefore, the value of Vv is as follows.

Moreover, the value of Sv is the following value.

It should be noted that how short Sv is with respect to the value of Vv is as follows.

Next, based on the value of the distance L2 from the predetermined oblique moire elimination position to the parallax barrier 704, the interval Hh between the plurality of slit regions constituting the parallax barrier 704 adjacent in the horizontal direction is as follows. Ask for.

Note that how short Hh is with respect to N × αPh is as follows.

In addition, the value of the interval Hh between the plurality of slit regions constituting the parallax barrier 704 adjacent in the horizontal direction is the distance from the position where one moire in the oblique direction occurs to the parallax barrier 704, and two kinds of perspective , A predetermined distance L2n from a position closer to the parallax barrier 704 to the parallax barrier 704, or a predetermined distance L2f from a position further to the parallax barrier 704 to the parallax barrier 704. It can also be obtained from the value of.

As an example, the value of Hh is obtained by setting the predetermined value of L2n to 1000 mm and the value of L2f to 3000 mm.

First, from a visible light transmitting portion 7041 of the parallax barrier 704 to the RGB staircase arrangement pixel unit at the left end of the display at a predetermined oblique moiré elimination position, the parallax barrier 704 is visible to the RGB staircase arrangement pixel unit at the right end of the display. The value of the number Mh of visible light transmitting portions 7041 in the horizontal direction between the light transmitting portions 7041 can be obtained by the following equation.

Therefore, based on the value of L2n (1000 mm), the value of Hh can be obtained by the following equation.

Further, based on the value of L2f (3000 mm), the value of Hh can be obtained by the following equation.

Note that the value of L2n can be determined based on the value of the moire elimination position L2 in the oblique direction.

That is, when the value of L2 is 2500 mm, the value of L2n is as follows.

Also, the value of L2f can be determined based on the value of the moire elimination position L2 in the oblique direction.

That is, when the value of L2 is 2500 mm, the value of L2f is as follows.

Next, the shape of the edge of the slit which is the visible light transmitting portion 7041 constituting the parallax barrier 704 is a stepped shape or a shape in which arcs, elliptical arcs, and polygons are continuous, or visible light constituting the parallax barrier 704. When the shape of the transmissive portion 7041 is a hole shape formed independently, the visible light transmissive portion 7041 having the continuous shape or the visible light transmissive portion 7041 having the plurality of holes connected in the vertical direction. The value of the interval Hv is obtained as follows.

In this embodiment, it is assumed that one visible light transmitting portion 7041 of the parallax barrier 704 is provided for each subpixel, and the value of β is 1.

Therefore, the value of Hv can be obtained as follows.

Note that how short Hv is with respect to the value of Pv is as follows.

Further, the value of the interval Hv between the continuous visible light transmitting portions 7041 or the plurality of hole-shaped visible light transmitting portions 7041 connected in the vertical direction varies from the position where one horizontal moire occurs. It is a distance to the lux barrier 704, and a predetermined distance L3n from a position closer to the parallax barrier 704 to the parallax barrier 704, or more to the parallax barrier 704. A predetermined distance from a distant position to the parallax barrier 704 can also be obtained based on the value of L3f.

As an example, the value of Hv is obtained by setting the predetermined value of L3n to 1000 mm and the value of L3f to 3000 mm.

In the present embodiment, since the β value is 1, the visible light transmission portion 7041 having the shape corresponding to the sub pixel at the upper end of the display from the visible light transmitting portion 7041 corresponding to the sub pixel at the upper end of the display at the predetermined horizontal direction moire elimination position. The unit Mv of the visible light transmitting portion 7041 having the continuous shape in the vertical direction up to the portion 7041 or the number Mv of the plurality of hole-shaped visible light transmitting portions 7041 is as follows.

Based on the value of L3n (1000 mm), the value of Hv can be obtained by the following equation.

Further, based on the value of L3f (3000 mm), the value of Hv can be obtained by the following equation.

Note that the value of L3n can be determined based on the value of the moire elimination position L3 in the horizontal direction.

That is, when the value of L3 is 2500 mm, L3n is as follows.

Note that the value of L3f can also be determined based on the value of the moire elimination position L3 in the horizontal direction.

That is, when the value of L3 is 2500 mm, the value of L3f is as follows.

Next, an appropriate stereoscopic visible region is obtained.

The shortest distance L1n of the appropriate stereoscopic visible region is the following value.

The longest distance L1f of the appropriate stereoscopic visible region is the following value.

Therefore, the appropriate stereoscopic visible region is 2078 mm to 4988 mm.

In addition, when calculating Vh as 1.2 Ph in this way, L1n: L1 has a relationship of approximately 0.8: 1.

Here, a second embodiment in the case of a full-high vision 40-inch autostereoscopic display will be described.

In the second embodiment, the L1n (the shortest distance to the optimum stereoscopic visible region), the L2n (the shortest distance to the moire appropriate elimination area in the oblique direction), and the L3n (the shortest distance to the moire appropriate elimination area in the horizontal direction). A case will be described in which (distance) is set to the same distance.

Since L1n, L2n, and L3n are different concepts as described above, they are not limited to the case where all of them are set to the same distance as described in the present embodiment.

In this case, as in the first embodiment, the horizontal resolution Ir is 1920, the vertical resolution Jr is 1080, the horizontal width Ph of the subpixel is 0.15375 mm, the height of the subpixel is 0.46125 mm, and the number of viewpoints N is 5. The distance W between the pupils of the right and left eyes of the viewpoint and the image presentation subject is 65 mm, the distance from the parallax barrier 704 to the optimum stereoscopic view position is 2500 mm, and between the centers of the pixels for stereoscopic display displaying the images of the adjacent viewpoints The distance αPh is 0.15375 mm, the width Vh of the effective visible region visually recognized with one eye of the image presentation subject through the visible light transmitting portion 7041 of the parallax barrier 704 is 0.1845 mm, and the visible light transmitting portion of the parallax barrier 704 The height Vv of the effective visible region visually recognized by the image presentation target person through 7041 is 0.415125 mm.

In addition, from a visible light transmitting portion 7041 of the parallax barrier 704 to the RGB staircase arranged pixel block at the left end of the display at a predetermined oblique direction moire elimination position, the parallax barrier 704 is visible to the RGB staircase arranged pixel block at the right end of the display. The number Mh of the visible light transmitting portions 7041 in the horizontal direction up to the light transmitting portion 7041 is 1152, the number γ in the left-right direction of the visible light transmitting portion 7041 corresponding to one subpixel is 1, and one sub A unit β of the visible light transmitting portion 7041 having the shape corresponding to the pixel or the number β in the vertical direction of the plurality of hole-shaped visible light transmitting portions 7041 is 1.

First, it is L1n, but L1n can be obtained by the following equation using Z, W, and Vh.

Therefore, L1n has the following value.

In this case, L1f (the longest distance to the appropriate stereoscopic visible region) is the following value.

That is, the appropriate stereoscopic visible region is 2078 mm to 4988 mm.

Therefore, it is assumed that L2n and L3n are also set to 2078 mm, which is the same distance as L1n.

Next, the distance Z from the image display surface of the display to the parallax barrier 704 is obtained.

The value of Z can be obtained based on the value of L1n.

Therefore, Z has the following value.

Next, an interval Hh between visible light transmitting portions 7041 adjacent in the horizontal direction is obtained.

The value of Hh can be obtained based on the value of L2n.

Therefore, Hh has the following value.

Next, an interval Hv between the visible light transmitting portions 7041 connected in the vertical direction is obtained.

The value of Hv can be obtained based on the value of L3n.

Therefore, Hv has the following value.

Next, the width Sh of the visible light transmitting portion 7041 of the parallax barrier 704 is obtained.

The value of Sh can be obtained based on the following equation.

Therefore, Sh has the following value.

Next, the height Sv of the visible light transmitting portion 7041 of the parallax barrier 704 is obtained.

The value of Sv can be obtained based on the following equation.

Accordingly, Sv has the following value.

49 to 54 are diagrams for explaining an example of the shape of the slit of the parallax barrier 704. FIG.

FIG. 49 is a diagram showing a case where the shape of the edge of the slit is stepped. Here, the case where the shape of the edge of the slit is stepped means the case as shown in FIG. 49A, and the case where the shape of the edge of the slit is an arc is shown in FIG. 49B. Say such a case.

Further, the case where the shape of the edge of the slit is an elliptical arc refers to a case as shown as an example in FIGS. 50 (a) and 50 (b).

Also, the case where the shape of the slit edge is a shape in which polygonal openings are continuous refers to a case as shown as an example in FIGS. 51 (a) and 51 (b).

Further, the case where the visible light transmitting portion 7041 is a hole-shaped opening formed with a plurality of independent shapes is the periphery of the visible light transmitting portion 7041 as shown as an example in FIGS. 52, 53, and 54. Is a hole surrounded by the mask portion of the parallax barrier 704.

The description of the embodiments of the present invention has been given above, but the present invention is not limited to the above-described embodiments, and various combinations are possible within the scope shown in the claims, and are disclosed in different embodiments. Embodiments obtained by appropriately combining technical means are also included in the technical scope of the present invention.

The present invention makes it possible to provide a highly practical stereoscopic image display system that is optimal for the development of stereoscopic technology at a very low price.

101 Intermediate Image Generation Device 1011 Central Processing Unit (Part 1)
1012 Storage device (1)
201 stereoscopic image generation apparatus 2011 central processing unit (2)
2012 Storage device (2)
301 stereoscopic image generation system 3011 first information processing apparatus 30111 central processing unit (part 3)
30112 Storage device (3)
30113 Compression device 30114 Transmission device 3012 Second information processing device 30121 Central processing unit (Part 4)
30122 Storage device (part 4)
30123 Decompression device 30124 Reception device 401 Intermediate image 4011 RGB parallel arrangement pixel unit 4012 Image frame 4013 Intermediate image generation table 4014 Pixel information 4015 Pixel matrix 501 Stereo image 5011 RGB staircase arrangement pixel unit 5012 RGB staircase arrangement pixel block 5013 Stereo image generation table 601 Viewpoint image 701 Stereoscopic image display device 702 Video cable 703 Control cable 704 Parallax barrier 7041 Visible light transmission unit 705 Virtual pixel 706 Virtual subpixel 707 Stereoscopic display pixel 708 First viewpoint 709 Second viewpoint 710 Third Viewpoint 801 Camera 8011 Gaze point 8012 Object 1001 Image output device

Claims

An intermediate image generation method for generating a plurality of intermediate images used for generating a stereoscopic image converted from a plurality of viewpoint images photographed and / or drawn from a plurality of viewpoints from one viewpoint to N viewpoints,
RGBY staircase-arranged pixel units in which RGBY sub-pixels are arranged in 4 rows diagonally in contact with corners in an oblique direction, and RGBY staircase-arranged pixel blocks in which the 1 to N viewpoints are connected in a horizontal direction are repeatedly arranged. In order to generate the stereoscopic image,
The plurality of viewpoint images in which the R value, G value, B value, and Y value of each of the sub-pixels constituting the RGBY staircase-arranged pixel unit correspond to the arrangement positions of the sub-pixels that constitute the RGBY staircase-arranged pixel unit. And interpolating from RGBY values of sub-pixels constituting at least one pixel unit arranged in the vicinity of the corresponding position in
An RGBY parallel arrangement pixel unit in which subpixels constituting the RGBY staircase arrangement pixel unit are arranged in the order of RGBY in the horizontal direction is arranged in accordance with an arrangement rule for collectively arranging the plurality of viewpoints. By generating the intermediate image of
An intermediate image generation method, wherein the total number of RGBY staircase arranged pixel units of the stereoscopic image and the plurality of intermediate image RGBY parallel arranged pixel units, or the total number of subpixels constituting each of them are the same.
The RGBY staircase arrangement pixel unit is:
The subpixel for each row is one column, and is composed of four subpixels having R value, G value, B value, and Y value,
The RGBY parallel arrangement pixel unit is:
2. The intermediate image generation method according to claim 1, wherein the four sub-pixels are arranged in four rows in the order of RGBY in one row.
The RGBY staircase arrangement pixel unit is:
There are two columns of subpixels for each row, and each column of the two columns is composed of four subpixels having an R value, a G value, a B value, and a Y value.
The RGBY parallel arrangement pixel unit is:
In one row, four subpixels arranged in four rows in the first column of the RGBY staircase arrangement pixel unit are arranged in four rows in the order of RGBY, and the RGBY staircase arrangement is connected to the arrangement in the horizontal direction. 2. The intermediate image generating method according to claim 1, wherein four sub-pixels arranged in four rows in the second column of the pixel unit are arranged in four columns in the order of RGBY.
The RGBY staircase arrangement pixel unit is:
There are three subpixels for each row, and each column of the three columns is composed of four subpixels having an R value, a G value, a B value, and a Y value.
The RGBY parallel arrangement pixel unit is:
In one row, four sub-pixels arranged in four rows in the first column of the RGBY staircase-arranged pixel unit are arranged in the order of RGBY, and are connected in the horizontal direction to the RGBY staircase-arranged pixel unit. 4 sub-pixels arranged in 4 rows in the second column of the RGBY are arranged in the order of RGBY, and are further connected to the 4 sub-pixels arranged in the third column of the RGBY staircase arrangement pixel unit. 2. The intermediate image generation method according to claim 1, wherein the pixels are arranged in the order of RGBY.
The RGBY staircase arrangement pixel unit is:
There are four subpixels for each row, and each column of the four columns is composed of four subpixels having an R value, a G value, a B value, and a Y value.
The RGBY parallel arrangement pixel unit is:
In one row, four sub-pixels arranged in four rows in the first column of the RGBY staircase-arranged pixel unit are arranged in the order of RGBY, and are connected in the horizontal direction to the RGBY staircase-arranged pixel unit. 4 sub-pixels arranged in 4 rows in the second column of the RGBY are arranged in the order of RGBY, and are further connected to the 4 sub-pixels arranged in the third column of the RGBY staircase arrangement pixel unit. The pixels are arranged in the order of RGBY, and in the case of the RGBY parallel arrangement pixel unit, the four subpixels arranged in the fourth column of the RGBY staircase arrangement pixel unit are arranged in the order of RGBY. The intermediate image generation method according to claim 1, wherein:
By arranging the plurality of intermediate images in a plurality of tiles composed of a first row to a fourth row that are at least vertically divided into four as image frames,
Subpixels constituting the RGBY staircase arranged pixel unit, and subpixels constituting the RGBY parallel arranged pixel unit,
The intermediate image generation method according to claim 1, wherein the number of the stereoscopic images and the plurality of intermediate images are the same in both vertical and horizontal directions.
The RGBY parallel arrangement pixel unit,
When the plurality of viewpoints are two viewpoints, the intermediate image of one viewpoint is assigned to the tiles of the first row and the second row, and the intermediate image of two viewpoints is assigned to the tiles of the third row and the fourth row. ,
In the case where the plurality of viewpoints are three viewpoints, 3/4 of the intermediate image of one viewpoint in the first row tile and 1 of the intermediate image of one viewpoint in the first tile of the second row. / 4, 1/2 of the intermediate image of 2 viewpoints to the second tile connected to this, 1/2 of the intermediate image of 2 viewpoints to the first tile of the third row, and connected to this 1/4 of the intermediate image of 3 viewpoints in the second tile, and 3/4 of the intermediate image of 2 viewpoints in the fourth row tile,
In the case where the plurality of viewpoints are four viewpoints, an intermediate image of each viewpoint is displayed on each row tile.
When the plurality of viewpoints are 5 to 8 viewpoints, the intermediate image of 1 to 4 viewpoints and the first line to the 4th line connected to the intermediate images of 1 to 4 viewpoints on the first tile of each line The intermediate image of the remaining viewpoint is placed on the placed tile.
When the plurality of viewpoints are 9 to 12 viewpoints, the intermediate image of 1 to 4 viewpoints and the first line to the 4th viewpoint connected to the intermediate image of 1 to 4 viewpoints on the first tile of each line. The intermediate images of 5 to 8 viewpoints on the tiles in the row, and the intermediate images of the remaining viewpoints on the tiles arranged in the first to fourth rows connected to the intermediate images of the 9 to 12 viewpoints,
Place and
The intermediate image generation method according to claim 5, wherein when the plurality of viewpoints are 13 viewpoints or more, a part or all of the intermediate images are similarly arranged on tiles in each row in order from one viewpoint.
Instead of the arrangement rule, the position of subpixels constituting the RGBY staircase arrangement pixel unit of the stereoscopic image and the position of subpixels constituting the RGBY parallel arrangement pixel unit of the intermediate image for each of the plurality of viewpoints. 2. The intermediate according to claim 1, wherein the RGBY parallel arrangement pixel unit is generated by arranging subpixels constituting the RGBY staircase arrangement pixel unit with reference to an intermediate image generation table that is created in advance and associated with each other. Image generation method.
In the case where each of the plurality of viewpoint images and the stereoscopic image have the same aspect ratio,
Among the RGBY staircase arrangement pixel units from the 1 viewpoint to the N viewpoints constituting the RGBY staircase arrangement pixel block,
Corresponding R value, G value, B value, and Y value of each of the sub-pixels constituting the RGBY staircase-arranged pixel unit at a predetermined reference viewpoint to the positions of the sub-pixels constituting the RGBY staircase-arranged pixel unit And interpolating from the RGBY values of the sub-pixels constituting the pixel unit arranged in the vicinity of the corresponding position in the viewpoint image of the reference viewpoint,
The R value, G value, B value, and Y value of each of the sub-pixels constituting the RGBY staircase-arranged pixel unit of the viewpoint other than the reference viewpoint are determined as the sub-pixels constituting the RGBY-stair-arrangement pixel unit of the reference viewpoint. It is obtained by interpolating from RGBY values of the viewpoint image of the sub-pixels constituting at least one or more pixel units arranged in the vicinity of the corresponding position in the viewpoint image of the viewpoint other than the reference viewpoint corresponding to the arrangement position. The intermediate image generation method according to claim 1.
An intermediate image generation device for generating a plurality of intermediate images by the method according to claim 1,
The intermediate image generation device includes:
At least a central processing unit and a storage device,
The central processing unit
An RGBY staircase-arranged pixel unit in which subpixels are arranged in four rows with diagonally touching corners is arranged repeatedly, and RGBY staircase-arranged pixel blocks in which the 1st to Nth viewpoints are connected in the horizontal direction are repeatedly arranged. In order to generate a stereoscopic image, the R value, the G value, the B value, and the Y value of each of the subpixels constituting the RGBY staircase arrangement pixel unit are set as the arrangement positions of the subpixels constituting the RGBY staircase arrangement pixel unit. Corresponding, obtained by interpolating from RGBY values of subpixels constituting at least one pixel unit arranged in the vicinity of the corresponding position in the plurality of viewpoint images stored in the storage device,
An RGBY parallel arrangement pixel unit in which subpixels constituting the RGBY staircase arrangement pixel unit are arranged in the horizontal direction in the order of RGBY is arranged in accordance with an arrangement rule for collectively arranging the plurality of viewpoints,
Generating an intermediate image for each of the plurality of viewpoints composed of the RGBY staircase-arranged pixel units of the stereoscopic image and the RGBY parallel-arranged pixel units having the same total number or the total number of sub-pixels constituting each. An intermediate image generating apparatus characterized by the above.
A method for generating a stereoscopic image from a plurality of intermediate images generated by the method according to claim 1,
A stereoscopic image is generated by arranging subpixels constituting the RGBY parallel arrangement pixel unit as the RGBY step arrangement pixel unit from the intermediate images for each of the plurality of viewpoints according to the reverse order of the arrangement rule. Image generation method.
The three-dimensional image generation method according to claim 11, wherein
Instead of the arrangement rule, the position of subpixels constituting the RGBY parallel arrangement pixel unit of the intermediate image for each of the plurality of viewpoints and the position of subpixels constituting the RGBY step arrangement pixel unit of the stereoscopic image Refer to the stereoscopic image generation table created in advance,
A stereoscopic image generation method, wherein subpixels constituting the RGBY parallel arrangement pixel unit are arranged as the RGBY staircase arrangement pixel unit.
A stereoscopic image generating apparatus for generating a stereoscopic image from a plurality of intermediate images by the method according to claim 11,
The stereoscopic image generating device
At least a central processing unit and a storage device,
The central processing unit
Storing an intermediate image for each of the plurality of viewpoints in the storage device;
A stereoscopic image is generated by arranging subpixels constituting the RGBY parallel arrangement pixel unit as the RGBY step arrangement pixel unit from the intermediate image for each of the plurality of viewpoints according to the reverse order of the arrangement rule. Image generation device.
At least a central processing unit, a storage device, a compression device, and a transmission device are used to generate a stereoscopic image converted from a plurality of viewpoint images photographed and / or drawn from a plurality of viewpoints from one viewpoint to N viewpoints. A first information processing device for generating a plurality of intermediate images;
A stereoscopic image generation system comprising: a second information processing device that generates a stereoscopic image from the plurality of intermediate images, comprising at least a central processing unit, a storage device, a decompression device, and a reception device;
The central processing unit of the first information processing device is:
RGBY staircase-arranged pixel blocks in which RGBY subpixels are arranged in 4 rows by diagonally touching corners at RGB corners, and RGBY staircase-arranged pixel blocks are arranged repeatedly in a horizontal direction from the 1 viewpoint to the N viewpoint. In order to generate the stereoscopic image,
The first information corresponding to the arrangement position of the sub-pixels constituting the RGBY staircase arrangement pixel unit, the R value, the G value, the B value, and the Y value of each of the subpixels constituting the RGBY staircase arrangement pixel unit The RGBY staircase-arranged pixel unit is configured by interpolating from RGBY values of sub-pixels constituting at least one pixel unit arranged in the vicinity of the corresponding position in the plurality of viewpoint images stored in the storage device of the processing device. The RGBY parallel arrangement pixel units in which the subpixels to be arranged in the order of RGBY in the horizontal direction are arranged according to an arrangement rule for collectively arranging the plurality of viewpoints,
Generating an intermediate image for each of the plurality of viewpoints composed of the RGBY staircase-arranged pixel units of the stereoscopic image and the RGBY parallel-arranged pixel units having the same total number or the number of pixels constituting each of the RGBY staircase-arranged pixel units; The intermediate image for each viewpoint is compressed by the compression device and transmitted to the second information processing device by the transmission device,
The central processing unit of the second information processing device is:
The intermediate images for each of the plurality of viewpoints transmitted from the first information processing device are received by the receiving device, the plurality of intermediate images are decompressed by a decompressing device, and the plurality of decompressed by the decompressing device A three-dimensional image generation system characterized in that sub-pixels constituting the RGBY parallel arrangement pixel unit are arranged as the RGBY step arrangement pixel unit from the intermediate image for each viewpoint according to the reverse order of the arrangement rule to generate the three-dimensional image. .
An intermediate image generation method for generating a plurality of intermediate images used for generating a stereoscopic image converted from a plurality of viewpoint images photographed and / or drawn from a plurality of viewpoints from one viewpoint to N viewpoints,
The N viewpoints are 3n + 1 (n is a natural number) viewpoints,
RGB staircase-arranged pixel units in which RGB subpixels are arranged in 3 rows with diagonally touching corners in an oblique direction, and RGB staircase-arranged pixel blocks in which the 1 to N viewpoints are connected in a horizontal direction are repeatedly arranged. In order to generate the stereoscopic image,
Corresponding positions in the plurality of viewpoint images in which the R value, G value, and B value of each of the sub-pixels constituting the RGB staircase-arranged pixel unit correspond to the arrangement positions of the sub-pixels constituting the RGB staircase-arranged pixel unit Obtained by interpolating from RGB values of subpixels constituting at least one or more pixel units arranged in the vicinity,
An RGB parallel arrangement pixel unit in which subpixels constituting the RGB staircase arrangement pixel unit are arranged in the order of R, G, and B in the horizontal direction is arranged according to an arrangement rule for collectively arranging the plurality of viewpoints. By generating the intermediate image for each of a plurality of viewpoints,
The total number of the RGB staircase arranged pixel units of the stereoscopic image and the RGB parallel arranged pixel units of the plurality of intermediate images, or the total number of subpixels constituting each,
Further, the plurality of intermediate images are arranged in a plurality of belts divided by the number of viewpoints in the horizontal direction as image frames.
An intermediate image generation method characterized by the above.
An intermediate image generation method for generating a plurality of intermediate images used for generating a stereoscopic image converted from a plurality of viewpoint images photographed and / or drawn from a plurality of viewpoints from one viewpoint to N viewpoints,
The N viewpoints are 4n + 1 (n is a natural number) viewpoints,
RGBY staircase-arranged pixel units in which RGBY sub-pixels are arranged in 4 rows diagonally in contact with corners in an oblique direction, and RGBY staircase-arranged pixel blocks in which the 1 to N viewpoints are connected in a horizontal direction are repeatedly arranged. In order to generate the stereoscopic image,
The plurality of viewpoint images in which the R value, G value, B value, and Y value of each of the sub-pixels constituting the RGBY staircase-arranged pixel unit correspond to the arrangement positions of the sub-pixels that constitute the RGBY staircase-arranged pixel unit. And interpolating from RGBY values of sub-pixels constituting at least one pixel unit arranged in the vicinity of the corresponding position in
An RGBY parallel arrangement pixel unit in which subpixels constituting the RGBY staircase arrangement pixel unit are arranged in the order of R, G, B, and Y in the horizontal direction is arranged according to an arrangement rule for collectively arranging the plurality of viewpoints. By generating the intermediate image for each of the plurality of viewpoints,
The total number of the RGBY staircase arranged pixel units of the stereoscopic image and the RGBY parallel arranged pixel units of the plurality of intermediate images, or the total number of sub-pixels constituting each,
Further, the plurality of intermediate images are arranged in a plurality of belts divided by the number of viewpoints in the horizontal direction as image frames.
An intermediate image generation method characterized by the above.
An intermediate image generation method for generating a plurality of intermediate images used for generating a stereoscopic image converted from a plurality of viewpoint images photographed and / or drawn from a plurality of viewpoints from one viewpoint to N viewpoints,
The N viewpoints are (3 + m) n + 1 (n and m are natural numbers) viewpoints,
A staircase in which staircase-arranged pixel units in which RGB and other m sub-pixels are arranged in 3 + m rows in contact with corners in an oblique direction are connected in a horizontal direction from the 1 viewpoint to the N viewpoint. In order to generate the stereoscopic image by repeatedly arranging the arrangement pixel blocks,
The R value, G value, B value, and other color values of each of the sub-pixels constituting the staircase-arranged pixel unit correspond to the arrangement positions of the sub-pixels constituting the staircase-arranged pixel unit. Obtained by interpolating from the values of sub-pixels constituting at least one or more pixel units arranged in the vicinity of the corresponding position in the viewpoint image,
A parallel arrangement pixel unit in which the subpixels constituting the staircase arrangement pixel unit are arranged in the order of R, G, B, and other colors in the horizontal direction is arranged according to an arrangement rule for arranging the plurality of viewpoints collectively. By generating the intermediate image for each of the plurality of viewpoints,
The total number of the staircase arranged pixel units of the stereoscopic image and the parallel arranged pixel units of the plurality of intermediate images, or the total number of subpixels constituting each,
Further, the plurality of intermediate images are arranged in a plurality of belts divided by the number of viewpoints in the horizontal direction as image frames.
An intermediate image generation method characterized by the above.
The intermediate image generation method according to claim 1, the intermediate image generation apparatus according to claim 10, and the intermediate image generation apparatus according to claim 11, wherein the subpixel includes any of RGBC, RGBM, and RGBW instead of RGBY. The stereoscopic image generation method according to claim 12, the stereoscopic image generation device according to claim 13, the stereoscopic image generation system according to claim 14, and the intermediate image generation method according to claim 16.
The intermediate image generation method according to claim 1, wherein the sub-pixel is formed by adding at least one of RGB among C, M, Y, W, or other colors. An intermediate image generation apparatus according to claim 11, a stereoscopic image generation method according to any one of claims 11 to 12, a stereoscopic image generation apparatus according to claim 13, and a stereoscopic image generation system according to claim 14.
An intermediate image generation method for generating a plurality of intermediate images used for generating a stereoscopic image converted from a plurality of viewpoint images photographed and / or drawn from a plurality of viewpoints from one viewpoint to N viewpoints,
A pixel unit in which sub-pixels composed of RGB or RGB and at least one of C, M, Y, W, or other colors are connected in the vertical direction is arranged from the one viewpoint in the horizontal direction. In order to repeatedly arrange pixel blocks arranged up to N viewpoints and generate the stereoscopic image,
At least one or more pixel units arranged in the vicinity of corresponding positions in the plurality of viewpoint images, each value of the sub-pixels constituting the pixel unit corresponding to the arrangement position of the sub-pixels constituting the pixel unit. Interpolated from the subpixel values to be
A parallel arrangement pixel unit in which sub-pixels constituting the pixel unit are arranged in a horizontal direction is arranged according to an arrangement rule for arranging the sub-pixels for each of the plurality of viewpoints, and the intermediate image for each of the plurality of viewpoints is generated. By
A method for generating an intermediate image, characterized in that the total number of pixel units of the stereoscopic image and the plurality of intermediate image pixel units arranged in parallel, or the total number of sub-pixels constituting each unit is the same.