JP2015043187A - Image generation device and image generation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a user to perceive depth feeling without deterioration of resolution.SOLUTION: An image generation device 10 comprises: a depth reproduction range calculation unit 103 for calculating a depth side maximum depth perception position on the basis of depth perception information; a depth side calculation unit 104 for calculating a first element image by a subject existing in a range from a virtual imaging surface to the depth side maximum depth perception position on the basis of subject information and camera information, in a condition that virtual imaging lenses are disposed on positions corresponding to respective lenses of a lens array of an IP stereoscopic graphic display device 20 with a virtual arrangement in which the virtual imaging surface is disposed in a depth side of the virtual imaging lenses; a maximum depth side calculation unit 105 for calculating a second element image of a subject existing in a depth side of the depth side maximum depth perception position with the virtual arrangement, on the basis of the subject information, simple eye camera information and the camera information; and a depth synthesis unit 108.

Description

  The present invention relates to an image generation apparatus and an image generation program.

  In conventional integral photography (IP), a subject is photographed with a single camera through a lens array in which a plurality of microlenses (also referred to as element lenses) are arranged. In this case, since the camera takes an image of the focal plane of the lens array, each minute lens functions in the same manner as the minute camera. As a result, an image obtained by imaging the subject through the lens array is an image (hereinafter referred to as an element image group) in which minute images (hereinafter referred to as element images) corresponding to the position of the minute lens are arranged. . This elemental image group is an image in which light information from a subject is recorded. Therefore, the number of rays that can be recorded in the element image group depends on the resolution of the camera. Further, the resolution of the reproduced image in the integral photography system depends on the number of light beams to be reproduced, that is, the resolution of the camera. In this way, by using a camera array in which a plurality of micro cameras are arranged closely instead of a single camera and lens array, it is possible to realize a large number of pixels in the imaging unit.

  Meanwhile, a technique for generating an element image of an IP stereoscopic image by computer graphics (CG) has been studied. According to this technology, a computer reproduces a three-dimensional subject and set in a virtual space so that they operate in the same way as in the real world, and generates an element image by tracking rays from the subject in the virtual space. To do. In the generation of an element image by the technique using the CG, a calculation method for tracing light rays from a subject through a depth control lens for controlling light rays from the subject in order to generate an image that protrudes from the display surface of the display device Is known (see, for example, Non-Patent Document 1).

Spyros S. Athineos et al., “Physical modeling of a microlens array setup for use in computer generated IP”, Proc. Of SPIE-IS & T Electronic Imaging, Vol. 5664 pp. 472-479, 2005

As described above, in the conventional element image generation method using CG, a three-dimensional subject and set are reproduced in a virtual space, as in the real world. On the other hand, in the IP system, the resolution of the three-dimensional image on the near side and the far side from the display surface of the display device decreases as the depth increases. For this reason, there is a problem that even when attempting to reproduce in the same manner as in the real world, the resolution is too low to perceive the depth of the stereoscopic image.
The problem to be solved by the present invention is to provide an image generation apparatus and an image generation program that allow a viewer to perceive a sense of depth and prevent deterioration of the resolution of an IP stereoscopic image.

  [1] In order to solve the above-described problem, an image generation apparatus according to one aspect of the present invention is an image generation apparatus that generates an element image for an integral photography stereoscopic video display apparatus, and includes an object in a virtual space. A subject information storage unit for storing subject information representing a position and a shape; first camera information representing a first position of the virtual camera array in the virtual space; and the virtual space provided in front of the virtual camera array. A virtual camera array information storage unit for storing monocular camera information representing the position of the virtual monocular camera and depth perception information representing the degree of perception of the depth of the integral photography stereoscopic image, and depth for reproducing the three-dimensional information of the subject The depth side maximum depth perception position in the reproduction range is set as the depth perception information stored in the virtual camera array information storage unit. And a virtual imaging lens is disposed at a position corresponding to each lens of the lens array of the integral photography stereoscopic image display device, and a virtual imaging surface is located on the back side of the virtual imaging lens. A first element image of a subject existing from the position of the virtual imaging surface to the depth side maximum depth perceived position calculated by the depth reproduction range calculation unit is assumed to be the arranged virtual arrangement, and the subject information storage unit From the back side maximum depth perceived position as the virtual arrangement, and a back side calculation unit that calculates based on the subject information stored in the virtual camera array information storage unit and the first camera information stored in the virtual camera array information storage unit The second element image of the subject existing on the back side is also converted into subject information stored in the subject information storage unit and a monocular stored in the virtual camera array information storage unit. A maximum back side calculation unit that is calculated based on the camera information and the first camera information, the first element image that is calculated by the back side calculation unit, and the second value that is calculated by the maximum back side calculation unit. A back side synthesis unit that synthesizes the element image.

[2] In the image generation device according to [1], the depth reproduction range calculation unit may convert a front maximum depth perception position in the depth reproduction range to the depth perception information stored in the virtual camera array information storage unit. A third elemental image of a subject existing from the position of the virtual imaging lens to the front maximum depth perceived position calculated by the depth reproduction range calculation unit as the virtual arrangement is calculated based on the subject. A front-side calculation unit that calculates based on subject information stored in the information storage unit and first camera information stored in the virtual camera array information storage unit; and the front-side maximum depth perception position as the virtual arrangement The fourth element image of the subject existing in front of the subject information is stored in the subject information stored in the subject information storage unit and the virtual camera array information storage unit A maximum front side calculation unit that is calculated based on the obtained monocular camera information and the first camera information, the third element image that is calculated by the front side calculation unit, and the fourth element that is calculated by the maximum front side calculation unit. The image processing apparatus further includes a front side synthesis unit that synthesizes the element image, and a synthesis unit that synthesizes the element image obtained by the synthesis of the back side synthesis unit and the element image obtained by synthesis of the front side synthesis unit.
[3] In the image generating device according to [2] above, the virtual camera array information storage unit is a second camera that represents a second position in front of the first position of the virtual camera array in the virtual space. Information is further stored, and the maximum depth side calculation unit, the second element image, the subject information stored in the subject information storage unit and the second camera information stored in the virtual camera array information storage unit The maximum front side calculation unit calculates the fourth element image from subject information stored in the subject information storage unit and second camera information stored in the virtual camera array information storage unit. Based on and.
[4] In the image generating device according to any one of [2] and [3], the back side synthesis unit synthesizes the second element image by overwriting the first element image, and the front side The combining unit combines the third element image by overwriting the fourth element image, and the element obtained by combining the element image obtained by combining the back side combining unit with the element image obtained by the front side combining unit. Synthesize by overwriting the image.

  [5] In order to solve the above-described problem, an image generation program according to an aspect of the present invention includes a subject information storage unit that stores subject information representing the position and shape of a subject in a virtual space, and a virtual camera in the virtual space. The first camera information representing the first position of the array, the monocular camera information representing the position of the virtual monocular camera provided in front of the virtual camera array in the virtual space, and the perception of the depth of the integral photography stereoscopic image. A virtual camera array information storage unit for storing depth perception information representing the degree, and the virtual camera array information indicating the maximum depth perception position on the depth side in the depth reproduction range for reproducing the three-dimensional information of the subject. A depth reproduction range calculation unit for calculating based on the depth perception information stored in the storage unit; The position of the virtual imaging surface is assumed to be a virtual arrangement in which a virtual imaging lens is arranged at a position corresponding to each lens of the lens array of the photographic stereoscopic image display device, and a virtual imaging surface is arranged on the back side of the virtual imaging lens. To the subject information stored in the subject information storage unit and the virtual camera array information storage unit, the first element image by the subject existing up to the depth side maximum depth perception position calculated by the depth reproduction range calculation unit. A back side calculation unit that calculates based on the stored first camera information, the second element image of the subject existing on the back side from the back side maximum depth perceived position as the virtual arrangement, Maximum back side calculation calculated based on the subject information stored in the subject information storage unit, the monocular camera information stored in the virtual camera array information storage unit, and the first camera information , Said first element images the far side calculation unit has calculated, the far side combining unit where the maximum back side calculating section combines the second image element is calculated, to function as a.

  According to the present invention, it is possible to make a viewer perceive a sense of depth and to prevent degradation of the resolution of an IP stereoscopic image.

It is a graph which shows the sharpness and the resolution with respect to the depth of an IP stereoscopic image. It is a graph which shows depth perception with respect to the depth of IP stereoscopic image. 1 is a block diagram illustrating a schematic functional configuration of an IP stereoscopic video display system to which an image generation apparatus according to a first embodiment is applied. It is a disassembled schematic diagram which shows the structure of IP stereoscopic video display apparatus. It is a schematic diagram explaining operation | movement of a back side calculation part and a maximum back side calculation part. It is a schematic diagram explaining operation | movement of a back side calculation part and a maximum back side calculation part. 3 is a schematic flowchart illustrating a processing procedure of the image generation apparatus according to the first embodiment. It is a schematic diagram explaining operation | movement of a back side calculation part. It is a flowchart explaining operation | movement of a front side calculation part. It is a schematic diagram explaining operation | movement of the front side calculation part. It is a schematic diagram explaining operation | movement of the largest back side calculation part in 2nd Embodiment. It is a schematic diagram explaining operation | movement of the largest back | inner side calculation part in the 1st modification of 2nd Embodiment. It is a schematic diagram explaining arrangement | positioning of the 2nd virtual camera in the 2nd modification of 2nd Embodiment.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.
[First Embodiment]
Physiological factors perceiving depth by a viewer viewing a display screen on which video is displayed, such as a television, are mainly adjustment, convergence, and binocular parallax. On the other hand, when the integral photography (IP) display device does not have sufficient resolution, the resolution of the IP stereoscopic image increases as the distance from the display surface of the display device (focal plane of the lens array) in the optical axis direction increases. And sharpness decreases. First, this phenomenon will be described.

  FIG. 1 is a graph showing the sharpness and resolution with respect to the depth of an IP stereoscopic image. Specifically, the graph of the figure shows an IP stereoscopic image display device in which a lens array is combined with a liquid crystal display device having a resolution of 458 (dot per inch; dpi) when viewed from a location 60 cm away from the display surface of the liquid crystal display device. It shows the sharpness of the IP stereoscopic image by subjective evaluation and the limit resolution in calculation when viewed. In the graph of the figure, the horizontal axis indicates the presentation position of the IP stereoscopic image with reference to the display surface. The negative direction on the horizontal axis is the rear when viewed from the viewer, and the positive direction is the front. In the graph of the figure, the vertical axis on the left is an evaluation value of sharpness normalized to a value from “0” to “1”. In the graph of the figure, the right vertical axis represents the spatial frequency (cycle per degree; cpd) of the IP stereoscopic image. In the graph of the figure, a “◯” marker is an evaluation value of the sharpness when an IP stereoscopic image is presented every 3 cm from the rear 18 cm to the front 18 cm on the display surface. A broken line is a resolution characteristic in the IP system.

  As is apparent from the graph of FIG. 1, the sharpness evaluation result decreases with increasing distance from the display surface, as with the resolution characteristics in the IP method. That is, the viewer perceives the IP stereoscopic image away from the display surface as a blurred stereoscopic image with reduced resolution.

  For example, under viewing conditions with a viewing distance of 10 m or less, depth perception due to binocular parallax becomes the greatest physiological factor. In other words, even if a stereoscopic image presented at a distance where depth is not perceived due to physiological factors of depth perception, including binocular parallax, is replaced with a two-dimensional image, the viewer can obtain the same depth feeling as before replacement. It can be said.

  FIG. 2 is a graph showing depth perception with respect to the depth of the IP stereoscopic image. Specifically, the graph in the figure shows an IP stereoscopic image display device in which a lens array having the same specifications is combined with each of liquid crystal display devices having resolutions of 458 (dpi), 229 (dpi), and 153 (dpi). The depth perception of the IP stereoscopic image by subjective evaluation when viewed from a location 60 cm away from the display surface of the apparatus is shown. In the graph of the figure, the horizontal axis indicates the presentation position of the IP stereoscopic image with reference to the display surface, as in FIG. In the graph of FIG. 2, the vertical axis represents the depth perception evaluation value normalized to a value from “−1” to “1”. In the graph of the figure, the “◯” marker indicates the depth perception evaluation when an IP stereoscopic image is presented every 3 cm from the rear 18 cm to the front 18 cm on the display surface of the liquid crystal display device having a resolution of 458 (dpi). Value. The “Δ” marker is a depth perception evaluation value when the resolution of the liquid crystal display device is 229 (dpi). The “x” marker is an evaluation value of depth perception when the resolution of the liquid crystal display device is 153 (dpi).

  According to the graph of FIG. 2, at any resolution, the change rate of the depth perception becomes smaller as the presentation position of the IP stereoscopic image moves away from the display surface. In other words, the depth perception perceived by the viewer tends to weaken as the presentation position of the IP stereoscopic image moves away from the display surface regardless of the resolution of the display device. Moreover, it can be said that the depth perception at an arbitrary presentation position is larger (the sense of depth is stronger) as the resolution is higher.

  In the graph of FIG. 2, when the perceivable range (depth perceptible range) of the IP stereoscopic image is obtained by a significant difference test with a risk rate of 5%, the resolution 458 (dpi) is 15 cm behind the display surface and 15 cm ahead of the display surface. In the resolution 229 (dpi), the display surface is 12 cm behind the display surface 12 cm in front, and in the resolution 153 (dpi) is 12 cm behind the display surface 6 cm in front of the display surface. That is, the result that the perceptible range of depth becomes wider as the resolution becomes higher is obtained.

  According to FIGS. 1 and 2, the image quality (for example, sharpness) changes according to the presentation position of the IP stereoscopic image, but the depth perception tends to be gradually saturated until reaching a certain depth position. It can also be seen that the depth distance at which the depth perception is saturated is shorter as the resolution of the display device is lower. That is, when the presentation position of the IP stereoscopic image is away from the display surface, the deterioration of the image quality is easily perceived, but the change in the depth position is difficult to perceive.

  Therefore, in the first embodiment, the position where the depth is not perceived is set as the maximum depth perceived position, and the subject at the position where the depth is not perceived is displayed as a two-dimensional image at the maximum depth perceived position. Improve subjective (apparent) image quality for subjects in unperceived places.

  FIG. 3 is a block diagram illustrating a schematic functional configuration of the IP stereoscopic video display system to which the image generation apparatus according to the first embodiment is applied. As shown in FIG. 1, the IP stereoscopic video display system 1 includes an image generation device 10 and an IP stereoscopic video display device 20. The IP stereoscopic video display device 20 is a display device that displays IP stereoscopic video. The image generation device 10 generates an element image group by computer graphics (CG) for the IP stereoscopic image display device 20, and displays an IP image signal obtained by arranging the element image group in time series as an IP stereoscopic image display. Input to the device 20.

  The image generation apparatus 10 includes a virtual camera array information storage unit 101, a subject information storage unit 102, a depth reproduction range calculation unit 103, a back side calculation unit 104, a maximum back side calculation unit 105, and a front side calculation unit 106. A maximum front side calculation unit 107, a back side synthesis unit 108, a front side synthesis unit 109, a synthesis unit 110, and a video signal generation unit 111.

  The virtual camera array information storage unit 101 includes first camera information representing a first position of the first virtual camera array in the virtual space, and a virtual monocular provided in front of the first virtual camera array in the virtual space. Monocular camera information representing the camera position is stored. In the first embodiment, the first camera information includes information representing the position of the principal point of each virtual imaging lens in the first virtual camera array and information representing the position of the virtual imaging surface corresponding to each principal point. Including. The monocular camera information includes information indicating the position of the principal point of the virtual monocular imaging lens in the virtual monocular camera and information indicating the position of the virtual monocular imaging surface corresponding to the principal point. The position of the principal point of each virtual imaging lens represented by the first camera information stored in the virtual camera array information storage unit 101 is determined when the IP stereoscopic video display device 20 is arranged in a virtual space for calculating an element image. This corresponds to the position of the principal point of each lens constituting the lens array of the IP stereoscopic image display device 20. In addition, the position of each virtual imaging surface represented by the first camera information is the position of the display surface on which the element image is displayed by the IP stereoscopic video display device 20 when the IP stereoscopic video display device 20 is arranged in the virtual space. Correspond. In other words, the virtual imaging surface is located at a location away from the virtual imaging lens by the focal length of the lenses constituting the lens array of the IP stereoscopic image display device 20. Note that the position of the principal point of the virtual lens and the position of the virtual imaging surface may change with the passage of time in the virtual space.

  In addition, the virtual camera array information storage unit 101 stores depth perception information indicating the degree of perception (depth perception) with respect to the depth of an IP stereoscopic image. Specifically, the virtual camera array information storage unit 101 evaluates the depth perception with respect to the depth distance from the display surface of the IP stereoscopic image according to the resolution of the IP stereoscopic video display device 20 (for example, the graph shown in FIG. 2). Data).

  The subject information storage unit 102 stores, for example, information representing the position, shape, color, luminance, and optical characteristics of the subject in the virtual space. For example, a polygon or a free-form surface may be used as the position and shape of the subject in the virtual space, CAD (Computer Assisted Drafting) data may be used, or other forms may be used. Note that the position, shape, color, luminance, and optical characteristics of the subject in the virtual space may change with the passage of time in the virtual space.

  Based on the depth perception information stored in the virtual camera array information storage unit 101, the depth reproduction range calculation unit 103 calculates a depth reproduction range that reproduces the three-dimensional information of the subject. Specifically, the depth reproduction range calculation unit 103 calculates the front maximum depth perception position and the depth maximum depth perception position in the depth direction based on the depth perception information. The front maximum depth perception position is a limit position of the depth reproduction range located on the near side of each virtual imaging lens of the first virtual camera array. The depth-side maximum depth perception position is a limit position of the depth reproduction range that is located on the back side of each virtual imaging surface of the first virtual camera array. More specifically, for example, the depth reproduction range calculation unit 103 reads the depth perception information from the virtual camera array information storage unit 101, and calculates the depth perceivable range based on the depth perception information by a significant difference test with a risk level of 5%. Then, the front side maximum depth perception position and the back side maximum depth perception position are obtained. The depth reproduction range calculation unit 103 supplies information indicating the depth side maximum depth perception position to the depth side calculation unit 104 and the maximum depth side calculation unit 105. Further, the depth reproduction range calculation unit 103 supplies information indicating the front maximum depth perception position to the front calculation unit 106 and the maximum front calculation unit 107.

  The front calculation unit 106 captures information indicating the front maximum depth perception position supplied by the depth reproduction range calculation unit 103, and includes a front element image (a third element image) of a subject existing between the position of the virtual imaging lens and the front maximum depth perception position. Element image) is calculated based on the subject information stored in the subject information storage unit 102 and the first camera information stored in the virtual camera array information storage unit 101. The front side element image is an element image of a subject located on the front side of the virtual imaging lens of the first virtual camera and on the back side of the front maximum depth perception position. The front calculation unit 106 determines that the pixel value of each pixel of the front element image is such that a straight line passing through the pixel on the virtual imaging plane and the principal point of the virtual imaging lens intersects the subject on the near side of the principal point of the virtual imaging lens. Of the points to be calculated, the calculation is performed with reference to information on the point farthest from the main point of the virtual imaging lens. This is a process opposite to the normal hidden surface process. In the first embodiment, as an example of a method for calculating a pixel value with reference to information about the farthest point, the color of the subject at the farthest point is read from the subject information storage unit 12, and a value representing the color is obtained from the pixel. Use the value method. The front calculation unit 106 calculates a front element image group for one frame.

  Note that, as described above, the front side calculation unit 106 may use a depth map when the value representing the color of the point farthest from the principal point of the virtual imaging lens is used as the pixel value. First, the front side calculation unit 106 is “1.0” when it is at infinity from the virtual imaging surface or the virtual imaging lens, and becomes a value that decreases as it approaches the virtual imaging surface or the virtual imaging lens (however, from “0”). (Large value) is assigned to the surface of the subject as a depth value. The front calculation unit 106 scans the surface of the subject and sets the pixel value representing the color of the surface as the pixel value of each pixel of the front element image.

  In normal hidden surface processing, the depth value is initialized to “1.0”, and only when the depth value of the subject from which the pixel value is acquired is smaller than the depth value in the depth map of the pixel to which the pixel value is to be written. Update with the acquired pixel value. However, in the first embodiment, since processing opposite to the normal hidden surface processing is performed, the front calculation unit 106 sets all the depth values of the depth map having depth values corresponding to the respective pixels of the front element image to “0”. ”In advance. The front calculation unit 106 acquires the pixel value of the pixel in the front element image only when the depth value of the subject from which the pixel value has been acquired is greater than the depth value in the depth map of the pixel to which the pixel value is to be written. The pixel value is updated, and the depth value of the pixel in the depth map is updated with the depth value of the subject.

  The maximum front side calculation unit 107 takes in information indicating the front maximum depth perception position supplied by the depth reproduction range calculation unit 103, and the maximum front side element image (fourth element) of the subject existing in front of the front maximum depth perception position Image) is calculated based on the subject information stored in the subject information storage unit 102, the monocular camera information stored in the virtual camera array information storage unit 101, and the first camera information. The maximum front side calculation unit 107 calculates a maximum front side element image group for one frame.

  The depth side calculation unit 104 takes in information indicating the depth side maximum depth perceived position supplied by the depth reproduction range calculation unit 103, and the depth side element image by the subject existing from the position of the virtual imaging plane to the depth side maximum depth perceived position. (First element image) is calculated based on the subject information stored in the subject information storage unit 102 and the first camera information stored in the virtual camera array information storage unit 101. The back side element image is an element image by a subject located on the back side of the virtual imaging surface of the first virtual camera and on the front side of the back side maximum depth perception position. The back side calculation unit 104 calculates a light ray passing through the principal point of the virtual imaging lens after passing through a certain pixel on the virtual imaging surface by a known method (for example, ray tracing method, photon mapping method), and A value representing a color is a pixel value of the pixel in the element image. The back side calculation unit 104 calculates a back side element image group for one frame.

  The maximum depth side calculation unit 105 captures information indicating the depth side maximum depth perception position supplied by the depth reproduction range calculation unit 103, and the maximum depth side element image of the subject existing on the back side from the depth side maximum depth perception position. A (second element image) is calculated based on the subject information stored in the subject information storage unit 102, the monocular camera information stored in the virtual camera array information storage unit 101, and the first camera information. The maximum back side calculation unit 105 calculates a maximum back side element image group for one frame.

The front synthesis unit 109 superimposes the maximum front element image calculated by the maximum front calculation unit 107 on the front element image calculated by the front calculation unit 106 and generates a pre-combination element image. The front synthesis unit 109 generates a pre-combination element image group for one frame. However, the maximum front element image that the front combining unit 109 overlaps with the front element image corresponds to the same virtual camera. Specifically, the front side synthesis unit 109 overwrites the front side element image with a pixel value other than “0” in the maximum front side element image.
The front synthesis unit 109 generates a mask image in which the pixel value of the area where the subject exists in the maximum front element image is “1” and the pixel value of the area where the subject does not exist is “0”. The element image may be combined by overwriting the maximum front element image with the pixel value of the front element image of the pixel having a pixel value “0” in the image.

The back side synthesis unit 108 superimposes the back side element image calculated by the back side calculation unit 104 on the maximum back side element image calculated by the maximum back side calculation unit 105 and generates a composite back side element image. The back side synthesis unit 108 generates a composite back side element image group for one frame. However, the back side element image that the back side combining unit 108 overlaps with the maximum back side element image corresponds to the same virtual camera. Specifically, the back side synthesis unit 108 overwrites the maximum back side element image with a pixel value other than “0” in the back side element image.
Note that the back side synthesis unit 108 generates a mask image in which the pixel value of the area where the subject exists in the back side element image is “1” and the pixel value of the area where the subject does not exist is “0”. The element image may be synthesized by overwriting the pixel value of the maximum back element image of the pixel having the pixel value “0” in the mask image on the back element image.

The synthesizing unit 110 superimposes the synthesized pre-side element image obtained by synthesizing with the synthesized back side element image obtained by synthesizing by the back side synthesizing unit 108 and generates an element image. The synthesizing unit 110 generates a group of element images for one frame. However, the pre-combination element images that the combining unit 110 overlaps with the combined back element image correspond to the same virtual camera. Specifically, the synthesis unit 110 overwrites the synthesized back side element image with a pixel value other than “0” in the pre-combination side element image.
Note that the synthesizing unit 110 generates a mask image in which the pixel value of the region where the subject exists in the pre-combination side element image is “1” and the pixel value of the region where the subject does not exist is “0”. The element image may be synthesized by overwriting the pre-combination element image with the pixel value of the composition back side element image of the pixel whose pixel value is “0”.

  The video signal generation unit 111 generates an IP video signal in which the element image groups generated by the synthesis unit 110 are arranged in time series, and supplies the IP video signal to the IP stereoscopic video display device 20.

  FIG. 4 is an exploded schematic view showing the configuration of the IP stereoscopic video display device 20. As shown in the figure, the IP stereoscopic video display device 20 includes, for example, a display panel 210 that is an image display device such as a liquid crystal display or an organic EL (Electro-Luminescence) display, and a lens disposed in front of the display panel 210. Array 220. The display panel 210 displays an element image group including element images 21-1 to 21 -N according to the IP video signal input to the IP stereoscopic video display device 20.

  The lens array 220 includes convex lenses 22-1 to 22-N corresponding to the element images 21-1 to 21-N. The lens array 220 is installed on the front surface separated from the display panel 210 by the focal length of the convex lenses 22-1 to 22-N. As a result, the viewer can see only a part of each of the element images 21-1 to 21-N according to the viewpoint.

  In the first embodiment, the arrangement of the convex lenses 22-1 to 22-N and the arrangement of the element images 21-1 to 21-N are delta arrangement, but may be a square lattice arrangement or other arrangement. . In addition, the IP stereoscopic video display device 20 may display IP stereoscopic video such as a display panel 210 that includes a plurality of projectors each projecting an element image and a screen on which the element image is projected. For example, other configurations may be used.

  5 and 6 are schematic diagrams for explaining the operations of the back side calculation unit 104 and the maximum back side calculation unit 105. FIG. FIG. 5 is a top view of a virtual space for calculating an element image displayed by the IP stereoscopic video display device 20 toward the right-hand direction in the drawing. The subject O (O) 1 is located behind the virtual imaging plane P1 and before the virtual plane K including the maximum depth perceived position on the back side. Specifically, the subject O1 is calculated on the back side when the first virtual imaging area formed by the principal point L1 of the virtual imaging lens and the virtual imaging plane P1 (first virtual camera) is cut by the virtual plane K. It is in area A. The back side calculation unit 104 calculates a back side element image for the subject O1 by tracing the light passing through the principal point L1 of the virtual imaging lens after passing through the pixels on the virtual imaging surface P1.

  The subject O (O) 2 is behind the virtual plane K, that is, outside the depth reproduction range. Specifically, the subject O2 is provided with a principal point L ′ of the virtual monocular imaging lens at a point where the outermost rays of all the virtual imaging surfaces including the virtual imaging surface P1 intersect through the corresponding principal points, When the virtual monocular imaging plane P ′ is provided at a position corresponding to the principal point L ′, the virtual plane K of the virtual monocular imaging area formed by the principal point L ′ of the virtual monocular imaging lens and the virtual monocular imaging plane P ′. It is in the area B on the far side. In the first embodiment, the subject O2 is virtually imaged by the principal point L ′ of the virtual monocular imaging lens and the virtual monocular imaging surface P ′ (virtual monocular camera). Specifically, this will be described with reference to FIG.

  In FIG. 6, a symbol S indicates a virtual plate (texture) provided on the virtual plane K. The maximum back side calculation unit 105 copies the subject O2 virtually captured by the principal point L ′ of the virtual monocular imaging lens and the virtual monocular imaging plane P ′ onto the virtual plate S (texture mapping). In this state, the back side calculation unit 104, after passing through the pixels on the virtual imaging plane P1, reversely traces the light passing through the principal point L1 of the virtual imaging lens, thereby performing the back side element image for the subject O1 and the virtual plate S. Is calculated.

  Although not shown, when the subject O (O) 3 is in front of the virtual plane K ′ including the front-side maximum depth perception position, that is, outside the depth reproduction range, the maximum front-side calculation unit 107 performs the virtual monocular imaging lens. The subject O2 virtually imaged by the principal point L ′ and the virtual monocular imaging surface P ′ is copied onto the virtual plate S ′ provided on the virtual plane K ′. In this state, the front calculation unit 106 sets the pixel value of each pixel of the front element image so that the straight line passing through the pixel on the virtual imaging plane P1 and the main point L1 of the virtual imaging lens is in front of the main point L1. Of the points intersecting with O3, the calculation is performed with reference to the information regarding the point farthest from the main point L1.

FIG. 7 is a schematic flowchart showing a processing procedure of the image generation apparatus 10 according to the first embodiment.
In step S1, the depth reproduction range calculation unit 103, based on the depth perception information stored in the virtual camera array information storage unit 101, indicates the front maximum depth perception position (maximum) indicating the depth reproduction range for reproducing the three-dimensional information of the subject. Previous: Zf) and depth side maximum depth perception position (maximum depth: Zb) are calculated.
Next, the depth reproduction range calculation unit 103 supplies information indicating the depth side maximum depth perception position to the depth side calculation unit 104 and the maximum depth side calculation unit 105. Further, the depth reproduction range calculation unit 103 supplies information indicating the front maximum depth perception position to the front calculation unit 106 and the maximum front calculation unit 107.

  In step S <b> 2, the maximum depth calculation unit 105 takes in information indicating the depth maximum depth perception position supplied by the depth reproduction range calculation unit 103, and calculates the maximum of the subject existing on the back side from the depth maximum depth perception position. The back element image (Bim_max) is calculated based on the subject information stored in the subject information storage unit 102, the monocular camera information stored in the virtual camera array information storage unit 101, and the first camera information. The maximum back side calculation unit 105 calculates a maximum back side element image group for one frame.

  In step S <b> 3, the depth-side calculation unit 104 takes in information indicating the depth-side maximum depth perception position supplied by the depth reproduction range calculation unit 103, and depends on the subject existing from the position of the virtual imaging surface to the depth-side maximum depth perception position. A back side element image (Bim) is calculated based on the subject information stored in the subject information storage unit 102 and the first camera information stored in the virtual camera array information storage unit 101. The back side calculation unit 104 calculates a back side element image group for one frame.

  In step S <b> 4, the front calculation unit 106 takes in information indicating the front maximum depth perception position supplied by the depth reproduction range calculation unit 103, and the front element image by the subject existing from the position of the virtual imaging lens to the front maximum depth perception position. (Fim) is calculated based on the subject information stored in the subject information storage unit 102 and the first camera information stored in the virtual camera array information storage unit 101. The front calculation unit 106 calculates a front element image group for one frame.

  In step S5, the maximum front side calculation unit 107 takes in information indicating the front maximum depth perception position supplied by the depth reproduction range calculation unit 103, and the maximum front element image of the subject existing in front of the front maximum depth perception position ( Fim_max) is calculated based on the subject information stored in the subject information storage unit 102, the monocular camera information stored in the virtual camera array information storage unit 101, and the first camera information. The maximum front side calculation unit 107 calculates a maximum front side element image group for one frame.

  In step S <b> 6, the back side synthesis unit 108 superimposes the back side element image calculated by the back side calculation unit 104 on the maximum back side element image calculated by the maximum back side calculation unit 105, and combines them. (Bim_fin) is generated. Specifically, the back side synthesis unit 108 overwrites the maximum back side element image with a pixel value other than “0” in the back side element image. The back side synthesis unit 108 generates a composite back side element image group for one frame.

  In step S <b> 7, the front combining unit 109 combines the maximum front element image calculated by the maximum front calculating unit 107 with the front element image calculated by the front calculating unit 106 to generate a pre-combination element image (Fim_fin). . Specifically, the front side synthesis unit 109 overwrites the front side element image with a pixel value other than “0” in the maximum front side element image. The front synthesis unit 109 generates a pre-combination element image group for one frame.

In step S <b> 8, the composition unit 110 superimposes the composite pre-element image obtained by the front composition unit 109 on the composite back element image obtained by the composition by the back composition unit 108 and composes the element image. Generate. Specifically, the synthesis unit 110 overwrites the synthesized back side element image with a pixel value other than “0” in the pre-combination side element image. The synthesizing unit 110 generates a group of element images for one frame.
Next, the video signal generation unit 111 generates an IP video signal in which the element image groups generated by the synthesis unit 110 are arranged in time series, and supplies the IP video signal to the IP stereoscopic video display device 20.

  In the flowchart of FIG. 7, the process of step S2 executed by the maximum back side calculation unit 105 and the process of step S3 executed by the back side calculation unit 104 may be parallel or in series. Also good. Similarly, the process of step S4 executed by the front calculation unit 106 and the process of step S5 executed by the maximum front calculation unit 107 may be parallel or serial. Similarly, the processing group of step S2, step S3, and step S6 and the processing group of step S4, step S5, and step S7 may be parallel or serial.

  FIG. 8 is a schematic diagram for explaining the operation of the back side calculation unit 104. This figure is a top view of a virtual space for calculating an element image displayed by the IP stereoscopic video display device 20 in the direction (right hand direction) indicated by the display direction Ar1. The subjects O1 and O2 are on the back side of the virtual imaging surfaces P1 to P5, and the subject O3 is on the near side of the main points L1 to L5 of the virtual imaging lens. An example in which the back-side calculation unit 104 uses a ray tracing method that reversely tracks light passing through the principal point of the virtual imaging lens after passing through the pixels on the virtual imaging surface will be described as an example.

  For example, the arrow denoted by reference numeral R1 in FIG. 8 is a light beam that passes through a certain pixel on the virtual imaging plane P3 and then passes through the principal point L3 of the virtual imaging lens corresponding to the virtual imaging plane P3. The back side calculation unit 104 traces the light ray R1 in reverse until it hits something. Here, since it hits the subject O2, the back side calculation unit 104 determines the optical characteristics (reflectance, diffusivity, transmittance, refractive index, etc.) at the point P21 where the subject O2 hits from the subject information storage unit 102. read out. Then, the back side calculation unit 104 calculates the direction of the light ray R1 by reflection, diffusion, transmission, refraction, and the like at the abutted point based on the read characteristics.

  FIG. 8 shows an example in the case where specular reflection occurs at the point P21, and a region in which the light quantity and color of light are given in advance by tracing the light ray R2 that becomes the light ray R1 by reflection further reversely. The process of tracing back is repeated until reaching (for example, a light source). By doing in this way, the back side calculation part 104 can obtain | require the component of the light contained in the light ray R1, and can calculate the color of the light ray R1. The back side calculation unit 104 calculates all the back side element images by performing such processing on all the pixels on all the virtual imaging surfaces.

  FIG. 9 is a flowchart for explaining the operation of the front calculation unit 106. The flowchart shown in the figure is a flowchart of processing for generating a front element image for one frame. First, the front calculation unit 106 initializes all pixel values of the front element image for one frame (Sa1). Then, the front calculation unit 106 selects one of the virtual imaging planes that have not been selected so far as a virtual imaging plane to be processed later (Sa2). Next, the front-side calculation unit 106 selects one of the pixels on the virtual imaging surface to be processed from the unselected pixels as the subsequent processing target pixel (Sa3).

  Next, the front calculation unit 106 detects an intersection of a straight line passing through the pixel to be processed, the main point of the virtual imaging lens corresponding to the virtual imaging surface to be processed, and a subject in front of the virtual imaging lens. (Sa4). Next, the front calculation unit 106 determines whether or not an intersection is detected in step Sa4 (Sa5). When it is determined that no intersection has been detected (Sa5: NO), step Sa6 is skipped and the process proceeds to step Sa7.

  On the other hand, when it is determined in step Sa5 that an intersection has been detected (Sa5: YES), the front calculation unit 106 selects the detected intersection farthest from the virtual imaging lens. Then, the front calculation unit 106 reads a value indicating the color of the subject at the selected intersection from the subject information storage unit 12, sets the value as the pixel value of the target pixel (Sa6), and proceeds to step Sa7. In step Sa7, if there is an unselected pixel in step Sa3 on the virtual imaging surface to be processed, the process returns to step Sa3, and if not, the process proceeds to step Sa8.

  In step Sa8, if there is an unselected virtual imaging surface in step Sa2, the process returns to step Sa2, and if not, the process ends. Thereby, the pixel value of the pixel determined to have an intersection in step Sa5 becomes the value determined in step Sa6, and the pixel value of the pixel determined to have no intersection in step Sa5 is determined in step Sa1. It is an initialized value.

  FIG. 10 is a schematic diagram for explaining the operation of the front calculation unit 106. This figure is a schematic diagram for explaining the processing of steps Sa4 and Sa6 in the flowchart of FIG. FIG. 10 is a top view of a virtual space for calculating an element image to be displayed by the IP stereoscopic video display device 20 in the direction (right-hand direction) indicated by the display direction Ar1, similarly to FIG. The subjects O1 and O2, the virtual imaging surfaces P1 to P5, and the principal points L1 to L5 of the virtual imaging lens are the same as those in FIG.

  Consider the case where the front side calculation unit 106 is processing one pixel PE of the virtual imaging surface P3. In FIG. 10, since the virtual imaging lens of the main point L3 corresponds to the virtual imaging surface P3, the arrow R1 ′ indicates the pixel to be processed and the main point of the virtual imaging lens corresponding to the virtual imaging surface to be processed. A straight line passing through Therefore, the intersection between the straight line R1 'and the subject O3 on the near side of the virtual imaging lens L3, which is detected by the front calculation unit 106 in the process of step Sa4, is a point P31 and a point P32.

  In step Sa6, the front side calculation unit 106 selects the intersection furthest from the virtual imaging lens L3 among the intersections P31 and P32, reads a value representing the color at that point from the subject information storage unit 12, and performs processing. It is set as the pixel value of this pixel. In this way, by selecting the farthest intersection among the intersections, when the viewpoint is ahead of the arrow R1 ′, the intersection P31 belonging to the region RO3 that cannot be seen behind the subject O3 when viewed from that viewpoint. Instead, the intersection P32 visible from the viewpoint can be selected.

  As described above, the image generation device 10 according to the first embodiment is a device that generates an element image for the IP stereoscopic video display device 20. The image generation apparatus 10 includes a subject information storage unit 102 that stores subject information representing the position and shape of the subject in the virtual space. The image generation apparatus 10 also includes first camera information that represents a first position of the virtual camera array in the virtual space, and monocular camera information that represents the position of the virtual monocular camera provided in front of the virtual camera array in the virtual space. A virtual camera array information storage unit 101 that stores depth perception information indicating the degree of perception of the depth of the integral photography stereoscopic image is provided. Further, the image generation apparatus 10 stores the depth perception information stored in the virtual camera array information storage unit 101 with the front-side maximum depth perception position and the depth-side maximum depth perception position indicating the depth reproduction range in which the three-dimensional information of the subject is reproduced. The depth reproduction range calculation unit 103 that calculates based on the above is provided. In addition, the image generation device 10 is a virtual arrangement in which a virtual imaging lens is arranged at a position corresponding to each lens of the lens array of the IP stereoscopic video display device 20 and a virtual imaging surface is arranged on the back side of the virtual imaging lens. Assuming that the third element image of the subject existing between the position of the virtual imaging lens and the front maximum depth perception position calculated by the depth reproduction range calculation unit 103 is the subject information stored in the subject information storage unit 102 and the virtual camera. A front side calculation unit 106 that calculates based on the first camera information stored in the array information storage unit 101 is provided. Further, the image generation apparatus 10 assumes that the virtual arrangement is the above, and uses the subject information stored in the subject information storage unit 102 and the virtual camera for the fourth element image of the subject existing in front of the front maximum depth perception position. A maximum front side calculation unit 107 that calculates based on monocular camera information and first camera information stored in the array information storage unit 101 is provided. Further, the image generation device 10 assumes that the virtual arrangement is the above, and the first element image by the subject existing from the position of the virtual imaging plane to the depth side maximum depth perceived position calculated by the depth reproduction range calculation unit 103, A back side calculation unit 104 that calculates based on the subject information stored in the subject information storage unit 102 and the first camera information stored in the virtual camera array information storage unit 101 is provided. In addition, the image generation apparatus 10 assumes that the virtual arrangement is described above, the second element image of the subject existing behind the back side maximum depth perception position, and the subject information stored in the subject information storage unit 102. A maximum back side calculation unit 105 that calculates based on the monocular camera information and the first camera information stored in the virtual camera array information storage unit 101 is provided. The image generation apparatus 10 also includes a front side synthesis unit 109 that synthesizes the third element image calculated by the front side calculation unit 106 and the fourth element image calculated by the maximum front side calculation unit 107, and a back side calculation unit 104. And a back side combining unit 108 that combines the first element image calculated by the above and the second element image calculated by the maximum back side calculating unit 105. In addition, the image generation apparatus 10 includes a synthesis unit 110 that synthesizes an element image obtained by synthesis by the front side synthesis unit 109 and an element image obtained by synthesis by the back side synthesis unit 108.

  With this configuration, the image generation device 10 reproduces the IP stereoscopic image within the depth reproduction range with high accuracy and reproduces the depth compressed outside the depth reproduction range, thereby suppressing a reduction in the resolution of the IP reproduction image. In addition, a sense of depth can be given to the viewer.

[Second Embodiment]
The IP stereoscopic video display system in the second embodiment has the same configuration as the IP stereoscopic video display system 1 of FIG. Therefore, in the description of the second embodiment, the configuration diagram of FIG. 3 is used. In the IP stereoscopic video display system 1 in the second embodiment, the operations of the maximum back side calculation unit 105 and the maximum front side calculation unit 107 in the image generation apparatus 10 are partially different from those in the first embodiment. Further, part of the information stored in the virtual camera array information storage unit 101 is different from that of the first embodiment.

  The image generating apparatus 10 according to the second embodiment generates a maximum depth element image and a maximum front element image by compressing the depth for a three-dimensional model and set outside the depth reproduction range.

  In the second embodiment, the virtual camera array information storage unit 101 stores the second position of the second virtual camera array in front of the first position of the first virtual camera array in the first embodiment. The second camera information to be represented is further stored. In the second embodiment, the second camera information includes information representing the position of the principal point of each virtual imaging lens in the second virtual camera array and information representing the position of the virtual imaging surface corresponding to each principal point. Including.

  FIG. 11 is a schematic diagram for explaining the operation of the maximum back side calculation unit 105 in the second embodiment. This figure is a top view of a virtual space for calculating an element image displayed by the IP stereoscopic video display device 20 in the right-hand direction in the figure. The subject O (O) 1 is located behind the virtual imaging plane P1 and before the virtual plane K including the maximum depth perceived position on the back side. Specifically, the subject O1 is calculated on the back side when the first virtual imaging area formed by the principal point L1 of the virtual imaging lens and the virtual imaging plane P1 (first virtual camera) is cut by the virtual plane K. It is in area A.

  The subject O (O) 2 is a second virtual imaging formed by the principal point L1 ′ of the virtual imaging lens and the virtual imaging plane P1 ′ (second virtual camera) provided in front of the first virtual camera. When the area is cut by the virtual plane K, it is within the maximum depth calculation area B ′. However, the surface where the back side calculation area A is in contact with the virtual plane K and the surface where the maximum back side calculation area B ′ is in contact with the virtual plane K are the same.

  With the first virtual camera and the subject O1 removed, the maximum back side calculation unit 105 stores the maximum back side element image in the subject information stored in the subject information storage unit 102 and the virtual camera array information storage unit 101. And the calculated second camera information. Specifically, the maximum depth side calculation unit 105 passes the pixels on the virtual imaging plane P1 ′, and then reversely traces the light passing through the principal point L1 ′ of the virtual imaging lens, whereby the maximum depth side element image with respect to the subject O2. Is calculated.

  Each virtual imaging lens of the first virtual camera array corresponds to each lens of the lens array of the IP stereoscopic video display device 20. Therefore, when the IP video signal generated by the image generation apparatus 10 according to the second embodiment is captured by the IP stereoscopic video display apparatus 20 and displayed, the depth of the subject O2 in the maximum depth calculation area B ′ is compressed. Is done.

  Although not shown, when the subject O (O) 3 is in front of the virtual plane K ′ including the front maximum depth perception position, that is, outside the depth reproduction range, the maximum front calculation unit 107 calculates the maximum front element image. Is calculated based on the subject information stored in the subject information storage unit 102 and the second camera information stored in the virtual camera array information storage unit 101. Specifically, the maximum front side calculation unit 107 calculates the pixel value of each pixel of the maximum front side element image as a straight line passing through the pixel on the virtual imaging plane P1 ′ and the principal point L1 ′ of the virtual imaging lens. The calculation is performed with reference to the information regarding the point farthest from the main point L1 'among the points crossing the subject O3 on the front side.

[First Modification of Second Embodiment]
The image generation apparatus 10 that is the first modified example compresses the depth by changing the compression rate stepwise with respect to the three-dimensional model and set outside the depth reproduction range, and obtains the maximum back element image and the maximum front element image. Generate. Specifically, the maximum back side calculation unit 105 generates a maximum back side element image so that the depth is compressed as the distance from the virtual imaging surface increases. Further, the maximum front side calculation unit 107 generates a maximum front side element image so that the depth is compressed as the distance from the principal point of the virtual imaging lens increases.

  FIG. 12 is a schematic diagram for explaining the operation of the maximum back side calculation unit 105 in the first modification of the second embodiment. This figure is a top view of a virtual space for calculating an element image displayed by the IP stereoscopic video display device 20 in the right-hand direction in the figure. The subject O (O) 1 is behind the virtual imaging plane P1 and in front of the virtual plane K1 including the maximum depth perceived position on the back side. Specifically, the subject O1 is calculated on the back side when the first virtual imaging area formed by the principal point L1 of the virtual imaging lens and the virtual imaging plane P1 (first virtual camera) is cut by the virtual plane K1. It is in area A.

  The subject O (O) 2 is a second virtual imaging formed by the principal point L1 ′ of the virtual imaging lens and the virtual imaging plane P1 ′ (second virtual camera) provided in front of the first virtual camera. The region is in the first maximum back side calculation region B ′ when the region is cut out by the virtual plane K1 and the virtual plane K2 on the back side from the virtual plane K1. However, the surface where the back side calculation area A is in contact with the virtual plane K1 and the surface where the first maximum back side calculation area B ′ is in contact with the virtual plane K1 are the same.

  The subject O (O) 3 is formed by a third point formed by the principal point L1 ″ of the virtual imaging lens and the virtual imaging plane P1 ″ (third virtual camera) provided in front of the second virtual camera. When the virtual imaging area is cut along the virtual plane K2, it is within the second maximum depth calculation area B ″. However, the surface where the first maximum back side calculation region B ′ is in contact with the virtual plane K2 and the surface where the second maximum back side calculation region B ″ is in contact with the virtual plane K2 are the same.

  In a state where the first virtual camera and the subject O1 are removed, the maximum back side calculation unit 105 reversely tracks light passing through the principal point L1 ′ of the virtual imaging lens after passing through the pixels on the virtual imaging surface P1 ′. Thus, the first maximum back element image for the subject O2 is calculated.

  Further, in a state where the first virtual camera, the subject O1, the second virtual camera, and the subject O2 are removed, the maximum back side calculation unit 105 passes the pixels on the virtual imaging plane P1 ″ and then the virtual imaging lens. The second maximum back side element image for the subject O3 is calculated by tracking the light passing through the principal point L1 ″ of the subject.

  Each virtual imaging lens of the first virtual camera array corresponds to each lens of the lens array of the IP stereoscopic video display device 20. Therefore, when the IP stereoscopic video display device 20 takes in the IP video signal generated by the image generation device 10 which is the first modification of the second embodiment and displays the IP video, the first maximum back side calculation region B The depth of the subject O2 in “is compressed, and the depth of the subject O3 in the second maximum depth calculation area B ″ is further compressed.

  Although not shown, the maximum front side calculation is also performed in the case of the front side of the virtual plane K1 ′ including the front maximum depth perception position, that is, outside the depth reproduction range, as in the second embodiment and the first modification. The unit 107 calculates a pixel value of each pixel of the maximum front element image.

[Second Modification of Second Embodiment]
In the second embodiment, when the maximum depth side calculation unit 105 calculates the maximum depth element image for the subject O2, it is necessary to remove the first virtual camera and the subject O1. In the second modification of the second embodiment, as shown in the top view of FIG. 13, a second virtual camera C is provided in a direction perpendicular to the display direction, and the virtual half mirror is inclined by 45 degrees from the display direction. By providing M, it is not necessary to remove the first virtual camera.

  As shown in FIG. 2, since the depth perception tends to be gradually saturated before reaching a certain depth position, the image generation apparatus 10 according to the second embodiment can detect the depth perception according to the degree of the depth perception. By compressing the depth, it is possible to prevent the deterioration of the apparent image quality of the IP reproduction image. In addition, when the element image is generated by CG, the detection accuracy of the depth position of the subject can be increased.

  Note that some functions of the image generation apparatus 10 according to the above-described embodiment may be realized by a computer. In this case, an image generation program for realizing the function is recorded on a computer-readable recording medium, the image generation program recorded on the recording medium is read into the computer system, and the computer system executes the program. The function may be realized. The computer system includes an operating system (OS) and hardware of peripheral devices. The computer-readable recording medium is a portable recording medium such as a flexible disk, a magneto-optical disk, an optical disk, or a memory card, and a storage device such as a magnetic hard disk or a solid state drive provided in the computer system. Furthermore, a computer-readable recording medium dynamically holds a program for a short time, such as a computer network such as the Internet, and a communication line when transmitting a program via a telephone line or a cellular phone network. In addition, a server that holds a program for a certain period of time, such as a volatile memory inside a computer system serving as a server device or client in that case, may be included. The image generation program may be for realizing a part of the above-described functions, and further, the above-described functions are realized by a combination with a program already recorded in a computer system. May be.

  According to each embodiment and modification, the image generation device can make the viewer perceive depth without degrading the resolution of the integral photography stereoscopic image.

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the specific structure is not restricted to that embodiment, The design of the range which does not deviate from the summary of this invention, etc. are included.

DESCRIPTION OF SYMBOLS 1 IP stereoscopic video display system 10 Image generation apparatus 20 IP stereoscopic video display apparatus 101 Virtual camera array information storage part 102 Subject information storage part 103 Depth reproduction range calculation part 104 Back side calculation part 105 Maximum depth side calculation part 106 Front side calculation part 107 Maximum front calculation unit 108 Back side synthesis unit 109 Front side synthesis unit 110 Synthesis unit 111 Video signal generation unit

Claims (5)

An image generation device for generating an elemental image for an integral photography stereoscopic video display device,
A subject information storage unit for storing subject information representing the position and shape of the subject in the virtual space;
First camera information representing a first position of a virtual camera array in the virtual space, monocular camera information representing a position of a virtual monocular camera provided in front of the virtual camera array in the virtual space, and an integral photography stereoscopic A virtual camera array information storage unit for storing depth perception information representing the degree of perception with respect to the depth of the image;
A depth reproduction range calculation unit for calculating a depth side maximum depth perception position in a depth reproduction range for reproducing the three-dimensional information of the subject based on the depth perception information stored in the virtual camera array information storage unit;
Assuming that the virtual imaging lens is arranged at a position corresponding to each lens of the lens array of the integral photography stereoscopic video display device, and the virtual imaging surface is arranged on the back side of the virtual imaging lens, the virtual imaging lens The first element image of the subject existing from the position of the imaging surface to the depth side maximum depth perceived position calculated by the depth reproduction range calculation unit, subject information stored in the subject information storage unit, and the virtual camera array A back side calculation unit for calculating based on the first camera information stored in the information storage unit;
As the virtual arrangement, the second element image of the subject existing behind the maximum depth perception position on the back side is stored in the subject information stored in the subject information storage unit and the virtual camera array information storage unit. A maximum back side calculation unit for calculating based on the stored monocular camera information and the first camera information;
A back side combining unit that combines the first element image calculated by the back side calculating unit and the second element image calculated by the maximum back side calculating unit;
An image generation apparatus comprising: The depth reproduction range calculation unit further calculates a front maximum depth perception position in the depth reproduction range based on the depth perception information stored in the virtual camera array information storage unit,
As the virtual arrangement, a third element image by a subject existing from the position of the virtual imaging lens to the front maximum depth perception position calculated by the depth reproduction range calculation unit is stored in the subject information storage unit. A front side calculation unit for calculating based on the subject information and the first camera information stored in the virtual camera array information storage unit,
As the virtual arrangement, the fourth element image of the subject existing in front of the front maximum depth perception position is stored in the subject information stored in the subject information storage unit and the virtual camera array information storage unit. A maximum front side calculation unit for calculating based on the monocular camera information and the first camera information;
A front side synthesis unit that synthesizes the third element image calculated by the front side calculation unit and the fourth element image calculated by the maximum front side calculation unit;
A combining unit that combines the element image obtained by combining the back side combining unit and the element image obtained by combining the front side combining unit;
The image generation apparatus according to claim 1, further comprising: The virtual camera array information storage unit further stores second camera information representing a second position that is a front side of the first position of the virtual camera array in the virtual space,
The maximum depth side calculation unit calculates the second element image based on subject information stored in the subject information storage unit and second camera information stored in the virtual camera array information storage unit. ,
The maximum front side calculation unit calculates the fourth element image based on subject information stored in the subject information storage unit and second camera information stored in the virtual camera array information storage unit.
The image generation apparatus according to claim 2. The back side synthesis unit synthesizes the second element image by overwriting the first element image,
The front side synthesis unit synthesizes the third element image by overwriting the fourth element image,
Synthesize by overwriting the element image obtained by the front side composition unit over the element image obtained by the back side composition unit,
The image generation apparatus according to claim 2. A subject information storage unit for storing subject information representing the position and shape of the subject in the virtual space; first camera information representing a first position of the virtual camera array in the virtual space; and the virtual camera array in the virtual space. A virtual camera array information storage unit for storing monocular camera information indicating the position of the virtual monocular camera provided in front and depth perception information indicating the degree of perception of the depth of the integral photography stereoscopic image; ,
A depth reproduction range calculation unit for calculating a depth side maximum depth perception position in a depth reproduction range for reproducing the three-dimensional information of the subject based on the depth perception information stored in the virtual camera array information storage unit;
Assuming that the virtual imaging lens is arranged at a position corresponding to each lens of the lens array of the integral photography stereoscopic video display device, and the virtual imaging surface is arranged on the back side of the virtual imaging lens, the virtual imaging lens The first element image of the subject existing from the position of the imaging surface to the depth side maximum depth perceived position calculated by the depth reproduction range calculation unit, subject information stored in the subject information storage unit, and the virtual camera array A back side calculation unit for calculating based on the first camera information stored in the information storage unit,
As the virtual arrangement, the second element image of the subject existing behind the maximum depth perception position on the back side is stored in the subject information stored in the subject information storage unit and the virtual camera array information storage unit. A maximum depth side calculation unit for calculating based on the stored monocular camera information and the first camera information;
A back side combining unit that combines the first element image calculated by the back side calculating unit and the second element image calculated by the maximum back side calculating unit;
Image generation program to function as

Images