JP2013191950A - Stereoscopic image data generation system, stereoscopic image data display system, and stereoscopic image data display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that stereoscopic videos displayed on a plurality of screens are difference in parallax on border surfaces of the respective screens and when videos project forward from the screens, visual angle fatigue is possibly caused because of picture frame sticking effect.SOLUTION: Image data for a plurality of screens are cut out of a plurality of view point image data using arrangement information and inclination information on screen surfaces of the respective screens constituting the respective screens, perspective of the image data is corrected using the inclination information, and parallax adjustments on stereoscopic image data to be displayed on the respective screens are made so that adjustment amounts are equal. An arrangement of frames at borders of the respective screen surfaces is found from the arrangement information, and the parallax adjustments are made so that parallaxes in regions adjoining frames are parallaxes in directions of inward withdrawal from the screen surfaces.

Description

  The present invention relates to a stereoscopic image data generation system, a stereoscopic image data display system, and a stereoscopic image data display method.

  Conventionally, various methods for presenting a three-dimensional image have been proposed. Among them, what is generally used is a so-called “binocular type” that uses binocular parallax. This method enables stereoscopic viewing by preparing a left-eye image and a right-eye image having binocular parallax and independently projecting them to the left and right eyes. In the following description, the images described above are referred to as left-eye image data and right-eye image data. Further, 3D is used as a term meaning 3D or 3D, 2D is used as a term meaning 2D, stereoscopic image data is called 3D image data, and normal 2D image data is called 2D image data. The image includes not only a still image but also a moving image.

Here, a frame sequential method, a parallax barrier method, and the like have been proposed as typical twin-lens methods, which will be described in detail below based on the conceptual diagram.
FIG. 29 is a conceptual diagram for explaining the frame sequential method. In general, the frame sequential method is a display that switches image frames at high speed for display and an active lens that controls the lens shutter of the glasses in synchronization with the display to open and close the left and right lenses alternately.・ It consists of shutter glasses.

  In FIG. 29, left-eye image data 100 and right-eye image data 101 are alternately displayed at high speed on the display. In accordance with this timing, when the left eye image data 100 is displayed, the active shutter glasses 102 control the light through the lens shutter 103 for the left eye, and the light is passed through the lens shutter 104 for the right eye. Control to shut off. On the contrary, when the right-eye image data 101 is displayed, the right eye lens shutter 104 is controlled to transmit light, and the left eye lens shutter 103 is controlled to block light. By doing so, images matching the parallax of each eye are presented to the left and right eyes in a time division manner, and the observer can observe the stereoscopic image data.

  FIG. 30 is a conceptual diagram for explaining the parallax barrier method. FIG. 30A is a diagram illustrating the principle of generating parallax. On the other hand, FIG. 30B is a diagram illustrating an example of a screen displayed by the parallax barrier method. In the configuration shown in FIG. 30A, an image in which left-eye image data and right-eye image data shown in FIG. 30B are arranged alternately every other pixel in the horizontal direction is displayed as an image. By installing the parallax barrier 106 displayed on the display panel 105 and having slits with slits narrower than the pixels of the same viewpoint on the front viewpoint side of the image display panel 105, the image data for the left eye is only the left eye 107, The right-eye image data can be stereoscopically viewed by observing only with the right eye 108.

Further, Patent Document 1 includes a plurality of display surfaces, and as an image to be displayed on each display surface, parameters of an inclination angle of an imaging surface of a camera and an inclination angle of an imaging surface of a virtual camera for generating a computer graphics image. A stereoscopic image data presentation device using a photographed or generated image with the same or substantially the same as the inclination angle of each image presentation surface is disclosed.
In Patent Document 1, when the display surface is set parallel to the ground, and stereoscopic image data is displayed on the display surface and observed from obliquely above, the viewpoint position of the observer is assumed and the perspective is corrected by trapezoid correction. It is stated that the amount of parallax needs to be set so that an appropriate stereoscopic image is reproduced, such as by changing (Perspective).

  In addition, in a 3D image using binocular parallax, the feeling of projecting forward to the three-dimensional object and the feeling of retracting to the back (rear) can be controlled by adjusting the parallax, but when displaying the subject in front of the display surface, If the subject falls on the frame frame at the edge of the screen, the corresponding point of the image data for the left eye and the image data for the right eye disappears, and the stereoscopic view is broken, or the amount of popping out from the original parallax is closer to the display surface side. It is known that an image frame sticking effect, which is retracted or seen, and a phenomenon commonly called occur (see Patent Documents 2 to 6).

JP 2006-21453 A JP 2004-178579 A Japanese Patent Laid-Open No. 2004-178581 JP 2004-206672 A US Pat. No. 6,389,236 US Pat. No. 6,614,427

  However, in the conventional stereoscopic image data presentation system having a plurality of screen surfaces, when performing the parallax adjustment on the stereoscopic images displayed on the plurality of screen surfaces, There is a problem in that the parallax is different at the boundary surface, and the continuity at the boundary surface of the stereoscopic video presentation position and size cannot be maintained.

  In addition, in a conventional stereoscopic image data presentation system having a plurality of screen surfaces, a direct-view type such as a liquid crystal panel, an organic EL, or a plasma display is used as a plurality of screen surfaces instead of a projection screen used in a projector or the like. When a panel is used, there is a problem that even if it is a direct-viewing panel with a narrow frame frame, unless the frame frame disappears, the frame frame is present at the boundary part, and the boundary part of the panel is not smoothly joined.

  In addition, when an object is displayed in a stereoscopic manner in an area that crosses the boundary part and the object has a parallax that is displayed in front of the frame position at the boundary, the pop-up image of the frame frame is displayed. As in the case, stereoscopic vision is broken or parallax struggle occurs due to disappearance of corresponding points, an image frame sticking effect, and the like, and a phenomenon of double-looking or double-looking appears. This phenomenon may cause visual fatigue, and there is a problem in that viewing safety of stereoscopic image data is lowered.

  The present invention has been made in view of such circumstances, and its purpose is to cut out an image in consideration of the frame frame at the boundary and the arrangement of each display surface, and to create a perspective according to the inclination of each display surface. By correcting, when a viewer displays one piece of stereoscopic image data on multiple displays, it generates stereoscopic image data that can be viewed stereoscopically without a sense of incongruity, regardless of the presence or absence of frame frames and panel layout. Another object of the present invention is to provide a stereoscopic image data generation system, a stereoscopic image data display system, and a stereoscopic image data display method for display.

In order to solve the above problems, a first technical means of the present invention is a stereoscopic image data generation system that generates stereoscopic image data generated from a plurality of viewpoint image data on a plurality of image display surfaces. And using the stereoscopic image input means for inputting the plurality of viewpoint image data and the arrangement information and the tilt information of the plurality of image display surfaces, the plurality of viewpoint image data for each of the plurality of image display surfaces. Image cutout means for cutting out image data, distortion correction means for correcting the perspective of the image data for each of the plurality of image display surfaces using the tilt information, and the perspective after correction of the perspective using the arrangement information And a parallax adjusting unit configured to perform parallax adjustment on the image data for each of the plurality of image display surfaces.

  According to a second technical means, in the first technical means, the parallax adjusting means performs parallax adjustment so that the parallax adjustment amount has the same value for the image data for the plurality of image display surfaces. It is characterized by that.

  According to a third technical means, in the first or second technical means, the parallax adjusting means obtains an arrangement of a non-display area at a boundary of each image display surface from the arrangement information, and each of the adjacent adjoining non-display areas. The parallax adjustment is performed so that the parallax in the region of the image display surface is a parallax in a direction of being retracted deeper than each of the image display surfaces.

  A fourth technical means is a stereoscopic image display system including an image display means having a plurality of image display surfaces, and the image display means is stereoscopic image data which is any one of the first to third technical means. The stereoscopic image data is received from a generation system, and stereoscopic images are displayed on the plurality of image display surfaces.

  A fifth technical means is a stereoscopic image data display method for displaying stereoscopic image data generated from a plurality of viewpoint image data on a plurality of image display surfaces, and generates the plurality of viewpoint image data. A stereoscopic image input step to input, an image cutout step of cutting out image data for each of the plurality of image display surfaces from the plurality of viewpoint image data, using arrangement information and inclination information of the plurality of image display surfaces; Distortion correction step for correcting the perspective of the image data for each of the plurality of image display surfaces using the tilt information, and images for each of the plurality of image display surfaces after correcting the perspective using the arrangement information A parallax adjustment step for performing parallax adjustment on data is provided.

  A sixth technical means is the fifth technical means, wherein the parallax adjustment step performs parallax adjustment so that the parallax adjustment amount is the same value for the image data for the plurality of image display surfaces. It is characterized by that.

  A seventh technical means is the fifth or sixth technical means, wherein the parallax adjusting step obtains an arrangement of a non-display area at a boundary of each image display surface from the arrangement information, and each adjacent to the non-display area. The parallax adjustment is performed so that the parallax in the region of the image display surface becomes a parallax in a direction retracting to the back of each image display surface.

  According to the present invention, when one stereoscopic image data is displayed on a plurality of stereoscopic displays, an image is cut out according to the presence or absence of a frame frame at the boundary portion of each display and the position of the display surface. When the viewer displays one stereoscopic image data on a plurality of displays by generating and displaying stereoscopic image data with a perspective corrected according to the distance to the viewer and the inclination of each display surface, A stereoscopic view can be made without any sense of incongruity without depending on the presence or absence of the frame and the arrangement of the panel, and a stereoscopic view that is safe and less fatigued can be achieved.

It is a block diagram which shows schematic structure of the stereo image data display system by embodiment of this invention. It is a flowchart explaining operation | movement of the stereo image data display system by embodiment of this invention. It is a figure explaining the mode of photography of stereoscopic image data input means. It is a figure explaining the mode of photography of the left camera of stereoscopic image data input means. It is a figure explaining the mode of photography of the left camera of stereoscopic image data input means. It is a figure which shows the picked-up image by the left camera and the right camera. It is a figure which shows the parameter information P at the time of imaging | photography. It is a figure explaining the method to cut out the input image data for left eyes in an image cutout means. It is a figure explaining the method to cut out the input image data for right eyes in an image cutout means. It is a figure explaining the method of distortion correction with respect to image data for left eyes. It is a figure explaining the method of distortion correction with respect to the image data for right eyes. It is a figure which shows the example of the stereo image data which performed distortion correction. It is a figure explaining the parallax of stereo image data. It is a figure explaining a convergence angle. It is a figure explaining the convergence angle by parallax when a three-dimensional image is displayed in front of a screen. It is a figure explaining the convergence angle by parallax when a three-dimensional image is displayed in the back of a screen. It is a figure explaining the stereoscopic image data in case a viewer adjusts parallax so that the observed stereoscopic image may be in the direction of jumping forward. It is a figure explaining the stereoscopic image data in case a viewer adjusts parallax so that the observed stereoscopic image may be in a direction to retract further. It is a figure for demonstrating a mode that a viewer appreciates the stereo image data displayed on the image display means. It is a figure for demonstrating the change of the presentation position of the stereo image at the time of performing parallax adjustment. It is a structural example of the image display means comprised by two panels with a frame frame around the screen. It is another structural example of the image display means comprised by two panels with a frame frame around the screen. It is a figure explaining the mode of imaging | photography of the left camera of a stereo image data input means. It is an example of the picked-up image data by a stereo image data input means. It is a flowchart figure regarding the determination operation | movement of whether the parallax adjustment in a parallax adjustment means is performed. It is a figure explaining the method of performing parallax adjustment with respect to the stereo image data which performed distortion correction. It is a figure explaining the presentation position of the three-dimensional image when parallax adjustment is performed. It is another structural example of the image display means comprised by two panels with a frame frame around the screen. It is a conceptual diagram for demonstrating a frame sequential system. It is a conceptual diagram for demonstrating a parallax barrier system.

  Exemplary embodiments of a stereoscopic image data generation system, a stereoscopic image data display system, and a stereoscopic image data display method according to the present invention will be described below in detail with reference to the accompanying drawings. Moreover, in the following description, the structure which attached | subjected the same code | symbol also in different drawing shall be the same, and the description is abbreviate | omitted.

  A stereoscopic image data display system according to an embodiment of the present invention will be described with reference to the drawings. In the following description, the fact that the stereoscopic image data appears to pop out before the screen is defined as popping out, and the appearance that the stereoscopic image data appears to be retracted deeper than the screen is defined as the withdrawal. In the following, for the sake of simplicity, when calculating the parallax, the parallax is calculated by searching the corresponding points of the right-eye image data for the left-eye image data. On the other hand, the parallax may be calculated by searching for corresponding points in the image data for the left eye.

FIG. 1 is a block diagram showing a schematic configuration of a stereoscopic image data display system according to an embodiment of the present invention. The stereoscopic image data display system 1 includes, for example, a digital still camera that generates and displays stereoscopic image data, a video camera, a mobile phone, a smartphone, a PC, and the like. The stereoscopic image data display system 1 includes a stereoscopic image data input unit. 2, an image cutout unit 3, a distortion correction unit 4, a parallax adjustment unit 5, and an image display unit 6.
The stereoscopic image data generated above may be displayed by a television receiver, an electronic photo frame, or the like. In the stereoscopic image data display system 1, the stereoscopic image data input means 2, the image cutout means 3, the distortion correction means 4, and the parallax adjustment means 5 constitute a stereoscopic image data generation system of the present invention.

  The stereoscopic image data input means 2 is a means for photographing stereoscopic image data and inputting the stereoscopic image data to this system, and is composed of a plurality of cameras. Hereinafter, in order to simplify the description, a case will be described in which two cameras are used and stereoscopic image data of two left and right viewpoints is handled. The stereoscopic image data input unit 2 captures the stereoscopic image data ID1 of the left and right viewpoints, and transmits the captured stereoscopic image data and the parameter information P at the time of capturing to the image clipping unit 3. The operation of the stereoscopic image data input unit 2 will be described later. In the above description, the stereoscopic image data input unit 2 includes a plurality of cameras. However, the stereoscopic image data input unit 2 may be configured by a single camera, and the stereoscopic image data may be captured by sliding and moving the camera. Absent.

  The image cutout means 3 receives stereoscopic image data ID1 and parameter information P, and based on the parameter information P, an area to be displayed on the screen of each panel according to the arrangement of n (n is an integer of 2 or more) panels. And the created n area stereoscopic image data ID2n is output. The operation of the image cutout means 3 will be described later.

  The distortion correction means 4 receives n area stereoscopic image data ID2n and parameter information P, and based on the parameter information P, n areas are arranged in accordance with the arrangement of n (n is an integer of 2 or more) panels. N pieces of stereoscopic image data ID3n corrected so as to eliminate the perspective of the stereoscopic image data ID2n are created and output. The operation of the distortion correction unit 4 will be described later.

  The parallax adjusting unit 5 receives the stereoscopic image data ID3n and the parameter information P, calculates the parallax of the boundary portion of the screen of each panel using the stereoscopic image data ID3n, and the parallax of the boundary portion of each panel pops out. The parallax adjustment is performed so that the parallax in the direction is not generated, and the created stereoscopic image data ID4n is output. The operation of the parallax adjusting unit 5 will be described later.

  The image display means 6 receives the display signal of the stereoscopic image data ID 4n output from the parallax adjustment means 5, and based on the signal, displays the stereoscopic display images respectively corresponding to the n screen surfaces provided in the image display means 6. indicate. This image display screen displays the left-eye image data and the right-eye image data alternately on a liquid crystal display, plasma display, projector, etc., and synchronizes with this display, and the liquid crystal shutter that the viewer holds. Polarized glasses that operate only the liquid crystal shutter of the glasses, or that are equipped with a special polarizing filter that has different polarization for each line, and transmits only light of different polarization between the left and right eyes. It may be a polarizing glasses system that performs stereoscopic viewing through a liquid crystal display capable of autostereoscopic viewing such as a parallax barrier system or a lenticular system.

  In addition to a twin-lens type, a liquid crystal display using image data from a plurality of viewpoints may be used. For example, it is a liquid crystal display that displays a plurality of viewpoints at the same time to perform autostereoscopic viewing, or a two-lens stereoscopic display that tracks the viewer's eyes and switches the viewpoint according to the position of the viewer's eyes. Also good. As described above, the image display screen only needs to have a function of displaying an image such as a screen or a display, and corresponds to the display surface of the present invention.

  Next, the operation of the stereoscopic image data display system 1 according to the embodiment of the present invention will be described in detail with reference to the drawings. In the following, a case where data of two viewpoints is handled, that is, a case where the range of n is 1 to 2, will be described. FIG. 2 is a flowchart for explaining the operation of the stereoscopic image data display system 1. As shown in FIG. 2, in step S1, the stereoscopic image data input unit 2 captures the stereoscopic image data ID1.

  FIG. 3 is a diagram for explaining a situation in which the stereoscopic image data input means 2 configured by two cameras captures stereoscopic image data from two left and right viewpoints. As shown in FIG. 3, the left camera 7 </ b> L that captures the left-eye image data and the right camera 7 </ b> R that captures the right-eye image data, which are arranged at a predetermined distance in the horizontal direction, respectively capture the subject 8. . The predetermined distance at this time is generally referred to as a baseline length, and the length of the baseline length is set to about 63 mm, which is the same as the distance between human eyes. Further, in order to suppress the total amount of parallax, the length of the base line may be about 30 mm. Conversely, when parallax is emphasized when shooting a distant view, for example, it may be several meters. At this time, the optical axes of the cameras may intersect or be parallel, but here they are assumed to be parallel.

  As shown in FIG. 3, two screen surfaces are arranged on the back and bottom surfaces of the subject 8. For example, the image display screen of the image display means 6 includes a front screen on a front panel arranged perpendicular to the optical axis direction of the camera and a floor screen on a floor panel arranged horizontally with the optical axis direction of the camera. Are assumed to have a front screen on the back of the subject 8 and a floor screen on the lower surface of the subject 8, and the same screen as the assumed screen at each screen location. A planar object having a size is arranged. Hereinafter, the subject representing the front screen on the back of the subject 8 is referred to as the front screen 9, and the subject representing the floor screen on the lower surface of the subject 8 is referred to as the floor screen 10.

  Here, it is assumed that the boundary between the front screen 9 and the floor screen 10 is in contact, and the four vertices of the front screen 9 are P9, P10, P11, and P12, and the four vertices of the floor screen 10 are Let P1, P2, P9, and P10, respectively. Further, as shown in FIG. 3, the points where the dotted line obtained by dividing the floor screen 10 in the vertical direction by a quarter intersects perpendicularly with the line segment connecting the points P1 and P9 are P3, P5 and P7. Points that intersect perpendicularly with the line segment connecting the points P2 and P10 are defined as P4, P6, and P8. At this time, the cameras 7L and 7R are arranged so that the optical axis 11L of the left camera 7L and the optical axis 11R of the right camera 7R each intersect perpendicularly with a line segment 12 that divides the front screen 9 into two in the vertical direction. At the same time, the point where the optical axis 11L and the optical axis 11R intersect with the line segment 12 is the point 13L and the point 13R, respectively, and the center of the line segment connecting the point 13L and the point 13R is the point 14, respectively. The cameras 7L and 7R are arranged so as to be the center of the front screen 9.

  Next, a captured image of the left camera 7L arranged so as to have the arrangement described above will be described. FIG. 4 is a diagram for explaining a state of photographing by the left camera 7L of the stereoscopic image data input means 2. As shown in FIG. 4, when the left camera 7L captures the subject 8 arranged on the front screen 9 and the floor screen 10, the captured image is projected and converted on the projection surface 15L of FIG. It becomes an image.

  FIG. 5 is a diagram for explaining how the left camera 7L of the stereoscopic image data input means 2 is photographed. In FIG. 5, the state of shooting with the left camera 7L is viewed from the right direction. Here, when the dotted line portion 16 in the figure is the shooting range in the vertical direction of the camera 7L, the shot image shown on the projection plane 15L is shown. Causes perspective distortion in the vertical direction according to the distance from the left camera 7L. Although not shown, the perspective distortion occurs in the left-right direction in the same manner in the left-right direction.

  At this time, the right camera 7R also generates perspective distortion in the vertical and horizontal directions according to the distance from the right camera 7R in the same manner as the left camera 7L. FIG. 6 shows images taken by the left camera 7L and the right camera 7R. In FIG. 6, perspective distortion occurs in the vertical and horizontal directions in the left-eye image data 17L captured by the left camera 7L and the right-eye image data 17R captured by the right camera 7R, respectively.

  The stereoscopic image data input means 2 outputs the captured left-eye image data 17L and right-eye image data 17R as the left and right two-viewpoint stereoscopic image data ID1, and at the same time, although not shown, The arrangement information of the screens of the two panels constituting the image display means 6 is inputted to the stereoscopic image data input means 2 from the outside, and the stereoscopic image data input means 2 creates parameter information P from the information. ,Output.

FIG. 7 shows parameter information P at the time of photographing at this time. The parameter information P includes information on the camera used for shooting, the number of cameras at the time of shooting, position information indicating the position of the center of the optical axis of the camera, and inclination of the center of the optical axis of the camera. And angle-of-view information indicating the size of the angle of view of the camera are included for the number of cameras used for shooting. The parameter information P includes screen information as the number of screens used for display, individual information for each screen, center position information indicating the center position of the screen, size information indicating the screen size, and inclination of the screen surface. The screen surface tilt information indicating the number of screens used for display is included.
Here, the screen surface tilt information is information indicating how many times the screen surface is tilted from the vertical surface, and the range thereof is 0 degree or more and less than 360 degrees. For example, the screen surface inclination information of the front screen 9 is 0 degree, and the screen surface inclination information of the floor screen 10 is 90 degrees.

The parameter information may be input directly by the user, or may be input by a three-dimensional measuring device that measures depth information using a camera or infrared rays. A predetermined value determined in advance in accordance with the constituent means of the display means 6 may be held as a default value in a storage device (not shown) in the stereoscopic image data input means 2 and read from the storage device for use. I do not care.
As described above, the stereoscopic image data input unit 2 captures the stereoscopic image data ID1 of the two left and right viewpoints, and transmits the captured stereoscopic image data ID1 and the parameter information P at the time of capturing to the image clipping unit 3, The process proceeds to step S2 shown in FIG.

  In step S <b> 2, stereoscopic image data ID <b> 1 and parameter information P are input to the image cutout unit 3. The image cutout unit 3 cuts out an image to be displayed on the image display unit 6 from the stereoscopic image data ID1 using the input parameter information P. Here, the left-eye image data 18L for the front screen 9 and the left-eye image data 19L for the floor screen 10 are cut out from the left-eye image data 17L. FIG. 8 is a diagram for explaining a method of cutting out the input image data for the left eye in the image cutout unit 3.

  In FIG. 8, the image cutout means 3 cuts out a rectangular area surrounded by the points P9, P10, P11, and P12 of the left-eye image data 17L and obtains the left-eye image data 18L for the front screen 9 as shown in FIG. Further, a trapezoidal region surrounded by the points P1, P2, P9, and P10 of the left-eye image data 17L is cut out, and left-eye image data 19L for the floor screen 10 is generated.

  Here, right-eye image data 18R for the front screen 9 and right-eye image data 19R for the floor screen 10 are cut out from the right-eye image data 17R, respectively. FIG. 9 is a diagram for explaining a method of cutting out the input right-eye image data in the image cutting-out means 3.

  In FIG. 9, the image cutout means 3 cuts out a rectangular area surrounded by the points P9, P10, P11, and P12 of the right eye image data 17R, and outputs the right eye image data 18R for the front screen 9. In addition, the trapezoidal region surrounded by the points P1, P2, P9, and P10 of the right-eye image data 17R is cut out to generate right-eye image data 19R for the floor screen 10.

  The image cutout means 3 uses the generated left-eye image data 18L and right-eye image data 18R as the three-dimensional image data ID 21 cut out for the front screen 9, and the left-eye image data 19L and the right-eye image data. 19R is output as the stereoscopic image data ID 22 cut out for the floor screen 10, respectively. In the above description, an example in which a planar object is installed as the front screen 9 or the floor screen 10 has been described in order to use it as an index for clarifying the cutout position. In the case where the location of the screen is specified and cut out based on the screen size, position, and tilt information obtained from the information, it is not necessary to install a planar object.

  As described above, the image cutout unit 3 obtains the stereoscopic image data ID21 cut out for the front screen 9 and the stereoscopic image data ID22 cut out for the floor screen 10 from the parameter information P and the stereoscopic image data ID1, respectively. The process proceeds to decision step S3 shown in FIG.

  In the determination step S3, the stereoscopic image data ID2n and the parameter information P are input to the distortion correction unit 4, and the distortion correction unit 4 sequentially reads the input stereoscopic image data ID2n, and at the same time, the screen included in the parameter information P. The screen surface inclination information corresponding to the stereoscopic image data sequentially read out is referred to from the surface inclination information. If the value is 0 degree or 180 degrees, the process proceeds to step S5. Otherwise, the determination step S4 is performed. Proceed to

In step S4, the input image data is determined to have a perspective distortion, and is corrected so as to eliminate the perspective distortion.
For example, here, since the stereoscopic image data ID 21 input to the distortion correction unit 4 is stereoscopic image data cut out for the front screen 9 and the screen surface inclination information is 0 degrees, the distortion correction unit 4 Although the image is output as it is without being corrected, the stereoscopic image data ID 22 input to the distortion correction unit 4 is also stereoscopic image data cut out for the floor screen 10, and the screen surface inclination information is 90 degrees. Therefore, the distortion correction unit 4 performs distortion correction and outputs the result.

  Here, as the distortion correction method, as described with reference to FIG. 5, the image on the projection surface 15 </ b> L may be perspectively projected so as to be at the position of the floor screen 10. Hereinafter, a distortion correction method at this time will be described in detail with reference to the drawings. FIG. 10 is a diagram for explaining a distortion correction method for the left-eye image data 19L constituting the stereoscopic image data ID22. In FIG. 10, the distortion correction means 4 corrects the perspective distortion in the vertical and horizontal directions of the left-eye image data 19L, so that the point Pm (m is an integer from 1 to 10) of the left-eye image data 19L is the left-eye image data. Keystone correction is performed so that a point Am of 20 L is obtained. For example, point P1 is point A1, point P2 is point A2, point P3 is point A3, point P4 is point A4, point P5 is point A5, point P6 is point A6, and point P7 is point A7. Distortion correction is performed by performing perspective projection so that point P8 is at point A8, point P9 is at point A9, and point P10 is at point A10.

  FIG. 11 is a diagram for explaining a distortion correction method for the right-eye image data 19R constituting the stereoscopic image data ID22. Similarly to the description of FIG. 10, the distortion correction unit 4 corrects the perspective distortion in the vertical and horizontal directions of the right-eye image data 19R, so that the point Pm (m is an integer from 1 to 10) of the right-eye image data 19R is Keystone correction is performed so that the point Bm of the right-eye image data 20R is obtained. For example, point P1 is point B1, point P2 is point B2, point P3 is point B3, point P4 is point B4, point P5 is point B5, point P6 is point B6, and point P7 is point B7. Distortion correction is performed by performing perspective projection so that point P8 is at point B8, point P9 is at point B9, and point P10 is at point B10.

  In the determination step S5, when the image sequentially read out by the distortion correction unit 4 is the last image, the distortion correction unit 4 outputs the stereoscopic image data ID3n, and proceeds to the determination step S6. Otherwise, the determination step Return to S3. Here, FIG. 12 shows an example of the stereoscopic image data ID3n. Among the stereoscopic image data ID3n, the stereoscopic image data ID31 is composed of the same stereoscopic image data as the left-eye image data 18L and the right-eye image data 18R of the stereoscopic image data ID21.

  On the other hand, the stereoscopic image data ID 32 has been subjected to distortion correction for the left-eye image data 19L and the right-eye image data 19R input to the distortion correction unit 4 so that perspective distortion in the vertical and horizontal directions is eliminated. It consists of left-eye image data 20L and right-eye image data 20R. As described above, the distortion correction unit 4 outputs the stereoscopic image data ID21 without perspective distortion as it is as the stereoscopic image data ID31 among the input stereoscopic image data ID2n, and only the stereoscopic image data ID22 with perspective distortion, After correcting the perspective distortion, the stereoscopic image data ID 32 is output.

  In determination step S6, the stereoscopic image data ID3n and the parameter information P are input to the parallax adjusting unit 5, and the parallax adjusting unit 5 determines whether or not to perform parallax adjustment on the input stereoscopic image data ID3n. If the parallax adjustment is performed, the process proceeds to step S7, and if not, the process proceeds to step S8. Here, regarding the method for determining whether or not the parallax adjustment is performed in the parallax adjustment unit 5, the parallax adjustment may be manually performed by accepting an input from an external viewer. A parallax adjustment amount calculation unit that performs a matching process for searching corresponding points of the left and right images of the input stereoscopic image data ID3n (not shown) is provided in the parallax adjustment unit 5, and the parallax map of the entire image with respect to the stereoscopic image data ID3n And the parallax adjustment may be automatically performed so that the obtained parallax falls within a predetermined range.

  Note that the unit of matching processing for obtaining the parallax map at this time is a block unit, but the size of the block is not particularly limited, and may be a pixel unit, or the entire screen may be one block. In addition, any method may be used as the corresponding point search method for obtaining the parallax map. Also, the corresponding point search method may be an ordinary general method, and is different from the main point of the present invention, so the description thereof will be omitted.

In the following, for example, stereo matching is performed on the left-eye image data and the right-eye image data on a pixel basis, and parallax is obtained on a pixel basis. Note that the parallax at this time is in units of pixels, but the display screen size information may be used by converting to distance information when actually displayed on the screen, or the number of pixels in the horizontal direction of the screen It may be converted into a percentage with respect to.
In the following, for the sake of simplification of description, the description will be made using the parallax angle. However, the relationship between the parallax and the parallax angle is uniquely determined from the display screen size information, the viewing distance, and the length of the binocular interval. Therefore, parallax may be used instead of the parallax angle.

  Here, the parallax will be described. FIG. 13 is a diagram illustrating the parallax of stereoscopic image data. FIG. 13A shows image data for the left eye, and when stereoscopic display is performed, the farthest area is the farthest point 21, and the most visible area is the nearest point 22. FIG. 13B shows the right-eye image data. Among these, when stereoscopic display is performed, the region that is the corresponding point of the farthest point 21 of the left-eye image data and that appears farthest is the farthest. The far point 23 is a corresponding point of the nearest point 22 of the left-eye image data, and the closest visible region is the nearest point 24. At this time, the distance from the left end of the left-eye image data to the farthest point 21 is dfL1, the distance to the nearest point 22 is dnL1, the distance from the left end of the right-eye image data to the farthest point 23 is dfR1, and the leftmost point The distance to the point 24 is dnR1.

  Then, when stereoscopic display is performed using the left-eye image data and the right-eye image data, the parallax in the part that is farthest from the viewer is the farthest view parallax, and the value is defined as dfL1-dfR1. Similarly, when stereoscopic display is performed using the image data for the left eye and the image data for the right eye, the latest scene parallax in the portion that is closest to the viewer is obtained, and the value is defined as dnL1-dnR1. At this time, the farthest view parallax and the most recent view parallax are obtained in units of pixels, but are converted into distance information when actually displayed on the screen using the display screen size information. The display screen size information indicates the pixel size W in the horizontal direction of the actual display area when displaying stereoscopic image data, and information on the resolution in the horizontal direction.

  When the parallax is a percentage of the number of pixels in the horizontal direction of the screen, the display screen size information may be the size of the actual display area in the horizontal direction when displaying stereoscopic image data. Here, when the parallax value is negative, it indicates that the position of the displayed area is behind the screen, and when it is positive, the position of the displayed area is in front of the screen. Indicates.

Next, the parallax angle will be described.
FIG. 14 is a diagram for explaining the convergence angle when viewing the screen. In FIG. 14, when the viewer views a point 28, which is one point on the screen 27, through the left eye 25 and the right eye 26, the point that intersects when the left eye 25 is connected from the point 28 with a line is the point 29 and the point 28 A point that intersects when connecting lines 26 with a line is defined as a point 30, and an intersection point of perpendicular lines drawn from the point 28 with respect to a line segment that connects the points 29 and 30 is defined as a point 31. At this time, the position of the point 31 is assumed to be located at the center of the line segment connected by the points 29 and 30.

式（１）より、輻輳角αは、式（２）のように表される。

Here, the line segment connecting point 29 and point 30 represents the distance between both eyes, which is the distance between the left eye and the right eye, the length is T, and the line segment connecting point 28 and point 31 is viewed. The viewing distance from the person to the screen is represented by the length L, and the angle formed by the line segment connecting the point 28 and the point 29 and the line segment connecting the point 28 and the point 30 is the convergence angle α. From the similar relationship, a relationship such as equation (1) is derived.

From Expression (1), the convergence angle α is expressed as Expression (2).

  FIG. 15 is a diagram for explaining a convergence angle due to parallax in a case where a stereoscopic video is displayed in a manner protruding from the screen. In FIG. 15, the viewer uses the left eye 25 as one point of the left eye image data on the screen 27, and the right eye 26 as the one point of the right eye image data corresponding to the point 32 on the screen 27. When each point 33 is viewed, the point 34 is presented as a stereoscopic image at a position in front of the screen.

  Incidentally, in the case of the stereoscopic image data shown in FIG. 12, the position of the subject 8 in the left-eye image data 18L and 20L is on the right side relative to the position of the subject 8 in the right-eye image data 18R and 20R. When the subject 8 of the data 18L and 20L and the right-eye image data 18R and 20R are alternately displayed on the screen, the viewer's eyes looking at the subject 8 intersect the eyes of both eyes before the screen. 8 forms an image in front of the screen, and it looks to the viewer as if the subject 8 protrudes from the screen.

式（３）より、輻輳角β１は、式（４）のように表される。

また同様に、三角形の相似の関係から、式（５）のような関係が導かれる。

式（４）、式（５）より、輻輳角β１は、式（６）のように表される。

The line segment connecting the points 31 and 34 represents the distance from the viewer to the position where the stereoscopic video is presented, the value thereof is m, and the points 32 and 33 represent the parallax and the length thereof. Is d1.
At this time, if the angle formed by the line segment connecting the points 29 and 34 and the line segment connecting the point 30 and the point 34 is defined as the convergence angle β1, the relationship as shown in the equation (3) is obtained from the similarity of the triangles. Is guided.

From Expression (3), the convergence angle β1 is expressed as Expression (4).

Similarly, a relationship such as equation (5) is derived from the similar relationship of triangles.

From Expressions (4) and (5), the convergence angle β1 is expressed as Expression (6).

Here, the parallax angle γ1 with respect to the parallax in the direction of jumping out from the screen, the convergence angle α when the viewer views the screen, and the stereoscopic video having the parallax in the direction of jumping out of the screen are viewed. When defined as the absolute value of the difference of the convergence angle β1, it is expressed as in equation (7).

  FIG. 16 is a diagram illustrating a convergence angle due to parallax when a stereoscopic image is displayed behind the screen. In FIG. 16, the viewer uses a point 35 that is one point of the left-eye image data on the screen 27 with the left eye 25, and one point of the right-eye image data that corresponds to the point 35 on the screen 27 with the right eye 26. When each point 36 is viewed, a point 37 is presented as a stereoscopic image at a position deeper than the screen.

式（８）より、輻輳角β２は、式（９）のように表される。

また同様に、三角形の相似の関係から、式（１０）のような関係が導かれる。

式（９）、式（１０）より、輻輳角β２は、式（１１）のように表される。

A line segment connecting the point 31 and the point 37 represents the distance of the stereoscopic video from the viewer, and its length is (L + n). Points 35 and 36 represent parallax, and the length thereof is d2.
At this time, if the angle formed by the line segment connecting the point 29 and the point 37 and the line segment connecting the point 30 and the point 37 is the convergence angle β2, the relationship as shown in the equation (8) is obtained from the similarity of the triangles. Is guided.

From Expression (8), the convergence angle β2 is expressed as Expression (9).

Similarly, a relationship such as equation (10) is derived from the similar relationship of triangles.

From Expressions (9) and (10), the convergence angle β2 is expressed as Expression (11).

Here, the parallax angle γ2 with respect to the parallax in the direction retracting from the screen, the convergence angle α when the viewer looks at the screen, and the convergence when viewing the stereoscopic video having the parallax in the direction of jumping out from the screen When defined as the absolute value of the difference of the angle β2, it is expressed as in Expression (12).

  Usually, the parallax angle is expressed as an absolute value, but in the following, in order to distinguish the parallax angle with respect to the parallax in the direction of jumping out from the screen and the parallax angle with respect to the parallax in the direction of retracting from the screen, The parallax angle with respect to the parallax in the direction of retracting further is a negative value. Normally, since the value of the parallax angle is an absolute value, the smaller the value, the closer the stereoscopic image appears to the screen, and the larger the parallax, the larger the pop-out and retraction of the stereoscopic image will appear. Thus, when a sign is added to the value of the parallax angle, attention should be paid to the point that the parallax angle value is not referred to directly, but the magnitude of the absolute value is considered.

  In step S7, in the case of manual parallax adjustment, the parallax adjustment unit 5 creates the stereoscopic image data ID4n by shifting the left and right images of the stereoscopic image data ID3n to the left and right by the shift width of the parallax adjustment specified by the viewer. Further, in the case of automatic parallax adjustment, the parallax adjustment unit 5 shifts the left and right images of the stereoscopic image data ID3n to the left and right with the shift width of the parallax adjustment calculated by the parallax adjustment amount calculation unit separately provided in the parallax adjustment unit 5. The stereoscopic image data ID 4n is created by shifting.

  At this time, in order to maintain the continuity of the presentation position of the stereoscopic image displayed on each screen or the retracting direction, the shift width of the parallax adjustment includes the parallax adjustment stereoscopic image data ID 41 displayed on the front screen, and the floor screen. The same value is set for the parallax adjustment stereoscopic image data ID 42 displayed on the screen.

  FIG. 17 is a diagram for explaining the stereoscopic image data ID4n when the viewer adjusts the parallax so that the stereoscopic image to be observed is more projected when the stereoscopic image is displayed in front of the screen. It is. For example, in FIG. 17, the subject 8 of the left-eye image data 18L and the subject 8 of the left-eye image data 20L are shifted to the right by the same amount of parallax, and the subject 8 of the right-eye image data 18R and the right-eye image The subject 8 of the data 20R is shifted to the left by the same amount of parallax. That is, the positions of the left-eye image data and the right-eye image data on the subject 8 screen are shifted further away. As a result, the parallax value between the subject 8 of the left-eye image data and the subject 8 of the right-eye image data becomes larger, so that the stereoscopic video appears to protrude more.

  FIG. 18 is a diagram for explaining the stereoscopic image data ID4n when the viewer adjusts the parallax so that the stereoscopic video to be observed is in the retracted direction when the stereoscopic image is displayed in front of the screen. It is. For example, in FIG. 18, the subject 8 of the left-eye image data 18L and the subject 8 of the left-eye image data 20L are shifted to the left by the same amount of parallax, and the subject 8 of the right-eye image data 18R and the right-eye image The subject 8 of the data 20R is shifted to the right by the same amount of parallax. As a result, the parallax value between the subject 8 of the left-eye image data and the subject 8 of the right-eye image data becomes small, so that the stereoscopic video appears to be more retracted.

Further, in FIG. 18, the positions of the left-eye image data subject 8 and the right-eye image data subject 8 are shifted, and the position of the left-eye image data subject 8 is relatively left of the position of the subject 8 of the right-eye image data. If it is shifted until, the line of sight of both eyes intersects at the back of the screen rather than the screen for the viewer looking at the subject 8. For this reason, the subject 8 forms an image at the back of the screen, and the viewer 8 can see the subject 8 retracted from the screen to the viewer. In this case, the parallax value is a negative value.
In the above description, the case where the positions of the left-eye image data subject 8 and the right-eye image data subject 8 are shifted by the same amount has been described. However, only one of the left-eye image data and the right-eye image data is shifted. The parallax may be adjusted. Further, the parallax may be adjusted so that the shift amount of the left-eye image data and the shift amount of the right-eye image data are different.
As described above, the parallax adjusting unit 5 outputs the stereoscopic image data ID4n, and the process proceeds to step S8.

In step S8, the stereoscopic image data ID 4n is input to the image display means 6. The image display means 6 displays the input stereoscopic image data ID4n on two panels installed on the front and floor of the viewer.
Here, stereoscopic images that can be viewed when the parallax adjustment is performed on the stereoscopic image data ID4n and when the parallax adjustment is not performed will be described below with reference to the drawings.

  First, a stereoscopic image that can be viewed when parallax adjustment is not performed will be described. FIG. 19 is a diagram for explaining how the viewer appreciates the stereoscopic image data displayed on the image display means 6. In FIG. 19, a viewer 38 views a stereoscopic video with the front panel 39 and the floor panel 40, but the stereoscopic video displayed here is not subjected to parallax adjustment, for example, the stereoscopic image shown in FIG. 12. Data. Accordingly, in FIG. 19, the front panel 39 has stereoscopic image data ID41 having the same parallax as the stereoscopic image data ID31 in FIG. 12, and the floor panel 40 has stereoscopic image data having the same parallax as the stereoscopic image data ID32 in FIG. Each ID 42 is displayed, and the viewer performs binocular stereoscopic viewing by viewing images with different parallax between the left and right eyes, and views the front screen 41, the floor screen 42, and the subject 43 in three dimensions. can do.

  Next, stereoscopic video that can be viewed when performing parallax adjustment will be described. FIG. 20 is a diagram for describing a change in a stereoscopic image when parallax adjustment is performed. In FIG. 20, when the parallax adjustment is not performed, the viewer 38 observes the stereoscopic video at the positions of the front screen 41, the floor screen 42, and the subject 43. Here, by performing parallax adjustment on the stereoscopic image data ID4n, the viewer can change the pop-out position of the stereoscopic video according to his / her preference.

  For example, the stereoscopic image data ID 4n is generated by performing parallax adjustment as shown in FIG. 17, and the position of the subject shown in the right-eye image data 18R and 20R on the screen and the subject shown in the left-eye image data 18L and 20L. The position of the three-dimensional image that can be observed is displayed on the front panel 39 and the floor panel 40 after being shifted so that the position on the screen is further away from the position before the parallax adjustment when viewed from the viewer. Becomes a direction to project more, and as shown in FIG. 20, the viewer 38 can observe the front screen 44, the floor screen 45, and the subject 46 as a stereoscopic image.

Further, for example, the stereoscopic image data ID4n is generated by performing parallax adjustment as shown in FIG. 18, and the left-eye image data 18L, with respect to the position on the screen of the subject reflected in the right-eye image data 18R, 20R, Displayed on the front panel 39 and the floor panel 40 after the relative position on the screen of the subject shown in 20L is shifted from the viewer so that it is further left than before parallax adjustment. The position of the stereoscopic video that can be observed is in the direction of retracting further, and as shown in FIG. 20, the viewer 38 can observe the front screen 47, the floor screen 48, and the subject 49 as a stereoscopic video. .
Even after the stereoscopic display, the viewer may manually adjust the parallax at an arbitrary timing.
As described above, the image display unit 6 displays the stereoscopic image data ID4n, and the process proceeds to step S9.

  In step S9, the stereoscopic image data display system 1 ends the process. As described above, the stereoscopic image data display system 1 according to the present invention is capable of inclining the screen surface of each panel with respect to stereoscopic image data cut out in accordance with the number of panels to be displayed from the stereoscopic image data captured by the camera. Correct the perspective distortion according to the angle, maintain the parallax continuity at the screen boundary, and adjust the parallax adjustment so that the parallax adjustment amount of the stereoscopic image displayed on each screen surface becomes the same value ing.

  As a result, the continuity of parallax at the boundary surface is maintained even for stereoscopic images displayed on a plurality of screen surfaces, and as a result, the continuity of the presentation position and size of the stereoscopic image at the boundary surface is also maintained. Even for stereoscopic images using, the viewer can perform natural stereoscopic viewing with parallax according to his / her preference, safety, little fatigue, and no sense of incongruity.

  In the above description, the floor screen 10 is parallel to the horizontal direction. However, the floor screen 10 may be inclined at a predetermined angle with respect to the horizontal direction. In that case, the perspective transformation projection described with reference to FIGS. 10 and 11 may be converted to the floor screen 10 inclined by a predetermined angle with respect to the horizontal direction. Similarly, in the above description, the case where the front screen 9 is parallel to the vertical direction has been described. However, the front screen 9 may be inclined at a predetermined angle with respect to the vertical direction. In that case, the perspective transformation projection destination described in FIGS. 10 and 11 may be converted as a front screen 9 inclined at a predetermined angle with respect to the vertical direction.

  In addition, when the plurality of screens constituting the display unit 6 are not direct projection type screens such as projectors but direct-view type panels such as liquid crystal displays and plasma displays, a frame is formed around the screen displayed on the panel. Since there is a frame, a discontinuous region occurs at the boundary between adjacent screens. A case where a stereoscopic image is displayed in consideration of the discontinuous region at the boundary will be described below.

A method of arranging the two panels at this time will be described with reference to FIGS.
FIG. 21 is a configuration example of the image display means 6 configured by two panels having a frame frame around the screen. 21A shows a front view of the arrangement of the two panels of the image display means 6, and FIG. 21B shows a side view of the arrangement of the two panels of the display means 6. . As shown here, a front panel 52 composed of a screen 50 and a frame frame 51 and a floor panel 55 composed of a screen 53 and a frame frame 54 are installed so that the ends of the respective frame frames touch each other. Yes.

Here, in order to reduce the area of the discontinuous region on the boundary surface, one of the frame frames is arranged to be hidden.
FIG. 22 shows another configuration example of the image display means 6. FIG. 22A shows a view of the arrangement of the two panels of the display means 6 from the front, and FIG. 22B shows a view of the arrangement of the two panels of the display means 6 seen from the side. As shown here, the floor panel 55 is left as it is so that the boundary portion on the frame frame 54 is hidden by the frame frame 51, but only the front panel 52 is in front of the case of FIG. It is installed. In this case, a discontinuous region is generated only in a portion corresponding to the front screen.

  Next, the operation of the stereoscopic image data display system 1 at this time will be described with reference to the drawings. When the arrangement of the panels is changed as described above, the center position information in the screen information of the parameter information P generated when photographing the stereoscopic image data changes, but the stereoscopic image data constituting the stereoscopic image data display system 1 is changed. The basic operations of the input unit 2, the image cutout unit 3, the distortion correction unit 4, the parallax adjustment unit 5, and the image display unit 6 are the same.

For example, the operation of the stereoscopic image data input means 2 is basically the same except that the positions of some subjects are changed. FIG. 23 is a diagram for explaining how the left camera 7L of the stereoscopic image data input means 2 is photographed. In FIG. 23, the state of shooting with the left camera 7L is seen from the right direction, and the arrangement of the camera, the subject 8, and the floor screen 10 is the same as in FIG. At this time, since the portion of the area 56 surrounded by the dotted line and shaded is not displayed, the position of the front screen 9 is arranged in the upward direction so as to overlap the position of the screen 50 and photographed.
FIG. 24 shows an example of photographed image data by the stereoscopic image input means 2 at this time. As shown in FIG. 24, the position of the front screen 9 is arranged at the top in accordance with the position of the screen in each of the left and right images.

  For the captured image data of the stereoscopic image input means 2, the image cutout means 3 cuts out an image with reference to the screen information of the parameter information P as described above. Here, when there is no frame shown in FIG. 8, a rectangular area surrounded by the points P9, P10, P11, and P12 of the left-eye image data 17L is cut out, and the front screen 9 is used. The left-eye image data 18L is generated, and a trapezoidal region surrounded by the points P1, P2, P9, and P10 of the left-eye image data 17L is cut out, and the left-eye image data 19L for the floor screen 10 is extracted. Was generated. However, when there is a frame 52 shown in FIG. 24, a trapezoidal region surrounded by the points P1, P2, P9, and P10 of the image data for the left eye is cut out, and the left eye for the floor screen 10 is cut out. 8 is the same as that shown in FIG. 8, but the left-eye image data for the front screen 9 is surrounded by the points P9, P10, P11, and P12 in FIG. A rectangular area surrounded by the points P13, P14, P15, and P16, which is located above the rectangular area, is cut out.

  Here, a region surrounded by the points P9, P10, P13, and P14 is a region having a frame frame. A region where no image is displayed at the boundary between the screens corresponds to a non-display region between the screens of the present invention, and a frame frame is an example of a non-display region. The same applies to the right-eye image data.

  The distortion correction unit 4 similarly performs distortion correction on the clipped image with reference to the screen information of the parameter information P. As described above, the stereoscopic image input unit 2, the image cutout unit 3, and the distortion correction unit 4 perform the same operations as described in steps S1 to S5 in FIG.

  The operation of the parallax adjustment unit 5 in this case is different only in the method of determining whether or not to perform parallax adjustment in the determination step S6 of FIG. 2, and therefore only the determination operation of the parallax adjustment unit 5 will be described below. . FIG. 25 is a flowchart relating to the determination operation of whether or not to perform parallax adjustment in the parallax adjusting unit 5. Hereinafter, the operation of the parallax adjusting unit 5 will be described by replacing the determination step S6 of FIG. 2 with the determination step S10 of FIG. 25, the step S11, the determination step S12, and the step S13. In addition, since the same operation | movement is performed about step S1, step S2, determination step S3, step S4, determination step S5, step S7, step S8, and step S9, those description is abbreviate | omitted.

  From step S5, the process proceeds to determination step S10. In determination step S10, stereoscopic image data ID3n and parameter information P are input to the parallax adjustment unit 5, and the parallax adjustment unit 5 includes each screen included in the input parameter information P. Referring to the center position information, size information, and screen surface tilt information, calculate the area of the discontinuous area at the border by the frame frame, and if the value is greater than or equal to the predetermined value, the panel border is discontinuous The process proceeds to step S11. Otherwise, the process proceeds to step S8. In the above description, the determination is performed based on the area. However, the distance between adjacent screens may be obtained, and if the value is equal to or less than a predetermined value, it may be determined that the distance is discontinuous.

  In step S11, the parallax adjusting unit 5 calculates the parallax of the region adjacent to the boundary in the input stereoscopic image data ID3n, and proceeds to the determination step S12. Note that the parallax calculation method is the same as that described above.

  In the determination step S12, the parallax adjusting unit 5 refers to the calculated parallax, determines whether or not the presentation position of the stereoscopic video reproduced in the region adjacent to the boundary is in front of the screen surface, and When it determines, it progresses to step S13, and when that is not right, it progresses to step S8. When the parallax value is positive, the stereoscopic image is presented in front of the screen surface. When the value is 0, the stereoscopic image is presented at the same position as the screen surface.

  In step S13, the parallax adjustment amount is obtained so that the presentation position of the stereoscopic image reproduced in the region adjacent to the boundary is the screen surface or a disparity that is behind the screen surface. move on. Here, the calculated parallax adjustment amount will be specifically described. What is necessary is just to adjust the parallax of stereo image data ID3n so that the value of the parallax calculated | required by step S11 may be 0 or less.

  In step S7, the parallax adjustment unit 5 performs parallax adjustment based on the calculated parallax adjustment amount, and creates stereoscopic image data ID4n. FIG. 26 is a diagram illustrating a method for performing parallax adjustment on the stereoscopic image data ID3n. Based on the calculated parallax adjustment amount, the parallax adjustment unit 5 sets the subject 8 of the left-eye image data 57L of the stereoscopic image data ID31 and the subject 8 of the left-eye image data 58L of the stereoscopic image data ID32 in the left direction, respectively. By shifting the amount of parallax and shifting the subject 8 of the right-eye image data 57R of the stereoscopic image data ID31 and the subject 8 of the right-eye image data 58R of the stereoscopic image data ID32 in the right direction by the same amount of parallax, respectively. The parallax adjustment is performed so that the value becomes 0 or less. Note that the display unit 6 displays the stereoscopic image data ID4n that has been parallax-adjusted by the parallax adjustment unit 5 in a three-dimensional manner by the same operation as described in step S8 of FIG.

  Here, the effect of performing parallax adjustment by the parallax adjusting means 5 will be described in detail with reference to the drawings. FIG. 27 is a diagram for describing a presentation position of a stereoscopic video when parallax adjustment is performed. In FIG. 27, when the parallax adjustment means 5 does not perform parallax adjustment and the stereoscopic image data ID3n is displayed as it is on the front panel 52 and the floor panel 55, the positions of the front screen 59, the floor screen 60, and the subject 61 are displayed. 3D images are presented.

  In this case, since the front panel 52 has a frame frame, an area that cannot be displayed is generated as shown by the dotted line area 62. As a result, the subject 61 is blocked by the frame frame. A sticking effect occurs. In general, in the binocular stereoscopic view, the effect of sticking the image frame is to the left and right eyes when the object that is projected in front of the screen and is displayed in 3D is obstructed by the image frame such as a frame frame. This is also called the frame effect, which is a phenomenon in which a difference appears and the field of view struggles to appear unstable, or the object sticks to the frame frame to reduce the three-dimensional effect and appear to be deformed. This phenomenon may cause visual fatigue, and there is a problem in that viewing safety of stereoscopic image data is lowered.

In FIG. 27, when the stereoscopic image data ID 4n created in step S7 is displayed on the front panel 52 and the floor panel 55, the three-dimensional image data ID 4n is displayed at the positions of the front screen 63, the floor screen 64, and the subject 65. Each video is presented.
As described above, in the region adjacent to the frame frame, when there is a subject presented in three dimensions before the frame frame, the parallax of the adjacent region is presented in three dimensions behind the frame frame. By performing the parallax adjustment, it is possible to eliminate the image frame sticking effect described above and realize more comfortable and safe stereoscopic vision.

  As described above, the stereoscopic image data display system 1 of the present invention is not a projection screen used in a projector or the like as a plurality of screen surfaces in a conventional stereoscopic image data presentation system having a plurality of screen surfaces. For example, even in the case of using a direct-view type panel where a frame frame exists and the boundary portion of the panel is not smoothly joined, such as a liquid crystal panel, organic EL, plasma display, etc., the frame frame at the boundary portion, The image is cut out in consideration of the arrangement of each display surface, the perspective is corrected according to the inclination of each display surface, and parallax adjustment is performed so that the image is presented in a three-dimensional manner behind the frame frame. This eliminates the effect of sticking to the image frame, which is a phenomenon in which stereoscopic vision is broken or parallax struggle occurs due to disappearance of corresponding points, image frame sticking effect, etc. A safe and comfortable stereoscopic view without visual fatigue can be realized without depending on the presence or absence of the frame and the arrangement of the panel.

  FIG. 28 shows another configuration example of the image display means 6. FIG. 28A shows a view of the arrangement of the two panels of the display means 6 from the front, and FIG. 28B shows a view of the arrangement of the two panels of the display means 6 seen from the side. As shown here, the panel 55 is left as it is so that the lower boundary portion of the frame frame 51 is hidden by the frame frame 54, but only the panel 52 is placed downward compared to the case of FIG. 21. Even in this case, in the same manner, in the area adjacent to the frame frame, when there is a subject presented in three dimensions before the frame frame, the parallax of the adjacent area is presented in three dimensions behind the frame frame. As described above, by adjusting the parallax, the sticking effect can be eliminated, and more comfortable and safe stereoscopic vision can be realized.

In the above description, the case of capturing stereoscopic image data has been described. However, when a planar (2D) image is captured, the processing is not performed in each unit and is output to the image display unit 6 as it is to display the planar image. Also good.
Alternatively, the stereoscopic image data input unit 2 newly creates image data of the stereoscopic image data by performing 2D-3D conversion processing, which is processing for generating a 3D image from the 2D image, on the captured 2D image. May be.

  In the above description, the case where a camera is installed as the stereoscopic image data input unit 2 to capture stereoscopic image data has been described. However, instead of the captured image from the camera, various recording media, It may be possible to directly input image data from the outside, such as Internet distribution and reception of broadcast waves. At this time, the stereoscopic image data input means 2 converts the received image data into image data of a predetermined format and outputs it to the image cutout means 3.

  Also, the stereoscopic image data input means 2 at this time is an HDMI that receives image signals from an external device such as a tuner that receives broadcast waves, a Blu-ray (registered trademark) disk player, for example, instead of a camera. (Registered Trademark) (High-Definition Multimedia Interface) receiver may be used. Here, the image data format of the stereoscopic image data is, for example, a top-and-bottom format (a format in which the left and right images are stored as a one-frame frame image) or a side-by-side format (the left and right images are arranged horizontally) There are various formats such as a format stored as one frame image) and a frame sequential format (a format in which a left image and a right image are input over time).

  In the above description, the case of an image with two left and right viewpoints has been described. For example, it may be multi-view stereoscopic image data taken by a multi-view imaging system. In the same manner as when each image is processed, the image may be cut out, the perspective is corrected, the parallax is adjusted, and the like.

  In the above description, the case where a plurality of panels are arranged in the vertical direction has been described. However, the panels may be arranged in the horizontal direction. In this case, cutout, correction of perspective distortion, and parallax adjustment on the boundary surface may be performed in the same manner as in the vertical direction in accordance with the position and orientation of the camera at the time of shooting and the screen surface used for display.

  The present invention is not limited to the above-described stereoscopic image data display system such as a stereoscopic television, but also a stereoscopic digital camera, a stereoscopic digital movie, a stereoscopic digital video recorder, a stereoscopic portable movie player, a stereoscopic mobile phone, and a stereoscopic car navigation. In a system, a 3D portable DVD player, a 3D PC, and the like, the present invention can be widely applied to a system that can output and display a 3D video signal for a plurality of screens.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design changes and the like without departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Stereo image data display system, 2 ... Stereo image data input means, 3 ... Image clipping means, 4 ... Distortion correction means, 5 ... Disparity adjustment means, 6 ... Image display means, 7L ... Left camera, 7R ... Right camera, 8, 43, 46, 49, 61, 65 ... subject, 9, 41, 44, 47, 59, 63 ... front screen, 10, 42, 45, 48, 60, 64 ... floor screen, 11L ... left camera Optical axis, 11R ... right camera optical axis, 12 ... line segment, 13L, 13R, 14, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37 ... point, 15L ... projection plane , 17L, 18L, 19L, 20L, 57L, 58L, 100 ... left-eye image data, 17R, 18R, 19R, 20R, 57R, 58R, 101 ... right-eye image data, 20, 22 ... farthest point, 21, 23 ... Recent points 24, 107 ... Left eye, 25, 108 ... Right eye, 26, 50, 53 ... Screen, 38 ... Viewer, 39, 52 ... Front panel, 40, 55 ... Floor panel, 51, 54 ... Frame frame, 56, 62 ... Area, 102 ... Active shutter glasses, 103 ... Lens shutter for left eye, 104 ... Lens shutter for right eye, 105 ... Image display panel, 106 ... Paralux barrier.

Claims (7)

A stereoscopic image data generation system for generating stereoscopic image data generated from a plurality of viewpoint image data for a plurality of image display surfaces,
Stereoscopic image input means for inputting the plurality of viewpoint image data;
Image cutout means for cutting out image data for each of the plurality of image display surfaces from the plurality of viewpoint image data using the arrangement information and the tilt information of the plurality of image display surfaces;
Distortion correction means for correcting the perspective of the image data for each of the plurality of image display surfaces using the tilt information;
A stereoscopic image data generation system comprising: parallax adjustment means for performing parallax adjustment on the image data for each of the plurality of image display surfaces after correcting the perspective using the arrangement information.   The stereoscopic image data generation system according to claim 1, wherein the parallax adjustment unit performs parallax adjustment so that the parallax adjustment amount is the same value for the image data for each of the plurality of image display surfaces. Stereo image data generation system characterized by   3. The stereoscopic image data generation system according to claim 1, wherein the parallax adjustment unit obtains an arrangement of a non-display area at a boundary of each image display surface from the arrangement information, and each image adjacent to the non-display area. A stereoscopic image data generation system that performs parallax adjustment so that a parallax in a region of a display surface is a parallax in a direction of being retracted deeper than each of the image display surfaces.   A stereoscopic image display system comprising image display means having a plurality of image display surfaces, wherein the image display means is used for stereoscopic viewing from the stereoscopic image data generation system according to any one of claims 1 to 3. A stereoscopic image data display system that receives image data and displays a stereoscopic image on the plurality of image display surfaces. A stereoscopic image data display method for displaying stereoscopic image data generated from a plurality of viewpoint image data on a plurality of image display surfaces,
A stereoscopic image input step of generating or inputting the plurality of viewpoint image data;
An image cut-out step of cutting out image data for each of the plurality of image display surfaces from the plurality of viewpoint image data using the arrangement information and the inclination information of the plurality of image display surfaces;
A distortion correction step of correcting the perspective of the image data for each of the plurality of image display surfaces using the tilt information;
A stereoscopic image data display method comprising: a parallax adjustment step of performing parallax adjustment on the image data for each of the plurality of image display surfaces after correcting the perspective using the arrangement information.   6. The stereoscopic image data display method according to claim 5, wherein the parallax adjustment step performs parallax adjustment so that the parallax adjustment amount has the same value for the image data for each of the plurality of image display surfaces. A method for displaying stereoscopic image data, comprising:   The stereoscopic image data display method according to claim 5 or 6, wherein the parallax adjustment step obtains an arrangement of a non-display area at a boundary of each image display surface from the arrangement information, and each of the adjoining non-display areas. A stereoscopic image data display method, wherein parallax adjustment is performed so that parallax in a region of an image display surface is parallax in a direction of being retracted deeper than each of the image display surfaces.

Images