JPWO2012140841A1 - Image display method, control device, glasses, image system, and composition device - Google Patents

Abstract

A plurality of images with points A, B, and C as viewpoints are sequentially displayed, and the inclination (θ) of the parallax line (SL) is detected. According to the detected inclination, a target image pair that is two of the plurality of images is selected, and when any image of the target image pair is displayed, the image of the left eye and the right eye is displayed. When one non-corresponding visual field is shielded and an image other than the target image pair is displayed, the left eye part and the right eye part of the glasses are controlled so as to shield both the left eye field and the right eye field. The target image pair has viewpoints at both end points of the straight line with the longest orthogonal projection of the parallax line (SL) to the straight line among the straight lines AB, AC, and BC having the distance between the eyes (L0). Two images. According to the technique, in a binocular stereoscopic image system, a favorable stereoscopic effect is given to an observer even when the observer's glasses are tilted or there are a plurality of observers with different positions. .

Description

  The present invention relates to a technique for obtaining a stereoscopic view from a binocular parallax image.

  As a stereoscopic image display method, a lenticular method (Patent Document 1), a binocular stereoscopic method, and the like are known.

  The binocular stereo viewing method is a method for obtaining a stereoscopic view using binocular parallax, displaying binocular parallax images continuously, and the glasses worn by the observer are images from the viewpoint of the left eye. When the image is displayed, the visual field of the right eye is shielded, and when the image from the viewpoint of the right eye is displayed, the visual field of the left eye is shielded to give the observer a stereoscopic view. Note that “binocular parallax” means a difference in the position or viewing direction of the image seen by the left eye and the right eye, and is also simply referred to as “parallax” in the following description. In addition, “binocular parallax images” are two images of the same subject with the viewpoints of two positions that are separated from each other by a distance that substantially coincides with the distance between both eyes.

Various systems have been proposed for binocular stereoscopic viewing.
For example, the systems disclosed in Patent Literature 2 and Patent Literature 3 continuously display a plurality of image signals captured from a plurality of directions with respect to a subject on an image display device, and also the observer of the image display device. Detect the direction of gaze. Then, the spectacles worn by the observer select two image signals captured from the direction matching the detected line of sight, and control the timing of shielding the left eye field and the right eye field.

  Further, the systems disclosed in Patent Document 4 and Patent Document 5 detect the inclination of the glasses, and two images to be displayed from images captured from a plurality of different directions with respect to the subject according to the detected inclination. A system for selecting and displaying is disclosed.

Japanese Patent Laid-Open No. 06-148863 Japanese Patent Laid-Open No. 04-241593 JP 2002-300606 A JP 2006-84963 A Japanese Patent Laid-Open No. 11-341517

  In the systems disclosed in Patent Literature 2 and Patent Literature 3, the eyeglasses select an image signal based on the direction of the observer's line of sight with respect to the display device. When the observer moves from the display screen to a line arranged in a radial pattern around the display device, the direction of the observer's line of sight changes according to the movement, but the observer is lying down When the eyeglasses worn by the person are inclined with respect to the line of sight, the line of sight of the observer with respect to the display device does not change. For this reason, in the systems disclosed in Patent Document 2 and Patent Document 3, when the viewer's glasses are tilted, the viewer cannot be given good stereoscopic vision.

  In the systems disclosed in Patent Literature 4 and Patent Literature 5, two appropriate images are selected and displayed by the display device. Therefore, these systems cannot be applied when there are a plurality of observers with different positions.

  The present invention has been made in view of the above circumstances, and in a binocular stereoscopic vision image system, even when the viewer's glasses are tilted, it is possible to give a good stereoscopic view to the observer, Provided is a technique applicable even when there are a plurality of observers with different positions.

  One aspect of the present invention is an image display method. This image display method sequentially displays a plurality of images of the same subject from a plurality of different viewpoints, and is worn by an observer of the displayed image, and can be used to shield the left and right eyes of the observer. The inclination of the parallax line that is a straight line connecting the left eye part and the right eye part is detected, and a target image pair consisting of two of the plurality of images is selected according to the detected inclination, and any of the target image pairs is selected. When the image is displayed, one of the left eye and the right eye that does not correspond to the image is blocked. When an image other than the target image pair is displayed, both the left eye and the right eye are blocked. The left eye part and the right eye part of the glasses are controlled as described above.

  The plurality of viewpoints are three or more viewpoints at different positions on the same plane facing the subject, and each viewpoint is at least one other viewpoint, and a viewpoint pair whose distance is an interval between both eyes. Form.

  The target image pair is two images each having a viewpoint at each end point of the target straight line in a straight line group that is a plurality of straight lines connecting the two viewpoints of the plurality of viewpoint pairs included in the plurality of viewpoints. When it is assumed that the parallax line has moved to the plane, the target straight line is the straight line in which the orthogonal projection of the parallax line to the straight line is the longest in the straight line group.

  Note that a display of the method according to the above aspect replaced with an apparatus or system, a spectacles control method in the above method, an apparatus for executing the control method, a program for causing a computer to execute the control method, and the like are also included in the present invention. It is effective as an embodiment.

  According to the technique of the present invention, in a binocular stereoscopic image system, even when the viewer's glasses are tilted, it is possible to give the viewer good stereoscopic vision and a plurality of observations at different positions. It can also be applied when there is a person.

It is a figure for demonstrating the principle of the technique concerning this invention. It is a figure for demonstrating a parallax angle. It is a figure which shows the relationship between an effective parallax length and a parallax angle. It is a figure which shows the example in the case of using only the equally dividing point of a circle for a viewpoint. It is a figure for demonstrating the case where the technique concerning this invention is applied to the example shown in FIG. 4 (the 1). It is a figure for demonstrating the case where the technique concerning this invention is applied to the example shown in FIG. 4 (the 2). It is a figure for demonstrating the case where the technique concerning this invention is applied to the example shown in FIG. 4 (the 3). It is a figure which shows the effective parallax length obtained when the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows another example in the case of using only the equally dividing point of a circle for a viewpoint. It is a figure which shows a more preferable example than the example shown in FIG. It is a figure for demonstrating the case where the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows the effective parallax length obtained when the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows another example in the case of using only the equally dividing point of a circle for a viewpoint. It is a figure for demonstrating the case where the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows the effective parallax length obtained when the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows another example in the case of using only the equally dividing point of a circle for a viewpoint. It is a figure for demonstrating the case where the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows the effective parallax length obtained when the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows the example in the case of using the equally dividing point and circle center of a circle for a viewpoint. It is a figure for demonstrating the case where the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows the effective parallax length obtained when the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows another example in the case of using the equally dividing point and circle center of a circle for a viewpoint. It is a figure for demonstrating the case where the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows the effective parallax length obtained when the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows another example in the case of using the equally dividing point and circle center of a circle for a viewpoint. It is a figure for demonstrating the case where the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows the effective parallax length obtained when the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows another example in the case of using the equally dividing point and circle center of a circle for a viewpoint. It is a figure for demonstrating the case where the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows the effective parallax length obtained when the technique concerning this invention is applied to the example shown in FIG. It is a figure which shows the image system concerning embodiment of this invention. It is a figure which shows the image acquisition apparatus in the image system shown in FIG. It is a figure which shows the synthetic | combination apparatus in the image system shown in FIG. It is a figure which shows the example of the flow of a process by the synthetic | combination apparatus shown in FIG. It is a figure which shows the spectacles in the image system shown in FIG. It is a figure which shows the control apparatus in the image system shown in FIG. In the image system shown in FIG. 31, it is a figure which shows the correspondence of the inclination of an observer's head, the selected target image pair, and the shielding condition of the visual field of the left eye and the right eye.

  Embodiments of the present invention will be described below with reference to the drawings. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate. It should be understood by those skilled in the art that each element described in the drawings as functional blocks for performing various processes can be realized in various forms by a combination of hardware and software (program). It is not limited to any of the above. Note that, in each drawing, the same element is denoted by the same reference numeral, and redundant description is omitted as necessary.

  Further, the above-described program can be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROM (Read Only Memory) CD-R, CD -R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)). The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

  Before describing specific embodiments of the present invention, the principle of the technology according to the present invention will be described. For convenience of explanation, some terms are first defined.

  As described above, in the binocular stereoscopic viewing method, two images included in the binocular parallax image are displayed in succession, and the eyeglasses worn by the observer display an image from the viewpoint of the left eye. This is a method of giving a viewer a stereoscopic view by shielding the field of view of the right eye when the camera is on, and shielding the field of view of the left eye when an image from the viewpoint of the right eye is displayed. In the following description, the same reference numerals are used for images and viewpoints corresponding to the images. For example, an image having point A as a viewpoint is referred to as “image A”, and the viewpoint of image A is point A.

  In addition, a straight line connecting the left eye part and the right eye part of the glasses worn by the observer is referred to as “parallax line”. The length of the parallax line is the distance L0 between the eyes. The distance L0 between the eyes can be, for example, an average distance between the eyes of both eyes.

Here, with reference to FIG. 1, “viewpoint pair”, “parallax angle”, “effective parallax length”, and “tilt of parallax line” are defined.
The left part of FIG. 1 shows the positional relationship between the viewpoints (A, B) of two images included in the binocular parallax image and the subject 1. The length of the straight line AB connecting the viewpoints A and B is the distance L0 between both eyes, and when viewed from the opposite side of the subject across the straight line AB, A is the left end point and B is the right end point. Hereinafter, the viewpoints of two images included in the binocular parallax image, that is, the two viewpoints whose distance is the distance between the eyes L0 are referred to as “viewpoint pairs”.

  When observing the displayed image, it is considered that the parallax line of the observer is usually included in a plane parallel to the display screen. A straight line SL shown in the right part of FIG. B is an orthogonal projection of the parallax line on the display screen 2, and an arrow of the straight line SL indicates a direction from the left eye part to the right eye part of the parallax line.

  A straight line A′B ′ on the display screen 2 is defined between the straight line AB and the subject 1 so that the straight line AB is located on the display screen and the point A is an end point corresponding to the left eye portion of the straight line SL. A straight line AB and a straight line AB when the subject is moved without changing the relative positional relationship.

  When observing the image A and the image B by the binocular stereoscopic method, when the straight line A′B ′ and the straight line SL overlap, that is, when the angle α formed by the straight line A′B ′ and the straight line SL is “0”. The best stereoscopic effect can be obtained. In this specification, the angle α formed by the straight line A′B ′ and the straight line SL is referred to as “parallax angle”, and the length of the orthogonal projection of the straight line SL onto the straight line A′B ′ (L1 in the figure) is “effective parallax length”. " It should be noted that the “parallax angle” has a different meaning from the angle normally called “convergence angle” in the binocular stereoscopic vision system.

  For easy understanding, the straight line A′B ′ will be described as a straight line AB and the straight line SL will be described as a parallax line.

  Further, “inclination of parallax line” is an angle formed between the straight line SL and a predetermined reference axis included in the display screen. This predetermined reference axis means a straight line whose angle formed with the parallax line is “0” in a normally assumed observation situation.

  For example, when the display screen of the image display device is perpendicular to the ground, a situation in which “the observer goes to the screen without tilting the head upward and the head to the left or right” is normally assumed. It can be said that it is an observation situation. In this case, the reference axis is a horizontal straight line from the left to the right as viewed from the screen.

  In addition, for example, when the display screen of the screen display device is horizontal with respect to the ground, the situation is that “the observer looks down on the screen from the assumed observation position so that the observer does not tilt the head left and right” Can be said to be the observation situation normally assumed. In this case, the reference axis is a horizontal straight line from the left to the right when the screen is viewed from a normally assumed observation position.

  Also, the viewpoint A and viewpoint B of the two images of the binocular parallax image are usually such that the straight line AB is parallel to the reference axis and the direction from the point A to the point B is the same as the reference axis. Is set. That is, in the normally assumed observation situation, there is no inclination of the parallax line, the parallax angle α is 0 degree, and the effective parallax length is the inter-eye distance L0.

  By the way, when the observer tilts his / her head or when the observer is at a position different from the normal observation position, the parallax line is inclined and the parallax angle α is not 0 degree.

  The relationship between the parallax angle α and the stereoscopic effect will be considered with reference to FIG. The reference axis is a straight line AB.

  As shown in FIG. 2, when the parallax angle α changes within a range of “0” to “360” degrees, the length of the orthogonal projection of the parallax line SL onto the straight line AB changes. The length of the orthogonal projection is hereinafter referred to as “orthographic projection length”, and in the example of FIG. 2, the orthogonal projection length L1 is the same as the effective parallax length.

The orthogonal projection length L1 can be represented by the product of the distance L0 between both eyes and the absolute value of the cosine value of the parallax angle α, as shown in the following equation (1).
L1 = L0 × | cos α | (1)

  FIG. 3 shows the relationship between the effective parallax length (orthographic projection length L1) and the parallax angle α. In FIG. 3, the solid curve indicates the effective parallax length, and the dotted curve indicates “L0 × cos α”. When the parallax angle α is in the range of “0 degrees to 90 degrees” and “270 degrees to 360 degrees”, the orthographic projection length L1 and “L0 × cos α” overlap each other, so that a dotted line representing “L0 × cos α” is displayed. Shown shifted down.

0 degree ≦ parallax angle α <90 degrees In this case, the orthogonal projection length L1 decreases as α increases. That is, when the parallax angle α is 0 degree, the effective parallax length is the inter-eye distance L0, and the best stereoscopic effect can be obtained. The larger the parallax angle α, the shorter the effective parallax length and the worse the stereoscopic effect. In this angle range, when observing the images A and B by the binocular stereoscopic method, the right eye field is displayed when the image A is displayed, and the left eye is displayed when the image B is displayed. The field of view will be shielded.

・ Parallax angle α = 90
In this case, there is no orthogonal projection of the parallax line SL onto the straight line AB, and the orthogonal projection length L1 is “0”. The observer sees two images that are shifted, and cannot obtain a stereoscopic effect.

90 degrees <parallax angle α ≦ 180 degrees In this case, when observing the images A and B by the binocular stereoscopic method, the image is opposite to the case of “0 degrees ≦ parallax angle α <90 degrees”. It is necessary to shield the left eye field of view when A is displayed and the right eye field of view when image B is displayed.

  Further, as can be seen from FIG. 3, when the parallax angle α is within this angle range, the orthogonal projection length L1, that is, the effective parallax length, increases as the parallax angle α increases. Therefore, the stereoscopic effect is improved as the parallax angle α is increased.

180 degrees ≦ parallax angle α <270 degrees Even in this case, the left eye field of view is blocked when the image A is displayed, and the right eye field of view is blocked when the image B is displayed.

  Further, as can be seen from FIG. 3, when the parallax angle α is in this angle range, the orthogonal projection length L1, that is, the effective parallax length, decreases as the parallax angle α increases. Therefore, the stereoscopic effect is also worsened as the parallax angle α is larger.

・ Parallax angle α = 270
In this case, there is no orthogonal projection of the parallax line SL onto the straight line AB, and the orthogonal projection length L1 is “0”. The observer sees two images that are shifted, and cannot obtain a stereoscopic effect.

270 degrees <parallax angle α ≦ 360 degrees In this case, as in the case of “0 degrees ≦ parallax angle α <90 degrees”, when the image A is displayed, the field of view of the right eye is displayed. The left eye field of view will be shielded.

  Further, as can be seen from FIG. 3, when the parallax angle α is within this angle range, the orthogonal projection length L1, that is, the effective parallax length, increases as the parallax angle α increases. Therefore, the stereoscopic effect is improved as the parallax angle α is increased.

  Thus, with the change in the parallax angle α, the effective parallax length varies between the shortest value “0” and the longest value “L0”.

  Based on this knowledge, the inventor of the present application can give a good stereoscopic view to an observer even when the eyeglasses of the observer are tilted in a binocular stereoscopic image system, and a plurality of different positions. We have established a method that can be applied even when there is an observer.

  This method sequentially displays a plurality of images of a subject from three or more viewpoints at different positions on the same plane facing the subject. Each viewpoint forms a viewpoint pair with at least one other viewpoint.

  Then, according to the inclination of a straight line (parallax line SL) connecting the left eye part and the right eye part of the glasses worn by the observer, a target image pair composed of two of the plurality of images is selected and the target When any image of the image pair is displayed, one of the left eye and the right eye that does not correspond to the image is blocked. When an image other than the target image pair is displayed, either the left eye or the right eye is displayed. The left eye part and the right eye part of the glasses are controlled so as to shield the visual field.

  Note that the target image pair is two images with the two end points of the target straight line as viewpoints. The target straight line is assumed to have moved the parallax line on the plane in which the plurality of viewpoints are located in a straight line group that is a plurality of straight lines connecting two viewpoints of a plurality of viewpoint pairs included in the plurality of viewpoints. In this case, the orthogonal projection of the parallax line SL onto the straight line is the longest straight line.

  The plurality of viewpoints can be, for example, a plurality of equally divided points that divide the circumference of a circle into n equal parts (n: an integer of 3 or more).

  Here, first, a case where n is “3”, that is, a case where a plurality of images each using three equal dividing points for dividing a circle into three is used will be described.

  As shown in FIG. 4, points A, B, and C are equally divided points that divide the circle RC into three equal parts. Further, the length of the straight line (AB, CB, AC) connecting any two of the three equally divided points is the distance L0 between the eyes. That is, points A, B, and C are three vertices of an equilateral triangle whose side length is the interval L0 between the eyes, and points A, B, and C are (A, B), (B, C), Three viewpoint pairs (A, C) are formed.

  The angle formed by the parallax line SL and the straight line AB, that is, the inclination of the parallax line SL is θ. Therefore, the angle formed by the parallax line SL and the straight line AC is “60−θ”, and the angle formed by the parallax line SL and the straight line BC is “θ + 60”. That is, when the straight line AB is selected as the target straight line, that is, when the image A and the image B are selected as the target image pair, the parallax angle α is θ. When selecting the straight line AC as the target straight line, that is, when selecting the image A and the image C as the target image pair, the parallax angle α is “60−θ”. When the straight line BC is selected as the target straight line, that is, when the image B and the image C are selected as the target image pair, the parallax angle α is “θ + 60”.

  The orthogonal projection lengths of the parallax lines SL onto the straight lines AB, AC, and BC are L1, L2, and L3, respectively, and can be obtained by the following equations (2) to (4).

L1 = L0 × | cos θ | (2)
L2 = L0 × | cos (60−θ) | (3)
L3 = L0 × | cos (θ + 60) | (4)

FIG. 5 shows the relationship between the orthogonal projection lengths L1, L2, and L3 and the angle θ.
As shown in FIG. 5, the orthogonal projection length L1 is the longest when “0 ≦ θ ≦ 30”. At this time, the minimum value of the orthogonal projection length L1 is “L0 × cos30”.

  When “30 ≦ θ ≦ 90”, the orthogonal projection length L2 is the longest. At this time, the minimum value of the orthogonal projection length L2 is “L0 × cos30”.

  When “90 ≦ θ ≦ 150”, the orthogonal projection length L3 is the longest. At this time, the minimum value of the orthogonal projection length L3 is “L0 × cos30”.

  When “150 ≦ θ ≦ 210”, the longest orthographic projection length becomes the orthographic projection length L1 again. At this time, the minimum value of the orthogonal projection length L1 is “L0 × cos30”.

  When “210 ≦ θ ≦ 270”, the longest orthographic projection length becomes the orthographic projection length L2 again. At this time, the minimum value of the orthogonal projection length L2 is “L0 × cos30”.

  When “270 ≦ θ ≦ 330”, the longest orthographic projection length becomes the orthographic projection length L3 again. At this time, the minimum value of the orthogonal projection length L3 is “L0 × cos30”.

  When “330 ≦ θ ≦ 360”, the longest orthographic projection length is the orthographic projection length L1. At this time, the minimum value of the orthogonal projection length L1 is “L0 × cos30”.

  When only the images (A, B) are selected as the target image pair, the orthogonal projection length L1 is the effective parallax length. As shown in FIG. 5, in this case, the effective parallax length, that is, the orthogonal projection length L1, varies within a range from “0” to “L0” according to the angle θ.

  When only the images (A, C) are selected as the target image pair, the orthogonal projection length L2 is the effective parallax length. Even in this case, the effective parallax length varies within a range of “0” or more and “L0” or less according to the angle θ.

  Similarly, when only the images (B, C) are selected as the target image pair, the effective parallax length, that is, the orthogonal projection length L3, fluctuates within a range of “0” or more and “L0” or less.

With this, the observer cannot obtain a good stereoscopic effect.
The technique according to the present invention is the straight line having the longest orthogonal projection length of the parallax line SL when selecting a target straight line from a straight line group including straight lines (here, straight lines AB, AC, BC) connecting the two viewpoints of each viewpoint pair. Is elected. As a result of the selection of the target straight line, images having respective viewpoints at both end points of the target straight line are selected as the target image pair.

  FIG. 6 shows a target image pair selected according to the angle θ when the angle θ changes from 0 degrees to 180 degrees. FIG. 7 shows a target image pair selected according to the angle θ when the angle θ changes from 180 degrees to 360 degrees.

  As shown in FIG. 6, when the angle θ increases from 0 degree to 30 degrees, the orthogonal projection length L1 is shortened from “L0” to (L0 × cos30), and the orthogonal projection length L2 is “L0 × 1 / 2 ”to“ L0 × cos30 ”. The orthogonal projection length L3 is shortened from “L0 × 1/2” to “0”. Therefore, when the angle θ is less than 30 degrees, the straight line AB is selected as the target straight line, and the image (A, B) is selected as the target image pair.

  When the angle θ is 30 degrees, the orthogonal projection lengths L1 and L2 are both “L0 × cos30”, and the orthogonal projection length L3 is “0”. Therefore, there are two target straight lines, a straight line AB and a straight line AC. At this time, either the straight line AC or the straight line AC may be selected as the target straight line.

  FIG. 8 shows the relationship between the effective parallax length and the angle θ when the target straight line and the target image pair are selected according to the angle θ that varies within the range of 0 to 360, as shown in FIGS. 6 and 7. Indicates.

  As shown in FIG. 8, regardless of the angle θ, the effective parallax length is always “L0 × cos30” or more. As described above, the best stereoscopic effect can be obtained when the parallax angle α is “0”, that is, when the effective parallax length is the inter-eye distance L0. In this example, even in the worst case, it is possible to obtain a stereoscopic effect that is “cos 30” times as large as the best (approximately 86%).

  Further, even when there are a plurality of observers at different positions, an optimal target image pair can be selected according to the inclination of the parallax line SL of each observer. Can be given.

  In the example shown in FIG. 4, when n is “3”, that is, three equal points (three vertices of an equilateral triangle) that divide the circumference of the circle RC into three equal parts are respectively viewed. This is an example in which a viewpoint pair is formed by a combination of all two viewpoints. The technique according to the present invention can also be applied when n is an arbitrary integer of 3 or more.

  FIG. 9 shows an example in which n is “4”, that is, four equal points (A, B, C, D) that divide the circle RC into four equal parts are used as viewpoints. Further, the example shown in FIG. 9 is an example in the case where two adjacent equal dividing points are set as viewpoint pairs.

  As illustrated, in this case, the straight line AB and the straight line DC are parallel to each other, and the straight line AD and the straight line BC are also parallel to each other. Therefore, regardless of the angle θ, the orthogonal projection length of the parallax line SL onto the straight line AB and the orthogonal projection length onto the straight line DC are the same, and the orthogonal projection length of the parallax line SL onto the straight line AD and the straight line BC The orthographic projection length to is the same. In this case, when the straight line AB is aligned with the reference axis, viewpoint pairs that form both end points of the straight line DC are wasted. Moreover, only the same orthogonal projection length can be obtained regardless of which of the straight line AD and the straight line BC is selected.

  Therefore, the inventor of the present application paid attention to a central angle formed by two of a plurality of equal dividing points that divide a circle into n equal parts (n: an integer of 3 or more) and a circle center. Then, a method has been conceived in which a viewpoint pair is formed by two equal dividing points having a central angle of (360 / n) × int (n / 2) among the plurality of equal dividing points. Here, int () is a function that returns an integer with the decimal point rounded down from the value in (). As shown in FIG. 4, this condition is naturally satisfied when each vertex of the triangle is the viewpoint.

  For example, when n is 4, two equal points that form a viewpoint pair become two equal points that form a central angle of 180 degrees. Therefore, as shown in FIG. B forms one viewpoint pair, and point D and point C form one viewpoint pair.

  As shown in FIG. 10, the equal dividing points A, B, C, and D divide the circle RC into four equal parts. The length of the straight line AB and the straight line BC, that is, the diameter of the circle RC is the distance L0 between the eyes. When the length of the orthogonal projection of the parallax line SL onto the straight line AB and the straight line BC is expressed by L1 and L2, the orthogonal projection lengths L1 and L2 can be obtained by the following equations (5) and (6).

L1 = L0 × | cos θ | (5)
L2 = L0 × | cos (90−θ) | (6)

FIG. 11 shows the relationship between the orthogonal projection lengths L1 and L2 and the angle θ.
As in the example shown in FIG. 4, at any angle θ, the straight line with the longest orthographic projection of the parallax line SL is selected as the target straight line from the straight line group (here, the straight lines AB and BC). An effective parallax length as shown in FIG. 12 can be obtained.

  As can be seen from FIG. 12, in this case, the effective parallax length is always “L0 × cos 45” or more regardless of the angle θ. That is, even in the worst case, it is possible to obtain a stereoscopic effect about “cos 45” times (approximately 71%) of the best case.

  A pair of viewpoints is formed by dividing the circle into n equal parts (n: an integer equal to or greater than 3), with a central angle of (360 / n) x int (n / 2) at two equal points. When n is an even number, only n / 2 viewpoint pairs as shown in FIG. 10 can be obtained. However, when n is an odd number, n viewpoint pairs can be obtained.

  FIG. 13 shows an example where n is 5. As shown, the equal points A, D, E, B, C divide the circle RC into five equal parts. The center of the circle RC is the point O. Since the central angles AOB, AOE, BOD, COE, COD, and COE are (360/5) × int (5/2), A and B, A and E, B and D, C and D, and C and E Five parallax pairs are configured. The length of the straight line connecting the two viewpoints of each parallax pair is the distance L0 between both eyes.

  When the lengths of the orthogonal projections of the parallax lines SL on the straight lines AB, BD, DC, CE, and AE are respectively expressed as L1 to L5, these orthogonal projection lengths can be obtained by Expressions (7) to (11). .

L1 = L0 × | cos θ | (7)
L2 = L0 × | cos (36−θ) | (8)
L3 = L0 × | cos θ (72−θ) | (9)
L4 = L0 × | cos (108−θ) | (10)
L5 = L0 × | cos (144−θ) | (11)

FIG. 14 shows the relationship between the orthogonal projection lengths L1 to L5 and the angle θ.
As in the example shown in FIG. 4, at any angle θ, the straight line having the longest orthographic projection of the parallax line SL from the straight line group (here, straight lines AB, BD, DC, CE, AE) is set as the target straight line. When selected, an effective parallax length as shown in FIG. 15 can be obtained.

  As can be seen from FIG. 15, in this case, the effective parallax length is always “L0 × cos 18” or more regardless of the angle θ. That is, even in the worst case, it is possible to obtain a stereoscopic effect that is about “cos 18” times (approximately 95%) that of the best case.

  FIG. 16 shows an example where n is 7. As shown in the figure, the equivalence points A, E, F, G, B, C, and D divide the circle RC into seven equal parts. The center of the circle RC is the point O. Since the central angles AOB, AOG, BOE, COE, COF, DOF, and DOG are (360/7) × int (7/2), A and B, A and G, B and E, C and E, and C F, D and F, and D and G constitute seven parallax pairs. The length of the straight line connecting the two viewpoints of each parallax pair is the distance L0 between both eyes.

  When the length of the orthogonal projection of the parallax line SL to the straight line connecting the two viewpoints of each viewpoint pair is represented by L1 to L7, the relationship between these orthogonal projection lengths and the angle θ is as shown in FIG.

  Similarly to the above, at any angle θ, a straight line having the longest orthogonal projection of the parallax line SL is selected as a target straight line from a straight line group (here, straight lines AB, EG, DC, FB, AE, CG, DF). By doing so, an effective parallax length as shown in FIG. 18 can be obtained.

  As can be seen from FIG. 18, in this case, the effective parallax length is always “L0 × cos13” or more regardless of the angle θ. That is, even in the worst case, it is possible to obtain a stereoscopic effect about “cos 13” times (approximately 97%) of the best case.

  Therefore, the larger the number n, the more the deterioration of the stereoscopic effect due to the inclination of the parallax line SL and the change in the position of the observer can be suppressed.

  On the other hand, the technique according to the present invention sequentially displays a plurality of images each having a plurality of viewpoints, and when any image of a target image pair included in the plurality of images is displayed, And the field of view that does not correspond to the image of the right eye is shielded, and when the other image is displayed, the fields of view of both the left eye and the right eye are shielded. Therefore, as n is larger, the number of the plurality of images increases, and the ratio of the time during which the visual fields of both eyes are shielded to the total time for displaying the plurality of images increases. As a result, the observer may feel dark. This problem is particularly noticeable in the case of an image that is originally not bright.

  Therefore, it is preferable to determine n according to the balance between the stereoscopic effect and the brightness according to the purpose and the like. For example, if a stereoscopic effect of about 86% or more at the best time is sufficient, n may be set to 3.

  In the above, a case has been described in which a plurality of images whose viewpoints are equally divided points that equally divide the circumference of the circle RC are used. The technique according to the present invention may use an image having a circular center as a viewpoint in addition to the equally divided points.

  In this case, the radius of the circle RC is the interval L0 between the eyes, and the viewpoint pair is formed by a circle center and each equally dividing point that divides the circle RC into n equal parts (n: an integer of 3 or more).

  First, the case where n is “3”, that is, the case where four images having the viewpoints of the three equal dividing points dividing the circumference into three and the center of the circle are described in detail.

  As shown in FIG. 19, the circle RC has a circle center A and a radius L2 between the eyes. B, C, and D divide the circle RC into three equal parts. The viewpoint pairs are (A, B), (A, C), (A, D).

  The angle formed by the parallax line SL and the straight line AB, that is, the inclination of the parallax line SL is θ. Therefore, the angle formed between the parallax line SL and the straight line AC is “θ + 120”, and the angle formed between the parallax line SL and the straight line AD is “θ + 240”. That is, when the straight line AB is selected as the target straight line, that is, when the image A and the image B are selected as the target image pair, the parallax angle α is θ. When the straight line AC is selected as the target straight line, that is, when the image A and the image C are selected as the target image pair, the parallax angle α is “θ + 120”. Further, when the straight line AD is selected as the target straight line, that is, when the image A and the image D are selected as the target image pair, the parallax angle α is “θ + 240”.

  The orthogonal projection lengths of the parallax lines SL onto the straight lines AB, AC, and AD are L1, L2, and L3, respectively, and can be obtained by the following equations (12) to (14).

L1 = L0 × | cos θ | (12)
L2 = L0 × | cos (θ + 120 | (13)
L3 = L0 × | cos (θ + 240) | (14)

FIG. 20 shows the relationship between the orthogonal projection lengths L1, L2, and L3 and the angle θ.
As shown in FIG. 20, the orthogonal projection length L1 is the longest when “0 ≦ θ ≦ 30”. At this time, the minimum value of the orthogonal projection length L1 is “L0 × cos30”.

  When “30 ≦ θ ≦ 90”, the orthogonal projection length L3 is the longest. At this time, the minimum value of the orthogonal projection length L3 is “L0 × cos30”.

  When “90 ≦ θ ≦ 150”, the orthogonal projection length L2 is the longest. At this time, the minimum value of the orthogonal projection length L2 is “L0 × cos30”.

  When “150 ≦ θ ≦ 210”, the longest orthographic projection length becomes the orthographic projection length L1 again. At this time, the minimum value of the orthogonal projection length L1 is “L0 × cos30”.

  When “210 ≦ θ ≦ 270”, the longest orthographic projection length becomes the orthographic projection length L3 again. At this time, the minimum value of the orthogonal projection length L3 is “L0 × cos30”.

  When “270 ≦ θ ≦ 330”, the longest orthographic projection length becomes the orthographic projection length L2 again. At this time, the minimum value of the orthogonal projection length L2 is “L0 × cos30”.

  When “330 ≦ θ ≦ 360”, the longest orthographic projection length is the orthographic projection length L1. At this time, the minimum value of the orthogonal projection length L1 is “L0 × cos30”.

  Therefore, when the target straight line is selected from the straight line group having the straight lines AB, AC, and AD, the straight line having the longest orthogonal projection length of the parallax line SL is selected. As a result of the selection of the target straight line, images having respective viewpoints at both end points of the target straight line are selected as the target image pair.

  For example, when the angle θ is 30 degrees, the orthogonal projection lengths L1 and L3 are both “L0 × cos30”, and the orthogonal projection length L2 is “0”. Therefore, there are two target straight lines, a straight line AB and a straight line AD. At this time, when the straight line AD is selected as the target straight line, the image (D, A) is selected as the target image pair, and the image A becomes the image for the right eye. However, since the image A is selected as the image for the left eye when θ is less than 30 degrees, if the image A becomes the image for the right eye when θ is 30 degrees, the viewer feels uncomfortable. . Therefore, when the angle θ is 30 degrees, it is preferable to provide a hysteresis according to the rotation direction of the parallax line SL so that the straight line AB is selected as the target straight line.

  FIG. 21 shows the relationship between the effective parallax length and the angle θ obtained as a result of the selection. As shown in the drawing, the effective parallax length is always “L0 × cos30” or more regardless of the angle θ. Therefore, in this example, a stereoscopic effect that is “cos 30” times as large as the best (approximately 86%) can be obtained even in the worst case.

  Further, even when there are a plurality of observers at different positions, an optimal target image pair can be selected according to the inclination of the parallax line SL of each observer. Can be given.

  The example shown in FIG. 19 is an example in the case where n is “3”, that is, when four images are used with the viewpoints of three equal points that divide the circumference of the circle RC into three equal parts and the center of the circle. It is. n may be any number of 3 or more.

  For example, as shown in FIG. 22, when n is “12”, that is, when 13 images having 12 equal dividing points for dividing the circle RC into 12 points and a circle center are used, FIG. 23 shows the relationship between the orthogonal projection length of the parallax line SL on the straight line in the radial direction connecting the heart and the respective equally divided points and the angle θ. Therefore, when the straight line having the longest orthogonal projection length of the parallax line SL is selected as the target straight line according to the angle θ, the effective parallax length shown in FIG. 24 can be obtained.

  As can be seen from FIG. 25, in this case, an effective parallax length of L0 × | cos15 | or more can always be obtained regardless of the angle θ. That is, even in the worst case, it is possible to obtain a stereoscopic effect that is “cos 15” times as large as the best (approximately 96%).

  In the above description, a case has been described in which all the equally dividing points that equally divide the circumference of the circle RC and a plurality of images each having the circle center as the viewpoint are used. Only a part of these equally divided points may be used. For example, when n is an even number of 4 or more, “n / 2” equally divided points that are continuous along the circumference among the equally divided points that divide the circle RC into n equal parts and the center of the circle are used. You may do it.

  As an example, a case where n is “4”, that is, a case where three images using two of the four equal dividing points that divide the circumference RC into four and the center of the circle are used in detail is used. explain.

  As shown in FIG. 25, the equally dividing points B, C, D, and E divide the circle RC having a radius L0 between the eyes into four equal parts. Here, only the equally divided points B and C are used. That is, only (A, B) and (A, C) are used as viewpoint pairs. When the length of the orthogonal projection of the parallax line SL onto the straight line AB and the straight line AC is expressed by L1 and L2, the orthogonal projection lengths L1 and L2 can be obtained by the following equations (15) and (16).

L1 = L0 × | cos θ | (15)
L2 = L0 × | cos (θ + 90 | (16)

FIG. 26 shows the relationship between the orthogonal projection lengths L1 and L2 and the angle θ.
When the straight line having the longest orthogonal projection of the parallax line SL is selected as the target straight line from the straight line group (here, the straight lines AB and AC) at any angle θ, an effective parallax length as shown in FIG. 27 is obtained. be able to.

  As can be seen from FIG. 27, in this case, the effective parallax length is always “L0 × cos 45” or more regardless of the angle θ. That is, even in the worst case, it is possible to obtain a stereoscopic effect that is approximately “cos 45” times (approximately 70%) that is the best.

  Next, a case where n is “6”, that is, a case where three images out of six equal points that divide the circumference RC into six and four images respectively having the viewpoint of the center of the circle will be described.

  As shown in FIG. 28, the equally dividing points B, C, D, E, F, and G divide the circle RC having a radius L0 between the eyes into 6 equal parts. Here, only the equivalence points B, C, and D are used. That is, only (A, B), (A, C), (A, D) are used as the parallax pairs. When the lengths of the orthogonal projections of the parallax lines SL onto the straight line AB, the straight line AC, and the straight line AD are represented by L1, L2, and L3, respectively, these orthogonal projection lengths are obtained by the following equations (17) to (19). Can do.

L1 = L0 × | cos θ | (17)
L2 = L0 × | cos (θ + 60 | (18)
L2 = L0 × | cos (θ + 120) (19)

FIG. 29 shows the relationship between the orthogonal projection lengths L1, L2, and L3 and the angle θ.
When the straight line having the longest orthogonal projection of the parallax line SL is selected as the target straight line from the straight line group (here, straight lines AB, AC, AD) at any angle θ, the effective parallax length as shown in FIG. Can be obtained. As can be seen from FIG. 30, in this case, the effective parallax length is always “L0 × cos30” or more regardless of the angle θ. Therefore, even in the worst case, it is possible to obtain a stereoscopic effect that is “cos 30” times as large as the best (approximately 86%).

  That is, as shown in FIG. 28, when using four images each having a viewpoint of three consecutive dividing points among six equal dividing points dividing the circle RC into six equal parts and the center of the circle, The same stereoscopic effect can be obtained also when using four images having three equally divided points of the circle RC and the center of the circle as shown in FIG.

  Similarly, in the case of using seven images each having six viewpoints of the six equal dividing points and the circle center among the twelve equal dividing points that divide the circle RC into twelve, as shown in FIG. Thus, the same stereoscopic effect can be obtained as in the case of using 13 images with 12 equal points and a circle center as viewpoints.

Here, the example shown in FIG. 2 is compared with the example shown in FIG.
FIG. 2 is an example in which three viewpoint pairs are formed by three equal dividing points that divide a circle into three equal parts. FIG. 28 is an example in which three viewpoint pairs are formed by three consecutive points out of six equal points that divide a circle into six and a circle center. In any case, the effective parallax length is “L0 × cos30” or more. However, in the case of the example of FIG. 2, only three images are necessary, whereas in the example of FIG. 28, four images (A, B, C, D) are necessary.

  Similarly, in order to obtain the same effective parallax length as in the case of forming five viewpoint pairs with five equally divided points that divide the circle into five as in the example shown in FIG. 13, the circle is divided into ten equal parts. It is necessary to form six viewpoint pairs with five consecutive points of ten equal points and a circle center. Also, as in the example shown in FIG. 16, in order to obtain the same effective parallax length as in the case where seven viewpoint pairs are formed by seven equal dividing points that divide the circle into seven equal parts, the circle is divided into 14 equal parts. It is necessary to form eight viewpoint pairs with seven consecutive points and a circle center of the equal dividing points.

  That is, assuming that the same effective parallax length is obtained when n is an odd number of 3 or more, when a viewpoint pair is composed of only n equally dividing points that divide the circle into n, The number of necessary images can be reduced by one as compared to a case where a viewpoint pair is formed of n consecutive dividing points and a circle center among the equally dividing points (N = 2 × n).

Next, the example shown in FIG. 10 is compared with the example shown in FIG.
FIG. 10 shows an example in which two viewpoint pairs are formed by four equal dividing points that divide a circle into four equal parts. FIG. 25 is an example in which two viewpoint pairs are formed by two consecutive points out of four equal points that divide a circle into four and a circle center. In any case, the effective parallax length is “L0 × cos45” or more. However, in the case of the example of FIG. 10, four images (A, B, C, D) are required, whereas in the example of FIG. 25, only three images (A, B, C) are required. is necessary.

  Therefore, when it is sufficient to obtain a stereoscopic effect of about 71% or more at the best time, it is preferable to use a viewpoint and a viewpoint pair as shown in FIG.

  When n is an even number, (360 / n) × int (n / 2) is always “180”, and the viewpoints and viewpoint pairs are only as shown in FIG.

  Therefore, the viewpoint and the viewpoint pair may be determined according to the minimum required effective parallax length on the assumption that the required number of images is as small as possible. In any case, the minimum execution parallax length increases as the number of viewpoint pairs increases, but the number of images displayed increases as the number of viewpoint pairs increases. The greater the number of images required, the longer the device and time required to acquire these images. In addition, as described above, the larger the number of the plurality of images that are candidates for the two images of the target image pair, the ratio of the time that the visual fields of both eyes are shielded together to the total time for displaying the plurality of images And the observer may feel dark. Therefore, for example, in order to obtain a stereoscopic effect of about 86% or more at the best time, each vertex of the equilateral triangle ABC is set as a viewpoint as shown in FIG. 4 rather than the case shown in FIGS. It is preferable to use three images.

Based on the above principle, a system embodying the technology according to the present invention will be described.
FIG. 31 shows an image system 100 according to an embodiment of the present invention. The image system 100 includes an image acquisition device 110, a composition device 120, an image display device 130, and a control device 150. In the figure, A, B, and C indicate the viewpoint of the image.

  In the image system 100 of the present embodiment, as an example, the images A, B, and C are images each having the equilateral triangle ABC shown in FIG. 4 as viewpoints, and the viewpoints A, B, and C have three viewpoint pairs. Form.

  The image system 100 sequentially outputs an image A and an image B obtained by simultaneously capturing images to the synthesizer 120. The synthesizing device 120 synthesizes the image C from the image A and the image B to obtain the image C, and sequentially outputs the images A, B, and C to the image display device 130.

  The image display device 130 sequentially displays the images A, B, and C, and outputs a synchronization signal S1 that can indicate which of the images A, B, and C is currently displayed to the control device 150.

  The eyeglasses 140 are worn by an observer of the image displayed on the image display device 130, and output tilt information S2 described later to the control device 150.

  The control device 150 controls the glasses 140 by generating a control signal S3 based on the synchronization signal S1 and the tilt information S2 and outputting the control signal S3 to the glasses 140.

  Note that transmission of signals between the image display device 130 and the control device 150 and between the control device 150 and the glasses 140 is performed via wireless communication means (not shown) such as infrared rays.

  FIG. 32 is a diagram illustrating the image acquisition device 110. As illustrated, the image acquisition device 110 includes a first imaging unit 112, a second imaging unit 114, and an output unit 116. The first imaging unit 112 captures the subject from the viewpoint A, obtains an image A, and outputs the image A to the output unit 116. The second imaging unit 114 captures the same subject from the viewpoint B, obtains an image B, and outputs the image B to the output unit 116. The distance between the viewpoints A and B is the distance L0 between both eyes.

  The output unit 116 outputs the image A from the first image capturing unit 112 and the image B from the second image capturing unit 114 to the synthesizing device 120 successively in the order of A and B.

  FIG. 33 is a diagram illustrating the synthesis device 120. The synthesizer 120 includes a vector acquisition unit 122, a vector rotation unit 124, an image generation unit 126, and an output unit 128.

  The vector acquisition unit 122 calculates a disparity vector for the images A and B and outputs the parallax vectors to the vector rotation unit 124 and the image generation unit 126. Here, the parallax vector will be described.

  Image A and image B are images obtained by capturing the subject from the viewpoint A and the viewpoint B at the same time. Such two images are obtained by imaging from the same viewpoint when the subject moves in the direction from point A or point B by the distance between point A and point B (interval between both eyes L0). Can be regarded as two frames of a moving image. Image A corresponds to the frame before the subject moves, and image B corresponds to the frame after the subject moves.

  Therefore, assuming that the image A and the image B are two frames of a moving image, a motion vector for each block can be calculated between the two frames. The disparity vector is a component of the motion vector in the direction from point A to point B.

  The vector rotation unit 124 rotates each parallax vector so that the direction of each parallax vector calculated by the vector acquisition unit 122 is a direction from point A to point C. Here, since the relationship between the viewpoint and the display image is opposite to the center of gravity of the viewpoint, the vector rotation unit 124 rotates each parallax vector calculated by the vector acquisition unit 122 downward by 60 degrees. The vector rotation unit 124 outputs the parallax vector after rotation to the image generation unit 126.

  The image generation unit 126 generates an image C using the image A and the rotated parallax vector from the vector rotation unit 124. Images A and C are also moving images obtained from the same viewpoint when the subject moves in the direction toward point A or point C by the distance between point A and point C (interval L0 between both eyes). The image A corresponds to a frame before the subject moves, and the image C corresponds to a frame after the subject moves. Therefore, assuming that the image A and the image C are two frames of a moving image, and the disparity vector after rotation from the vector rotation unit 124 is a motion vector, the image C can be generated.

FIG. 34 shows an example of the flow of the image C composition processing by the composition device 120.
<Step 1>
First, the vector acquisition unit 122 calculates a difference in the x direction for each block of the image A and the image B. The block area where there is no difference between image A and image B is the background area. Then, the vector acquisition unit 122 removes the background region from the image A (the foreground of the image A, ie, the subject image A1), and removes the background region from the image B (the foreground of the image B, ie, the image B). The parallax vector of each block is calculated from the image B1) of the subject portion by a technique such as block matching.

<Step 2>
The vector rotation unit 124 rotates each vector calculated by the vector acquisition unit 122 downward 60 degrees.

<Step 3>
The image generation unit 126 moves each block of the foreground image A1 of the image A according to the direction and distance indicated by the parallax vector rotated by the vector rotation unit 124 corresponding to the block. The foreground image A1 after the movement becomes the foreground image C1 of the image C.

<Step 4>
Then, the image generation unit 126 combines the background region image obtained in step 1 with the foreground image C1 obtained in step 3 to obtain an image C. At the time of synthesis, the insufficient portion of the background region image obtained in step 1 may be generated by interpolation or the like with reference to the matching portion between image A and image B.

<Step 5>
The output unit 128 outputs the images A and B from the image acquisition device 110 and the image C obtained by the image generation unit 126 in the order of A, B, and C.

  As described above, the image display device 130 sequentially displays the images A, B, and C from the synthesizing device 120 and outputs the synchronization signal S1 indicating which image is displayed.

  FIG. 35 is a diagram showing the eyeglasses 140. The spectacles 140 includes a left eye 142, a right eye 144, a sensor 146, and a shutter driving unit 148.

  The left eye part 142 and the right eye part 144 have a shutter (not shown) driven by a shutter driving part 148, and can block the left eye field and the right eye field of the observer wearing the glasses 140, respectively.

  The sensor 146 is provided on the frame of the eyeglasses 140, detects the inclination of the eyeglasses 140, that is, the inclination of a straight line (parallax line SL) connecting the left eye part 142 and the right eye part 144, and obtains inclination information S2 indicating the inclination. Output. Note that the inclination detected by the sensor 146 corresponds to the angle θ described above. The sensor 146 is an acceleration sensor, for example.

  The shutter drive unit 148 drives the shutters of the left eye unit 142 and the right eye unit 144 so as to shield one or both of the left eye field and the right eye field in accordance with the control signal S3 from the control device 150.

  FIG. 36 is a diagram illustrating the control device 150. The control device 150 includes a reception unit 152 and a control execution unit 154.

  The receiving unit 152 receives the synchronization signal S1 from the image display device 130 and the tilt information S2 from the glasses 140.

  The control execution unit 154 selects two images in the images A, B, and C as target image pairs based on the synchronization signal S1 and the inclination information S2, and any image of the target image pair is displayed. Sometimes, the shutter driving of the glasses 140 is performed so that one of the left eye and the right eye that does not correspond to the image is blocked, and when an image other than the target image pair is displayed, both the left eye and the right eye are blocked. The control signal S3 is output to the unit 148.

  The method of selecting a target image pair by the control execution unit 154 is as described in the example shown in FIG.

  FIG. 37 shows a correspondence relationship between the tilt of the observer's head (or the position of the observer), the selected target image pair, and the occlusion state of the left-eye and right-eye views in the image system 100.

  The upper part of FIG. 37 shows the inclination of the observer's head viewed from the front of the observer. This inclination corresponds to the angle θ described above.

  For example, when the observer heads from the assumed observation position to the screen and the head tilt is within 30 degrees left and right (θ ≦ 30, θ ≧ 330), the image (A, B) is selected as the target image pair. Is done. Then, when the image A is displayed, the visual field of the right eye is shielded (OFF), and when the image B is displayed, the visual field of the left eye is shielded. In addition, when the image C is displayed, the visual fields of both eyes are shielded.

  In addition, for example, when the observer faces the screen from a position opposite to the position normally assumed and the head tilt is within 30 degrees (150 ≦ θ ≦ 210), the image (B, A) is selected. When the image A is displayed, the left eye field of view is blocked, and when the image B is displayed, the right eye field of view is blocked. In addition, when the image C is displayed, the visual fields of both eyes are shielded.

  As described above, the best stereoscopic effect of 86% or more can always be obtained regardless of the inclination of the head of the observer and the position of the observer. In addition, since the tilt information S2 and the control signal S3 are in a one-to-one relationship, even when a plurality of observers go from different positions to the screen, an optimal stereoscopic effect can be given to each observer. .

  The present invention has been described above based on the embodiment. The embodiment is an exemplification, and various modifications and changes may be made to the above-described embodiment without departing from the gist of the present invention. It will be understood by those skilled in the art that modifications in which these changes and increases / decreases are also within the scope of the present invention.

  For example, in the image system 100, the image acquisition device 110 acquires the images of the viewpoint A and the viewpoint B using the first imaging unit 112 and the second imaging unit 114, and synthesizes the image of the viewpoint C using the image generation unit 126. However, the image acquisition device 110 may be provided with three imaging units, and the images of the viewpoints A, B, and C may be acquired by these three imaging units.

  Alternatively, the synthesis device 120 may be included in the image acquisition device 110 or the image display device 130.

  When the combining device 120 is included in the image display device 130, the image display device 130 preferably combines the image C while displaying the images A and B. By doing so, it is not necessary to delay the display of the images A and B for the synthesis of the image C, so that the processing of the entire image system 100 is accelerated.

  In the image system 100, the control device 150 controls only one spectacle 140, but if the inclination information S2 from the spectacles 140 and the control signal S3 from the control device 150 have a one-to-one relationship, One control device 150 may receive each inclination information S2 from the plurality of glasses 140 and output a control signal S3 to each of the glasses 140.

  Further, the control device 150 may be included in the glasses 140.

  Further, the image system 100 is an example using three images having three vertexes of an equilateral triangle as viewpoints as images of a plurality of viewpoints, but the principle of the present invention is described as the images of the plurality of viewpoints. You may use the image of the various viewpoints described when doing.

  The method of synthesizing the image C from the image A and the image B is not only the synthesis of the image having the third vertex from the two images each having the two vertices of the equilateral triangle as viewpoints, but also the principle of the present invention. The present invention can be applied to the synthesis of images from other viewpoints from the image A and the image B with respect to the various examples described in the description.

  This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2011-087254 for which it applied on April 11, 2011, and takes in those the indications of all here.

DESCRIPTION OF SYMBOLS 1 Subject 2 Display screen 100 Image system 110 Image acquisition device 112 1st imaging part 114 2nd imaging part 116 Output part 120 Synthesis | combination apparatus 122 Vector acquisition part 124 Vector rotation part 126 Image generation part 128 Output part 130 Image display apparatus 140 Eyeglasses 142 Left eye part 144 Right eye part 146 Sensor 148 Shutter driving part 150 Control device 152 Reception part 154 Control execution part S1 Synchronization signal S2 Inclination information S3 Control signal α Parallax angle SL Parallax line

Claims (23)

Each of a plurality of three or more viewpoints at different positions on the same plane forms a viewpoint pair whose distance is the distance between the eyes with at least one other viewpoint, and a subject corresponding to each of the viewpoints Display each of the images in sequence,
Detecting the inclination of the parallax line that is a straight line connecting the left eye part and the right eye part of the spectacles that is worn by the observer of the displayed image and can shield the visual field of the left eye and the right eye of the observer;
According to the detected inclination, a target image pair composed of two of the images of the subject is selected, and when any one of the target image pairs is displayed, the left eye and the right eye The left eye part and the right eye part of the glasses are controlled so as to shield both the left eye field and the right eye field when an image other than the target image pair is displayed. And
The target image pair is
Assuming that the parallax line is moved to the plane, among the straight lines in the straight line group composed of straight lines connecting the viewpoints forming the viewpoint pairs, the orthogonal projection to the parallax line is An image display method characterized in that the images are two images respectively having the longest end point of one straight line as viewpoints. The plurality of viewpoints are:
The image display method according to claim 1, comprising a plurality of equally divided points that divide the circumference of the circle on the plane into n equal parts (n: an integer of 3 or more).   The viewpoint pair is two equal points where the central angle (360 / n) × int (n / 2) is the angle formed by any two of the center of the circle and the equal points. The image display method according to claim 2, wherein the image display method is formed.   The image display method according to claim 3, wherein the n is an odd number.   The image display method according to claim 4, wherein n is three. The circle has a radius between the eyes,
The plurality of viewpoints further include a center of the circle,
The image display method according to claim 2, wherein the viewpoint pair is any one of the circle center and the plurality of equally divided points. N is an even number of 4 or more;
The viewpoint pair is any one of the circle center and “n / 2” equally divided points that are continuous along the circumference of the plurality of equally divided points. The image display method according to claim 6. Each of a plurality of three or more viewpoints at different positions on the same plane forms a viewpoint pair whose distance is the distance between the eyes with at least one other viewpoint, and a subject corresponding to each of the viewpoints The left eye part and the right eye part, the left eye part and the right eye which are attached to the observer of the images displayed on the image display device which sequentially displays each of the images, and which can shield the visual field of the left eye and the right eye of the observer, respectively. In a control device corresponding to eyeglasses provided with a sensor that detects the inclination of a parallax line that is a straight line connecting parts,
A receiving unit that receives tilt information indicating the tilt detected by the sensor;
A target image pair consisting of two of the plurality of images is selected according to the tilt indicated by the tilt information, and when any one of the target image pairs is displayed, the left eye and the right eye Of the eyeglasses, and when the image other than the target image pair is displayed, the left eye part and the right eye part of the glasses are shielded so as to shield both the left eye field and the right eye field. A control execution unit for controlling,
The target image pair is
Assuming that the parallax line is moved to the plane, among the straight lines in the straight line group composed of straight lines connecting the viewpoints forming the viewpoint pairs, the orthogonal projection to the parallax line is A control device characterized in that the images are two images each having the longest end point of one straight line as viewpoints. The plurality of viewpoints are:
The control device according to claim 8, comprising a plurality of equally divided points that divide the circumference of the circle on the plane into n equal parts (n: an integer of 3 or more).   The viewpoint pair is two equal points where the central angle (360 / n) × int (n / 2) is the angle formed by any two of the center of the circle and the equal points. The control device according to claim 9, wherein the control device is formed.   The control device according to claim 10, wherein n is an odd number.   The control device according to claim 11, wherein n is three. The circle has a radius between the eyes,
The plurality of viewpoints further include a center of the circle,
The control device according to claim 9, wherein the viewpoint pair is any one of the circle center and the plurality of equally divided points. N is an even number of 4 or more;
The viewpoint pair is any one of the circle center and “n / 2” equally divided points that are continuous along the circumference of the plurality of equally divided points. The control device according to claim 13. A pair of spectacles attached to an observer of an image displayed on an image display device that sequentially displays each of the images of the subject according to any one of claims 8 to 14,
A left eye part capable of shielding the visual field of the left eye of the observer;
A right eye part capable of shielding a field of view of the right eye of the observer;
A sensor that detects the inclination of a parallax line that is a straight line connecting the left eye part and the right eye part;
An eyeglass comprising the control device according to one of claims 8 to 14. An image display device;
Glasses worn by an observer of an image displayed on the image display device;
A control device for controlling the glasses,
The image display device sequentially displays each of the images of the subject described in any one of claims 8 to 14,
The glasses are
A left eye part capable of shielding the visual field of the left eye of the observer;
A right eye part capable of shielding a field of view of the right eye of the observer;
A sensor that detects the inclination of a parallax line that is a straight line connecting the left eye part and the right eye part,
15. The image system according to claim 8, wherein the control device is the one described in one of claims 8 to 14. The image display device includes:
Two images of each of the subject images are provided,
The image system according to claim 16, further comprising: a combining unit configured to combine another image of each of the images of the subject from the two images. The synthesis unit is
When the first image and the second image of the two images are assumed to be two frames of a moving image, the motion vector is directed from the viewpoint of the first image to the viewpoint of the second image. A vector acquisition unit for obtaining a disparity vector that is a component in a direction;
A vector rotation unit that rotates the parallax vector so that the direction of the parallax vector obtained by the vector acquisition unit is a direction from the viewpoint of the first image toward the viewpoint of the other image;
Assuming that the first image is one frame of a moving image, a frame having a motion vector similar to the parallax vector rotated by the vector rotation unit between the first image and the first image is the other image. The image system according to claim 17, further comprising: an image generation unit that generates the image as a unit.   The image system according to claim 16, further comprising an image acquisition device that acquires the plurality of images to be provided to the image display device. The image acquisition device includes:
The image system according to claim 19, further comprising a plurality of imaging units respectively provided at the plurality of viewpoints corresponding to the images of the subject. The image acquisition device includes:
Two imaging units respectively provided at two viewpoints of the plurality of viewpoints corresponding to the images of the subject;
The image system according to claim 19, further comprising: a combining unit that combines other images of the images of the subject from the two images obtained by the two imaging units. The synthesis unit is
When the first image and the second image of the two images are assumed to be two frames of a moving image, the motion vector is directed from the viewpoint of the first image to the viewpoint of the second image. A vector acquisition unit for obtaining a disparity vector that is a component in a direction;
A vector rotation unit that rotates the parallax vector so that the direction of the parallax vector obtained by the vector acquisition unit is a direction from the viewpoint of the first image toward the viewpoint of the other image;
Assuming that the first image is one frame of a moving image, a frame having a motion vector similar to the parallax vector rotated by the vector rotation unit between the first image and the first image is the other image. The image system according to claim 21, further comprising: an image generation unit that generates the image as a unit. Three points included on a plane facing the subject, the first and first of the three points having the same distance from the first point to the second and third points. A synthesizing device for synthesizing a third image having the third point as a viewpoint from a first image and a second image each having a point as a second point;
A vector for obtaining a disparity vector that is a component in a direction from the first point toward the second point of a motion vector when the first image and the second image are assumed to be two frames of a moving image. An acquisition unit;
A vector rotation unit that rotates the parallax vector so that the direction of the parallax vector obtained by the vector acquisition unit is a direction from the first point toward the third point;
Assuming that the first image is one frame of a moving image, a frame having a motion vector similar to the disparity vector rotated by the vector rotation unit is provided between the first image and the first image. An image generation unit that generates an image.

Images