JP2012237970A - Zoomable stereoscopic photograph viewer - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a zoomable stereoscopic photograph viewer, when browsing a stereoscopic photograph, preventing a fusion from being diverted into a double image by zoom-in, and allowing a user to browse a natural fusion.SOLUTION: The zoomable stereoscopic photograph viewer includes: a stereoscopic display device for presenting a browsing person with a left-eye image and a right-eye image; and a display control device for supplying an image, and provides the browsing person with the left-eye and right-eye images. An interval D between the left and right images displayed on the stereoscopic display device is changed according to a zoom-in (zoom-out) magnification ratio m so that the position of the fusion stands still or moves forward (in the back). Especially, when α and β are defined as constant numbers, and p is defined as 0≤p≤1, then D=α×(1/m-β). The zooming is performed by changing a distance t between end-points of the left/right stereoscopic photographs as a function of the magnification ratio m. When n is defined as 1≤n≤2 and a, b, and k are defined as constant numbers, then the distance t is a function of the magnification ratio m: a×(1/m-k)+b, and the constant numbers a, b, and k are determined at the limit point of the zoom-in and zoom-out.

Description

  The present invention relates to a zoomable stereoscopic photograph viewer (SPV) that allows a viewer to view a natural fusion of a stereoscopic image even when part of the stereoscopic photograph is enlarged.

  The SPV described below is roughly a stereoscopic picture browsing apparatus that can introduce stereoscopic picture browsing software into a personal computer, for example, so that a stereoscopic picture can be browsed while smoothly zooming.

  When zooming with a conventional stereoscopic image browsing device, in the process of enlarging a small area on the stereoscopic display device, the stereoscopic image generated so far is split into two images, and the viewer can view the stereoscopic image. The state that cannot be detected arises. This state arises from the fact that the viewer's stereoscopic ability exceeds that limit and the stereoscopic image cannot be fused. In the present specification, these two images will be referred to as “double images by fusion decay” (or simply referred to as double images). The double image at this time is simply a flat two image shifted and overlapped. (In the field of stereoscopic display devices, there is also a phenomenon similar to a double image called “ghost due to crosstalk” mainly caused by hardware characteristics, but this is not dealt with in this specification).

  The present invention is an SPV in which the depth position of a fusion image does not change even when zooming is applied by applying an image processing technique of two-dimensional computer graphics (2DCG), and a double image is not generated. This is based on the “fusion formula” based on the analysis of the mechanism for generating a stereoscopic image by the inventor of the present invention. This fusion formula governs zooming in SPV and will be described in Example 1.

  The stereoscopic photograph that is the subject of the present invention is one in which two cameras are attached to a support base called a pan head and the distance between the lenses is determined empirically and left and right images (referred to as a pair of stereoscopic photographs) are taken, or It was taken using the same technique. In this method, since the distance between the two lenses can be freely changed on the camera platform, it is possible to take a three-dimensional photograph from a near view to a distant view. If there is a deviation in the vertical or inclination between the pair of stereoscopic photographs, it is assumed that the stereoscopic photographs have been removed in advance using the correction function provided in the SPV. This is an important requirement that allows a stereoscopic image to be visually recognized from a pair of stereoscopic photographs.

  Such a pair of stereoscopic photographs has been enjoyed by enthusiasts for a long time in a manner of viewing using various stereoscopic photograph browsing devices. This stereoscopic photograph browsing device has been made using lenses, mirrors, and prisms since the middle of the 19th century.

  In the case of using a stereoscopic display device, a stereoscopic photograph as described above is usually not an effective pair of stereoscopic photographs simply by arranging the captured images on the left and right. There is a mistake in attaching the camera to the camera platform that the photographer is not aware of, and due to the tilt of the camera platform during shooting, the overlay between the left and right images is slightly negligible Misalignment often occurs.

  Therefore, it is necessary to trim the two images of the three-dimensional photograph above and below, to the left and right, or to rotate the image to correct the inclination so as to make an appropriate pair of three-dimensional photographs. In recent years, this work has been edited using, for example, an image / video editing software called Premier (registered trademark) of Adobe Corporation. For this reason, there is a need for an SPV that can perform such editing.

  As is well known, the sense of depth of a stereoscopic photograph taken by two cameras, that is, the stereoscopic effect, is determined by the distance between the two cameras on the camera platform, and is determined at the time of shooting. In addition to the above three-dimensional effect, the three-dimensional effect of each three-dimensional photo also varies depending on how the three-dimensional photo is taken and the shooting scenery at that time. Therefore, it can be said that each stereoscopic photograph has a “unique” stereoscopic feeling fixed as a photograph. Therefore, this stereoscopic effect is referred to as “inherent stereoscopic effect” of the stereoscopic photograph. In principle, the depth of each stereoscopic photograph due to this “inherent stereoscopic effect” cannot be changed.

  However, the stereoscopic effect due to the stereoscopic image browsing environment in which the viewer is placed (for example, displayed on a stationary stereoscopic display device with personal computer stereoscopic display software) is a stereoscopic effect slightly different from the “inherent stereoscopic effect”. That is, even if the stereoscopic photographs are the same stereoscopic photograph, if the stereoscopic photograph browsing environment is different, the stereoscopic image obtained from the stereoscopic photographs can be seen in front of, behind, or near the screen of the stereoscopic display device. The stereoscopic effect that the viewer feels in this way is not the “inherent stereoscopic effect” of the stereoscopic photograph, but a stereoscopic effect in which some action is added to the “inherent stereoscopic effect”.

Therefore, the stereoscopic effect felt by the viewer is referred to as “perspective stereoscopic effect”, and some action is referred to as “perspective factor”. At this time, it is understood that the perspective stereoscopic effect is obtained by modifying the intrinsic stereoscopic effect with the perspective factor. This is schematically represented as follows.
"Perspective stereoscopic effect" = "Inherent stereoscopic effect" x "Perspective factor"
This model indicates that the stereoscopic effect of the stereoscopic photograph felt by the viewer is a stereoscopic effect as a result of the action of some perspective on the stereoscopic effect unique to the stereoscopic photograph. Therefore, the “perspective stereoscopic effect” felt by the viewer will be described below by dividing it into “inherent stereoscopic effect” and “perspective factor” of a stereoscopic photograph.

  Thus, by dividing the “perspective stereoscopic effect” into two parts, “intrinsic stereoscopic effect” and “perspective factor”, and considering the stereoscopic effect of the stereoscopic photograph, the prospect of designing the SPV is improved. For example, the above-described contrivance for suppressing the generation of a double image due to fusion breakdown can be realized from the analysis of “intrinsic stereoscopic effect” and “perspective factor”.

First, the above-described inherent stereoscopic effect will be described in more detail.
For example, it is assumed that three cars are shown in the stereoscopic photograph S, and they are arranged in the order of car A, car B, and car C from the front to the back. However, as described above, if there is a deviation in the vertical or inclination, it is assumed that it has been removed. As described above, one of the factors that generate the “unique stereoscopic effect” in the stereoscopic photograph S is the distance between the two lenses of the two cameras that photographed the scenery. As the interval increases, the depth of the stereoscopic image viewed from the stereoscopic photograph is felt to increase. However, when the lens interval becomes larger than a certain value, it becomes difficult for the viewer to fuse the stereoscopic image, and when the limit is exceeded, the stereoscopic image collapses into a double image. Also, when the lens interval is small, the sense of depth is reduced. When the interval is zero, the result is the same as when two identical images are viewed, and a three-dimensional effect is not felt.

  In general, such shooting conditions for stereoscopic photography are already well known, and a pair of stereoscopic photography needs to be taken with an appropriate lens interval between the two cameras. In principle, the two cameras should be the same type and have the same performance, but if not, correct appropriate corrections in advance for the stereoscopic images taken. It is necessary to keep. In addition, when the display device is installed horizontally, it is installed on the camera platform so that the directions of the optical axes of the lenses of the respective cameras are in the same plane and parallel, and the camera platform is held horizontally. And This is taken as shooting condition A.

  As a method of generating a unique stereoscopic effect in a stereoscopic photograph, there is an effect due to convergence in addition to the effect of setting the interval between lenses within a predetermined range. This effect is an effect obtained by installing the two lenses so that the directions of the two lenses intersect with each other (imaging condition B). This is a special case of the above-described shooting condition A when the shooting target is sufficiently far away, and can be approximated as a solid angle that is focused on a specific shooting target. In SPV, the photographing condition A is adopted, but when the photographing condition B is applied, the photographed stereoscopic photograph is a correction target.

Next, the perspective stereoscopic effect will be described.
As the SPV of the present invention, for example, it is assumed that a display image is prepared on a personal computer and displayed on a stationary stereoscopic display device. The stereoscopic effect when the viewer views the stereoscopic photograph through the SPV, that is, the “perspective stereoscopic effect” will be described using the example of the stereoscopic photograph S described above. This stereoscopic display device uses, for example, stereoscopic glasses and a liquid crystal display by a polarized glasses method or a liquid crystal shutter method. When the stereoscopic photograph S is displayed on the stereoscopic display device, the viewer can feel a perspective stereoscopic effect as described above. Depending on the display conditions, this perspective stereoscopic effect may be (a) all three cars appear to pop out from the display screen of the three-dimensional display device. (B) all three cars are behind the screen. Or (c) one or two of the three cars appear to be near the screen. However, even if the display conditions are changed as described above, it can be seen that the relative depth of each of the three cars is unchanged. This invariant stereoscopic effect is a unique stereoscopic effect, and the perspective stereoscopic effect is modified by a perspective factor that determines the position at which the three cars are represented from the front of the screen to the back. It is what.

  In viewing the stereoscopic photo S described above, even if the viewer first feels the stereoscopic effect of the scene (a), if the SPV has a “perspective factor” function, the viewer operates it. You can feel the three-dimensional effect of the scene (b) or the three-dimensional effect of the scene (c). A specific image thereof is shown in FIG. In the three drawings, the black frame means the screen, and the cube indicated by the dotted line means the three-dimensional space. FIG. 10 (a) shows the three-dimensional space in front of the screen, FIG. 10 (b) shows it behind the screen, and FIG. 10 (c) shows it in the vicinity of the screen. felt.

  Such a stereoscopic sensation is a sensation that clearly occurs when a desktop type or a larger stereoscopic display device is used. Although the degree of stereoscopic effect varies depending on the situation, such as when different methods such as stereoscopic glasses are used, or when the display size is different, the perspective stereoscopic effect felt by the viewer is modified by the perspective factor described above. Being a thing does not change. In other words, it is important for the viewer what kind of “perspective” positional relationship the entire stereoscopic space generated by the stereoscopic photograph is arranged with respect to the screen position of the stereoscopic display device. Depending on how the three-dimensional space is arranged, that is, the degree of the perspective factor, in the above-described vehicle arrangement, the three-dimensional effects appear in the above case (A), case (B), and case (C).

  In addition, although the visibility of a stereoscopic image is subjective, the stereoscopic image is somewhat more of a 3D display device than that projected in front of the 3D display device screen (closer to the 3D display device screen). In general, it is said that a viewer that is deep behind the screen (slightly far from the screen of the stereoscopic display device) is easier for the viewer to visually recognize.

  By the way, if you look at a stereoscopic photograph using a classic peeping-type stereoscopic photograph browsing device using a lens or a mirror or a stereoscopic HMD (Head Mount Display), Although there is a difference, the “perspective stereoscopic effect” is weak, and at first glance, the “inherent stereoscopic effect” created by the stereoscopic photograph can be felt stronger. In order to strongly sense “perspective stereoscopic effect”, it is desirable that the angle of view is larger.

  Japanese Patent Laid-Open No. 10-513019 discloses a method of viewing a stereoscopic image by shifting the positions of the left and right images to change the degree of mismatch and moving the perceived relative position of the object. It is disclosed. However, this Patent Document 1 does not disclose how to shift.

Japanese National Patent Publication No. 10-513019

Takashi Kawai, Miwa Tanaka, "Introduction to Next Generation Media Creator 1-Solid Expression", Cut Systems (2003) Shigeru Nakayama, "Java (registered trademark) 2-Introduction to graphics programming-", Gihodo Publishing (1999)

  When zooming in with a conventional stereoscopic photograph browsing device, the fused stereoscopic image moves away from the viewer in the process of enlarging a fine area of the image of the stereoscopic display device, and eventually becomes in a state where it cannot be fused. Further, when zooming out, it is felt that the stereoscopic image comes to the front.

  Therefore, in the present invention, even if zooming is performed, the fusion is not solved, and the viewer can perceive the natural fusion of the stereoscopic image with respect to the depth position of the fused stereoscopic image. A 3D photo viewer (SPV).

The zoomable SPV of the present invention includes a stereoscopic display device that presents a left eye image and a right eye image to the viewer's left eye and right eye, respectively, and the left eye image on the stereoscopic display device. And a display control device for supplying a right-eye image, and an SPV for presenting a left-eye image and a right-eye image for stereoscopic image display to a viewer,
For the left eye in zooming in, the distance D between the same objects of the left image and the right image in the image displayed at the back of the display surface of the stereoscopic display device is monotonously decreased as the magnification increases due to zooming in. Zooming so that the position of the stereoscopic image by the fusion of the image and the image for the right eye moves to the front as viewed from the viewer,
Alternatively, the distance D between the same objects of the left image and the right image in the image displayed in the back of the display surface of the stereoscopic display device can be zoomed out by monotonically increasing the distance D as the magnification is reduced by zooming out. Zooming is performed so that the position of the three-dimensional image resulting from the fusion of the image for the left eye and the image for the right eye moves to the back as viewed from the viewer.

  The distance D between the same objects of the left image and the right image in the image displayed on the stereoscopic display device is such that the magnification is m, α and β are constants, and 0 ≦ p ≦ 1,

By zooming in such a way that the position of the three-dimensional image resulting from the fusion at the zoom-in moves to the front as viewed from the viewer, or the three-dimensional image resulting from the fusion at the zoom-out. Zooming is performed so that the position moves to the back as viewed from the viewer.

The left-eye image and right-eye image displayed on the stereoscopic display device are generated from a pair of stereoscopic photo images,
In the generation, the left-eye image and the right-eye image are generated by enlarging or reducing the pair of stereoscopic photograph images at a magnification m,
The zooming is based on a state in which the end points of the left-eye image and the right-eye image are aligned, and the distance t between the end-point of the left-eye image and the end-point of the right-eye image is used as a function of the magnification m. Done by changing,
The position of the three-dimensional image by the fusion in the zoom-in moves toward the viewer as viewed from the viewer, or
Zooming is performed so that the position of the three-dimensional image resulting from the fusion in the zoom-out moves to the back as viewed from the viewer.

In particular, given that the distance is t, the function consists of the sum of a term proportional to (1 / m-constant) ^{n + 1} and a constant term for n where 0 ≦ n ≦ 1. That is, when a and b are constants and the lower limit and upper limit of the magnification m are m ₁ and m ₀ , respectively, the right side of the following equation.

The constant term b is used to zoom in on the relatively distant view area and the relatively close view area selected in the zoom-in target area of the stereoscopic photograph image to the enlargement limit magnification m _0, and The viewer determines the stereoscopic image by the image on the condition that the stereoscopic image is visually recognized at a position on the display screen of the stereoscopic display device.

Further, the proportional coefficient of (1 / m-constant) ^{n + 1} , that is, the constant a in a / (1 / m ₁ -1 / m ₀ ) ^{n + 1 is} in the zoom-in target area of the stereoscopic image. The stereoscopic image is zoomed out to the reduction limit magnification m ₁ for the selected relatively distant view region and relatively close view region, and a 3D image by fusion of the relatively close view region is displayed on the display screen of the 3D display device. It is determined by the viewer on the condition that it is visually recognized at an upper position or a position behind it.

The constant term b is used to zoom in on the relatively distant view area and the relatively close view area selected in the zoom-in target area of the stereoscopic photograph image to the enlargement limit magnification m _0, and When a viewer is determined by a viewer on the condition that a stereoscopic image by an image is viewed at a position on the display screen of the stereoscopic display device,
The constant at (1 / m-constant) ^{n + 1} is the reciprocal of the enlargement limit magnification m ₀ .

  As the stereoscopic display device, any one of an anaglyph method, a liquid crystal shutter method, a polarizing filter method, a stereoscopic glasses method, or a parallax barrier method or a lenticular lens method can be used.

  The left image and the right image in the image displayed on the stereoscopic display device are obtained by removing in advance a vertical shift and a tilt shift. For this reason, if the above-mentioned misalignment exists between the left and right images, the SPV needs to have a function of removing (or correcting) it, but if the misalignment is slight, it can be tolerated without removal. There is also.

  When the zoom-in and zoom-out operations are performed, the fused image is not solved and becomes a double image, and the viewer can perceive a natural three-dimensional image.

The stereoscopic photograph image in the present invention is a diagram showing that a three-dimensional object to be photographed is captured by left and right cameras, respectively. It is a figure which shows that the image fixed to the imaging surface is projected as a left-right stereoscopic photograph image by the left and right projection lenses, and becomes a stereoscopic photograph. It is a figure which shows the fusion position in the case of aligning and displaying the end surface of the image for left eyes on the said three-dimensional display apparatus, and the end surface of the image for right eyes. In this case, the black dot of the left eye image on the stereoscopic display device is located to the right of the white dot of the right eye image for the same subject. A three-dimensional image that is a fusion of these black and white dots is fused at a position close to the viewer side from the display screen of the image, and the viewer perceives an image popping out from the screen. It is a figure which shows the change of the position of a fusion | melting when the space | interval D of the image for left eyes and the image for right eyes on a stereoscopic display apparatus is shortened, and when the distance between a viewer and a stereoscopic display apparatus is changed. . First, the fusion position can be brought closer to the viewer by reducing the interval D, that is, by setting D ₁ > D ₂ . Also, the position of the fusion can be brought closer to the viewer by reducing the distance L, that is, by setting L ₁ > L ₂ . The distance between the viewer's left eye and right eye is e, the distance from the left eye image on the stereoscopic display device to the right eye image is D, and the depth interval from the display screen of the stereoscopic display device to the fusion is D. u, where h is the distance between the viewer and the stereoscopic display device, there is a relationship u = h × D / (e−D) using the ratio of the height of the triangle to the length of the base. (Non-Patent Document 1). (A) is when D is positive, and (b) is when D is negative. (A) is a figure which shows the relationship between u and D in the case of e = 6.5cm and h = 100cm, u = h * D / (e-D). (B) is a figure which shows adjusting the position of a fusion | melting by operating the space | interval of the image for left eyes (black circle) and the image for right eyes (white circle) of the attention area on a display screen. For example, when the stereoscopic photograph D is distributed in the section P [x ₀ , x ₁ ], the distribution can be shifted to the section P ′ by performing the movement r ₁ . A three-dimensional image by fusion is visually recognized in the back. It is a figure which shows the conventional zooming method. Conventionally, a right-eye image and a left-eye image are displayed together on a display screen in a proportionally reduced and enlarged manner around an arbitrary point. A three-dimensional image by fusion is felt as being farther away when zooming in, and is approaching when zooming out. When enlarging an image centering on each round point while fixing the position of each round point of interest in the image for the left eye and the image for the right eye on the stereoscopic display device, a fusion of two round points It is a figure which shows that it can expand, without changing the depth position about the three-dimensional image by. The depth position is obtained by enlarging / reducing the left eye image and the right eye image around the position (point) while fixing the position (point) of the region of interest without changing the values of h, e, and D. You can zoom in and out without changing u. (A), (b) is the figure which showed typically the expansion by the projection lens in FIG. (A) is a photograph taken with an equal magnification zoom projection lens. The black circle point is assumed to be a distance from the center line of the enlargement whose position does not change by zooming. When this is zoomed m times as shown in (b), the distance between the zoom projection lens and the display screen does not change, and the distance from the enlargement center line is m × a. In order to facilitate such zooming with respect to the left-eye image and the right-eye image, as shown in FIG. 9C, the left and right photographs are based on a reference in proportion to the distance from each projection lens. And the magnification is m times. In the figure, P _L and P _R are parallel lines extending from the center points C _L and C _R of the enlargement. This usually corresponds to the center line of the left (or right) eye field of view. In the left-eye image and the right-eye image, the positions of the noticed round points displayed on the display screen 1 are measured from the respective lines P _L and P _R. In addition, it is assumed that a round dot is m times and displayed on the display screen 2. (D) is a figure in the case of aligning and displaying the lines P _L and P _R of (c). Thus, when a vector extending from the left-eye image (black circle point) to the right-eye image (white circle point) in the attention area on the display screen 2 is {Dd}, {Dd} = m ({b} − {a }). When the viewer views this image, the black dot is on the right, the white dot is on the left, and the distance between the black dot and the white dot increases according to the zooming magnification m. It can be seen that a three-dimensional image by the fusion is visually recognized near the viewer. It is a figure which shows the relative position with respect to the screen of a three-dimensional space, (a) is a three-dimensional space before this screen, (b) is the back of the screen, (c) is arranged near the screen, respectively. It is a schematic diagram which shows the case where a viewer feels like. Here, in the figure, a black frame means a screen, and a cube indicated by a dotted line means a three-dimensional space. It is a conceptual diagram which shows that a pair of three-dimensional photography (the image for left eyes and the image for right eyes) is drawn on a three-dimensional display apparatus screen, It is the bird's-eye view which looked at SPV from the upper part toward the negative direction of the y-axis. . The left-eye image and the right-eye image are pasted on the two flat plates, respectively, and the flat plate of the right-eye image is movable left and right. Two points corresponding to each of the left image and the right image (the distance between the left-eye image and the right-eye image is d) are projected on the screen. In the drawing, for projection onto the screen, the projected point by the point of the left image is indicated by a black circle point, and the projected point by the right image is indicated by a white circle point. Let D be the distance between the circle points on the screen. The viewer views the projected point on the screen by looking at the white circle point with the right eye and the black circle point with the left eye, thereby visually recognizing a three-dimensional image by fusion at that point. It is a figure which shows the position of the fusion by the value d of the shift | offset | difference of the position of the same object area | region in the image for left eyes, and the image for right eyes. The gap D between the same target areas on the left-eye image and the right-eye image on the display surface of the stereoscopic display device is proportional to d. (A) shows a case where d> 0 and a stereoscopic image by fusion is visually recognized behind the display surface of the stereoscopic display device. When the right image pasted on the flat plate is moved in the positive x-axis direction so that the displacement value d increases, the white circle point projected on the screen also moves in the same direction, and the stereoscopic image by fusion is viewed. Retreat from the person. (B) shows the case where d <0 and a stereoscopic image by fusion is visually recognized before the display surface of the stereoscopic display device. When the right image pasted on the flat plate is moved in the negative x-axis direction so that the displacement value d decreases, the white circle point projected on the screen also moves in the same direction, and the stereoscopic image by fusion is viewed. Approach the person. (C) shows a case where black circle points and white circle points of the left and right images overlap when d = 0. Both round dots on the screen are also overlapped with each other, and a three-dimensional image by fusion is visually recognized at a position on the display surface of the three-dimensional device. Expansion limit magnification m _0, is a diagram showing the relationship reduction limit magnification m _1, and of the number 2 of the t value t _0, t _1. When the exponent n of the forward / backward factor f is 1 (壱), the fusion equation is a function passing through the point (m ₀ , t ₀ ) and the point (m ₁ , t ₁ ). Further, the fusion equation when the index n is 0 is a hyperbolic function. It is a figure which shows the example of the shift | offset | difference of the undesirable image in a set of stereo photography. (A) is vertically shifted between the left and right images, (b) is right when the right image is greatly shifted to the left between the two images, (c) is the right image or the left image It is a figure which shows the case where it inclines (or rotates) centering on the center. It is a figure which shows the affine transformation for correct | amending the gap | deviation which arises between the right-and-left images of said set of three-dimensional photography. This affine transformation can be performed using the already known 2DCG coordinate transformation function. Misalignment correction is performed by the above-described coordinate transformation, that is, affine transformation that performs linear transformation and parallel movement by rearranging the right image flat plate on which the right image photograph of the stereoscopic photograph is pasted on the two-dimensional plane. I can do it. This affine transformation is expressed as a product of a matrix of 3 rows and 3 columns using an expansion coefficient matrix (affine transformation matrix M). It is a figure which shows the principal part of the program of an affine transformation. The program shown in (a) fixes the left image flat plate, translates along the y-axis and x-axis with respect to the right image flat plate (procedure 1 and procedure 2), and centers the center of the left image flat plate. The right image flat plate is rearranged with respect to the left image flat plate by performing the rotation (procedure 3). If the aspect ratio of the display screen and the stereoscopic photograph does not match, correction is performed (procedure 4). As a result, the shift between the left and right images and the aspect ratio can be adjusted. Further, in the part before the procedure 1, zoom conversion for zooming and parallel translation for shift are performed by the operation of the viewer. The program shown in (b) shows a usage example of a coordinate conversion method (Non-Patent Document 2) of Java (registered trademark) 2D-API and an example of a fusion formula. It is a figure which shows the example of SPV developed. In this SPV, the above Java (registered trademark) 2D-API was adopted as 2DCG. Java (registered trademark) 2D is based on the Java (registered trademark) program language. The Java (registered trademark) 2D is responsible for image processing and coordinate transformation (affine transformation). Source code using the Java (registered trademark) 2D-API is translated by a Java (registered trademark) compiler and executed by a Java (registered trademark) virtual machine. The Java (registered trademark) program part reads two left and right stereoscopic photograph images taken by two cameras, and synchronizes the zooming of the two images. Therefore, the SPV stereoscopic format adopts a side-by-side format that is standard in the world of stereoscopic photography and video. It is a figure which shows the several operation button used in order for a viewer to set about each stereo photograph in SPV of this invention. It is a figure which shows the position of the said operation button in a display screen.

  Embodiments of the present invention will be described below in detail with reference to the drawings. In the following description, devices having the same function or similar functions are denoted by the same reference numerals unless there is a special reason.

  The stereoscopic photograph image in the present invention is obtained by capturing an object to be photographed with left and right cameras as shown in FIG. The distance between the lenses of the camera is determined by the size of the subject to be photographed and the viewing angle, and is usually about 6.5 cm in the case of portrait photography, but in the case of a high-rise building, the distance is about 1 to 2 m or more. There is. At such a distance, the image by each lens is fixed on each imaging surface. Here, the imaging surface is a photosensitive surface of a film or a CCD imaging device.

  As shown in FIG. 2, the image fixed on the imaging surface is projected as a left and right stereoscopic photograph image by the left and right projection lenses. This projection lens is assumed to be a zoom lens that can change the projection magnification. It is well known that the viewer can perceive a stereoscopic image by directly viewing the photograph-left and photograph-right in FIG. 2 with the left eye and the right eye, respectively. In addition, the image of a three-dimensional photograph here does not only mean projecting the said image fixed to the imaging surface only, but includes also about the mapping of the image. For example, an image in which photographing data obtained by a CCD camera is displayed on a display device of a television apparatus or a computer is also a stereoscopic photograph image.

  FIG. 3 is a diagram showing a fusion position when the end face of the left-eye image and the end face of the right-eye image on the stereoscopic display device are displayed in alignment. In this case, the image for the left eye on the stereoscopic display device is generally located to the right of the image for the right eye, and the stereoscopic image by fusion is viewed at a position close to the viewer side from the display screen of the image. The viewer perceives the stereoscopic image that has popped out of the screen. Here, an example in which the liquid crystal shutter system is used in the stereoscopic display device of FIG. 3 is shown, but there is no reason to limit to this, and an anaglyph system or a polarizing filter system stereoscopic glasses system, a parallax barrier system, a lenticular lens system, etc. Even if the naked eye method is used, an image for the left eye and an image for the right eye can be prepared.

FIG. 4 shows (1) the case where the distance D between the image points in the left-eye image and the right-eye image on the stereoscopic display device is reduced (D ₁ > D ₂ ), and (2) the viewer and is a diagram showing changes in fusion position when changing the distance L between the stereoscopic display device (L _1> L _2). First, the fusion position can be brought closer to the viewer by reducing the interval D shown in (1). Also, the fusion position can be brought closer to the viewer by shortening the distance L shown in (2).

  In the case of (2), it is understood that the order of the positions of the fusion is maintained. In FIG. 4, for the pair of the image point right A and the image point left A and the pair of the image point right B and the image point left B having a narrower interval D, the first viewer respectively The near-side stereoscopic image B can be visually recognized, and the second viewer can visually recognize the stereoscopic image A ′ and the near-side stereoscopic image B ′, respectively. As described above, when the distance between the left-eye image and the right-eye image is narrower, it can be seen that the relationship that a stereoscopic image by fusion can be visually recognized at a deeper position does not change depending on the position of the viewer.

  As shown in FIG. 5, when there is no convergence angle, the distance between the viewer's left eye and right eye is e, the distance from the left eye image to the right eye image on the stereoscopic display device is D, When the depth distance from the display screen of the display device to the stereoscopic image by fusion is u and the distance between the viewer and the stereoscopic display device is h, e> D, and the height of the triangle and the length of the base Using the ratio, the following relationship is established (Non-patent Document 1).

FIG. 6A shows the relationship between the distance u and the distance D when e = 6.5 cm and h = 100 cm.
From this relationship, the following can be understood.
(1) The depth distance u increases as the distance D increases, and the fusion position becomes deeper.
(2) For D between [−∞, e], the depth distance u is a monotonically increasing function of the interval D.

In particular, when D = 0, u = 0, and a stereoscopic image by fusion can be visually recognized at the position of the display screen of the stereoscopic display device.
When the interval D is negative, the positions of the left-eye image and the right-eye image on the stereoscopic display device are switched from the above case, and the value of the depth distance u becomes negative. The position of the fusion is in front of the stereoscopic display device.

  Next, enlargement / reduction of an image in zooming will be described. First, FIG. 7 shows a conventional zooming method. Conventionally, an image for the right eye and an image for the left eye are displayed together on the display screen of the stereoscopic display device while being proportionally reduced and enlarged together. In addition, the position of the fusion of the three-dimensional image is shown. The left and right pairs of A on the display screen can visually recognize the fusion image at the position of the stereoscopic image A. When this is proportionally enlarged like a pair of left and right of B, the fusion image can be visually recognized at the position of the stereoscopic image B. Further, in the case of a further enlarged pair of left and right C, a three-dimensional image cannot be viewed without melting. From the position of the fusion of these three-dimensional images, it can be seen that when zooming in by a proportional enlargement operation on the display screen, the three-dimensional image can be viewed farther away despite zooming in. Conversely, in the case of zooming out, it is visually recognized as approaching. This is different from the normal sensation.

  On the other hand, as shown in FIG. 8, when the position (point) of a certain region in the left-eye image and the right-eye image on the stereoscopic display device is fixed, the position (point) is enlarged around the position. Can do so without changing the position of the fusion of the stereoscopic image. That is, the left eye image and the right eye image are enlarged / reduced with the h and e values fixed, the interval D value fixed, and the center position (point) of the region of interest fixed. Thus, zoom-in and zoom-out can be performed without changing the fusion position u.

  Further, as shown in FIG. 6, since the depth distance u is a monotonically increasing function of the interval D, the image of interest in the left-eye image and the right-eye image is enlarged while the left-eye image and the right-eye image are enlarged. It can be seen that the fusion position in zooming in can be moved forward by reducing the center distance D between the points.

Next, the fusion of a three-dimensional image when zooming in will be described. FIGS. 9A and 9B are diagrams schematically showing enlargement by the projection lens in FIG. FIG. 9A is a photograph taken with an equal magnification zoom projection lens. The black circle point is assumed to be a distance from the center line of the enlargement whose position does not change by zooming. When this is zoomed m times as shown in FIG. 9B, the distance between the zoom projection lens and the display screen does not change, and the distance from the enlargement center line becomes m × a. In order to facilitate such zooming with respect to the left-eye image and the right-eye image, as shown in FIG. 9C, the left and right photographs are based on a reference in proportion to the distance from each projection lens. And the magnification is m times. In the figure, P _L and P _R are parallel lines extending from the center points C _L and C _R of the enlargement. This usually corresponds to the center line of the left (or right) eye field of view. As shown in FIG. 9C, the positions of the noticed round dots displayed on the display screen 1 in the left-eye image and the right-eye image are measured from the respective lines P _L and P _R.

The distance from the line P _L of the left-eye image of black dots of interest on the display screen 1, when the vector {a}, is on the display surface 2, the distance from the line P _L, m {a }. Similarly, the distance from the line P _R of the right-eye image of the white circle point of interest on the display screen 1, when the vector {b}, is on the display screen 2, the distance from the line P _L, m {b}. Also, a vector extending from the center point C _L to C _R is {z}, a vector extending from the noticed black circle point on the display screen 1 to the white circle point is {Dz}, and a noticed black circle point on the display screen 2 When the vector extending from the white circle to {D}
{Z} = {Dz} + ({a} − {b}) = {D} + m ({a} − {b})
There is a relationship.

Here, as shown in FIG. 9D, the lines P _L and P _R of FIG. 9C are aligned and displayed. As a result, it is clear that the image-left and right end faces are aligned. Thus, {Dd} = m ({b} − {a}), where {Dd} is a vector extending from the noticed black circle point to the white circle point on the m-fold image. When the viewer views this image, the black circle point is on the right and the white circle point is on the left, and the interval between the black circle point and the white circle point increases as the enlargement magnification m increases. It can be seen that a three-dimensional image by fusion can be visually recognized nearby.

In the above case, a three-dimensional image by fusion is visually recognized in front of the display screen, but fusion of this to the back of the display screen adds some vector so that {Dd} becomes a vector pointing to the right. This can be achieved. For example, when a stereoscopic photograph D is distributed in the section P [x ₀ , x ₁ ] in FIG. 6B, the distribution can be moved to the section P ′ by performing the movement r _1. In this case, the fusion image can be visually recognized at the back of the display screen.

Further, when zooming with the magnification M is performed, the section P moves to the section Q [Mx ₀ , Mx ₁ ], and the fusion image can be visually recognized in front of the display screen. Also in this case, by performing the movement r ₂ , the distribution can be moved to the section Q ′ and can be fused to the back of the display screen.

In the case of trying to fuse deeper than the display screen, the area in which the sections P ′ and Q ′ can be set is narrower than the area that the sections P and Q can take. Further, the width of the section P (or Q) and the width of the section P ′ (or Q ′) are the same. For this reason, when converting each section by uniformly reducing or enlarging, it may be assumed that the area is projected by the moving operation r. However, when a prior visual experiment was performed on a general stereoscopic photograph, when the distribution of D of the stereoscopic photograph was moved from the section Q to the section Q ′ by the moving operation r, most of the optimal stereoscopic images were fused. In this case, it was found that the section Q ′ falls within the range of about 0 cm to 3 cm. For this reason, it is possible to perform the above movement r ₁ or r ₂ accompanying zooming sufficiently without any particular problem.

  It is assumed that a flat plate on which the image for the right eye is attached is provided with a mechanism that can move left and right, and the flat plate on which the image for the left eye is attached is fixed. FIG. 11 is a conceptual bird's-eye view of the SPV mechanism as viewed from above in the negative y-axis direction. These two flat plates are provided so as to overlap each other, and a pair of left and right stereoscopic photograph images taken by two cameras for the right eye and the left eye are pasted to each by a texture function of 2DCG. . In FIG. 11, these flat plates are drawn in front and back for easy understanding, but conceptually they are the same thickness and overlap along the x-axis. An image for the left eye is pasted on the front plate, and an image for the right eye is pasted on the rear plate. However, an image taken with the left camera is used as an image for the left eye, and an image taken with the right camera is used as an image for the right eye. The center of the flat plate of the left-eye image is fixedly arranged at the origin of the xy plane coordinate system fixed to SPV. However, the flat plate of the image for the right eye is movable in the vertical and horizontal directions and rotates around the origin of the coordinate system around plus or minus. The xy plane coordinate system has a positive x-axis toward the right and a positive y-axis upward. Then, the left and right images attached to these flat plates are projected onto the display screen of the stereoscopic display device so as to selectively reach the left and right eyes. The viewer stereoscopically views the projected image. When the stereoscopic display device is of the glasses type, it is possible to visually recognize a stereoscopic image by fusion of the left and right images projected on the display screen while wearing the stereoscopic glasses, and when the stereoscopic display device is of the naked eye type, the naked eye is left as it is. At that time, the viewer views an arbitrary portion of the stereoscopic photograph (image) while zooming by a zooming operation using a mouse. Two points corresponding to each of the left image and the right image (that is, a black circle and a white circle, and the distance between the circle points of these images is d) are projected on the screen of the stereoscopic display device. In the image projected on the screen of the stereoscopic display device, the projection point by the point of the left image is indicated by a black circle point, and the projection point by the right image is indicated by a white circle point. Let D be the distance between the black and white dots on the screen. The viewer can visually recognize a white circle point with the right eye and a black circle point with the left eye, thereby visually recognizing a three-dimensional image by fusion of these points.

  As shown in FIG. 12A, the black circle point of the left image on the display screen of the stereoscopic display device is the starting point, and the distance d from the corresponding white circle point of the right image is the difference between the round dots between the left and right images. Value. If the white circle point of the image moves to the right side, the movement is positive, and if it moves to the left side, the movement is negative. The deviation d between the round points of the left and right images takes positive, zero, and negative values. Deviations between the circles in the image are projected on the screen. If the distance D between the dots on the screen at that time is the distance D, D and d are in a proportional relationship.

  According to the visual experiment, as shown in FIG. 12A, the right-eye image plate is moved in the x-axis positive direction so that the deviation value d between the circle points of the image on each plate increases. Then, the white circle points projected on the display screen similarly move in the positive direction of the x-axis, and the distance D between the circle points on the screen increases in the positive value region. Retreats from the viewer. Also, as shown in FIG. 12B, when the right image is moved in the negative x-axis direction so that the deviation value d between the round dots is reduced, the white circles projected on the screen are similarly x-axis. Since it moves in the negative direction and the distance D between the round dots on the display screen decreases in the negative value region, it is felt that the three-dimensional image by the fusion of both round dots on the screen approaches the viewer. Also, as shown in FIG. 12C, when the black and white dots of the left and right eye images overlap at a single point that is not misaligned, both the round points on the display screen also overlap with each other. The viewer feels that a fusion image is formed at a position on the screen.

  Next, the quantitative relationship between “the magnification m of the image projected on the display screen” and “the value d of the deviation between the round points of the two left and right images pasted on the flat plate” will be described below.

(1) First stage (inversely proportional relationship between d and m)
Below, using the magnification m of the projected image, the shift value d between the round dots of the left and right images pasted on the flat plate is expressed by a quantitative expression. First, as described above with reference to FIG. 9, when zooming is performed at a magnification m, a gap interval D between round dots on the display screen is m times the deviation value d. Thus, the gap D on the screen is proportional to the gap value d between the left and right images attached to the flat plate. As a result, as described above, the viewer feels as if the stereoscopic image is moving backward or forward as zooming in or out.

In order to prevent this, first, consider stopping the stereoscopic image at a specific position. For this purpose, the interval D between the round dots on the screen shown in FIG. 11 is constant even if the magnification m of the image is changed by zooming in. Also, when zooming out, the interval D between the round dots on the screen is fixed. The distance D between the black circle point and the white circle point projected on the screen is proportional to the deviation value d between the round points of the left and right images under the constant magnification m, and the sign of positive and negative is also the same. In order to make D constant regardless of the magnification m, d and m may be in an inversely proportional relationship. If the constant value is k ₁ and the inverse proportionality constant is k ₂ , the result is as follows.

Here, when the above equation is rewritten with k ₁ / k ₂ as a, the following inverse proportional equation regarding d and m is obtained.

From this, D becomes a constant value irrespective of the magnification m as follows.

  It is assumed that the position of the left image flat plate in FIG. 11 is fixed. In this case, the reason why a is a positive value is that the right image flat plate is moved to the left side (plus direction side) and the stereoscopic image is fused at the back of the screen. If the value is negative, the entire stereoscopic image is felt out of the screen, and generally a strain is applied to the eyes. Accordingly, when the position of the right image flat plate is fixed, a is a negative value.

(2) Second stage (forward / backward factor)
The interval D between the black and white circles according to Equation 6 is kept constant regardless of the magnification m of the screen, and the stereoscopic image does not move backward or forward by zooming. Even in such a state, the viewer does not feel unnaturalness in the movement of the stereoscopic image due to the fusion from the whole screen.

  However, if the three-dimensional image approaches the viewer in the case of zooming in and feels away in the case of zooming out, the viewer perceives it as a natural movement. Therefore, the following factors are introduced. This is hereinafter referred to as “forward / backward factor”.

The forward / backward factor f is 0 (zero) when the screen magnification m is ∞ (infinity) and 1 (壱) when the magnification m is m ₁ . m _{1 is referred} to as “reduction limit magnification”. The exponent n of the power is for adjusting the forward / backward movement of the stereoscopic image and takes a real value of 0 ≦ n ≦ 1. When zooming out to the limit magnification that can be browsed, the stereoscopic image gradually decreases, and there is a boundary magnification at which the stereoscopic image of the entire stereoscopic photograph becomes smaller than the screen size. This is the zoom-out limit magnification that can be viewed. The arbitrary magnification m and the reduction limit magnification m ₁ have the following relationship.

  The forward / backward factor f acts on the constant a in the equation as a correction factor. Using this factor, Equation 5 is corrected as follows.

  This is called an “ideal deviation formula”. Further, by substituting Equation 9 into Equation 4 for the interval D, the following equation regarding the interval D is obtained.

From Equation 10, it can be seen that the interval D when the index n is 1 (壱) is inversely proportional to the magnification m. Using this formula, in the case of a zoom-in operation (when m increases from m ₁ to ∞), the distance D between the projected round points goes from a positive value to 0 (zero), and the stereoscopic image is displayed to the viewer. Coming forward. In the case of a zoom-out operation (when m decreases from ∞ to m ₁ ), the interval D goes from 0 (zero) to a positive value, and the stereoscopic image moves backward from the viewer. In this way, by introducing the forward / backward factor f and multiplying it by the constant a of Equation 5 as a correction factor, the forward and backward movement of the entire stereoscopic image by zooming can be controlled to be a natural movement.

  Here, it will be shown below that the constant a in Equation 9 can be a parameter for adjusting the perspective of a stereoscopic image. From Equations 3 and 10, when the image magnification m is constant, the depth u of the stereoscopic image is a monotonically increasing function with respect to the interval D as shown in Equation 3, and the interval D is a constant as shown in Equation 10. It is proportional to a. Since the constant a is based on the ideal deviation equation as shown in Equation 9, the stereoscopic image position u is a monotonically increasing function with respect to the ideal deviation equation constant a. From this, it can be seen that the depth of the stereoscopic image increases as the value of the constant a in the ideal deviation equation increases, and the depth of the stereoscopic image decreases as the value of the constant a decreases. Actually, the value of the constant a is determined from a visual experiment, which will be described later.

(3) Third stage (ideal fusion type)
The following formula is further determined using the above formula (9).

Specifically:

  Expression 11 or Expression 12 is an extension of an ideal deviation equation by adding a constant b. Hereinafter, the constant b is referred to as a “double image prevention constant”. Also, Equation 12 is referred to as “ideal fusion type”. By adjusting the distance D according to Equation 12, it is possible to prevent the generation of a double image, and it is possible to perceive a three-dimensional image due to fusion so that an optimal perspective stereoscopic effect can be obtained for the viewer. This formula 12 (ideal fusion formula) is a mathematical expression of the above “perspective factor”.

  The double image prevention constant b is a constant related to prevention of generation of a double image. The constant b is a left image flat plate (position is fixed) and a right image flat plate when the value d of the ideal deviation formula of Equation 9 is 0 (or a value close thereto). This is the value of the gap between itself. If the value of the constant b is appropriately determined, stereoscopic vision can be achieved without generating a double image even at the zoom-in limit, but the value is determined from a visual experiment.

  The double image prevention constant b is related to the lens interval between the two cameras when a stereoscopic photograph is taken. The value d calculated from the ideal deviation equation of Equation 9 is a deviation value in the left and right images corresponding to the distance D between the double dots on the screen that can be visually confirmed by the viewer. The prevention constant b is the absolute value of the right image flat plate itself relative to the left image flat plate (the position is fixed) when the deviation d between the black and white dots in the left and right images is 0 (zero). This is the deviation value. Therefore, the fusion formula t of Formula 12 including the double image prevention constant b is used as a formula for coordinate conversion described later. The effect of the lens interval of the photographing camera is reflected in the double image prevention constant b, and the double image prevention constant b of the stereoscopic photograph to be stereoscopically viewed is determined by a visual experiment.

  In order to determine the ideal fusion constants a and b of the number 12 derived in the third stage, the following two methods are conceivable.

(A) Individual determination method: The magnification m of the image projected on the screen is brought close to a large value (m increases in the direction from positive value to ∞), and the projected left and right images (black circle point and white circle point) The value of the constant b is determined by visual experiment so as to overlap without deviation. Thereafter, the constant a is determined so that the depth of the stereoscopic image becomes the best by a similar visual experiment. This determination method will be described in detail later.
(B) Cooperative determination method: With respect to the magnification m for the fusion equation, a certain magnification m _a is determined, and the value t _a of the fusion equation t of Formula 12 such that an optimal stereoscopic image is perceived is determined by visual experiment. . Another magnification m _b is determined again, and the value t _b of the fusion formula t in Expression 12 is determined by a similar visual experiment so that the optimum stereoscopic image can be perceived at the same time. Then, a binary simultaneous equation with the constants a and b of the fusion equation as variables is established and the equation is solved.

However, the above two methods both have problems.
With regard to the above (A) individual determination method, when the magnification m is increased as much as possible (m increases in the direction from positive to ∞), the magnification becomes theoretically infinite beyond the resolution limit of the image, and the stereoscopic image It becomes difficult to fuse.
Regarding the (B) linkage determination method, even when a binary simultaneous equation is automatically processed inside the SPV, the linkage between the experiment operation of the viewer and the procedure for solving the equation becomes troublesome. That is, the procedure of obtaining the solutions of the simultaneous equations after the visual experiments for determining the constants a and b are performed in association with each other imposes restrictions on the viewer, which is inconvenient in terms of usability.

(4) Fourth stage (practical fusion type)
Here, (3) a fusion formula that allows the viewer to easily determine the constants a and b in order to avoid the problems of (A) the individual determination method and (B) the cooperative determination method described in the third stage. To improve. The improved fusion equation is an approximation of an ideal fusion equation, but if it is used, the individual decision method can be easily applied in the determination of the two constants a and b. The reason is that (A) the problem of infinite image magnification can be avoided, (B) the constants a and b can be determined independently in a visual experiment, and (C) the role of the constants a and b in the determination. The meaning of is also clarified. In addition, there is an advantage that the viewer can reduce the work to one pattern without any experimental restrictions. Therefore, in order to be able to apply the individual determination method, the ideal fusion formula is improved below.

First, a magnification m ₀ that maximizes the enlargement of the image on the screen is introduced. Hereinafter, this is referred to as “magnification limit magnification”. The enlargement limit magnification m ₀ is a magnification determined depending on the resolution characteristics of the stereoscopic camera. The enlargement limit magnification m ₀ and the arbitrary magnification m have the following relationship. This is because if the magnification m exceeds the enlargement limit magnification m ₀ , the resolution of the image is exceeded, so that a three-dimensional image by fusion becomes meaningless.

In the visual experiment, the enlargement limit magnification m ₀ is usually determined as a value of about 10 to 20 times the reduction limit magnification m ₁ . By using this enlargement limit magnification m _0, as shown below, the ideal fusion equation of Formula 9 can be improved to a fusion equation that is easy to handle experimentally.

In the equation 5 defined in the first stage, the reciprocal (1 / m) of the magnification m is replaced with (1 / m−1 / m ₀ ) as follows. This replacement is performed by setting the magnification m to the enlargement limit magnification m ₀ and setting the value d of the deviation equation to 0 (zero), thereby creating the situation of FIG. As a result, the black and white circles overlap on the screen.

  Along with this, the forward / backward factor f in Expression 7 is changed as follows.

The changed forward / backward factor f is a function that takes a value of 0 (zero) when the magnification m is m ₀ and 1 (１) when the magnification m is m ₁ (m is m ₁ ≦ In the range of m ≦ m ₀ , f is in the range of 0 ≦ f ≦ 1). m ₁ is the above reduction limit magnification. The exponent n of the power is to adjust the forward / backward movement of the stereoscopic image in the same manner as described above, and takes a real value of 0 ≦ n ≦ 1.

  Then, the forward / backward factor f of Formula 15 is applied to the constant a of Formula 14 as a correction factor as follows.

  This equation is hereinafter referred to as a “practical deviation equation”. Also, simply speaking, the equation of deviation indicates the equation (16).

  Further, a new fusion equation is obtained as follows by the same method as that obtained in the above (3) third stage.

The newly determined fusion equation of Equation 17 approaches the ideal fusion equation of Equation 12 as a function when the magnification m ₀ is a value significantly larger than m ₁ . This formula is hereinafter referred to as “practical fusion formula”. Further, simply speaking, the fusion type refers to this practical fusion type.

When a practical fusion formula is used, the distance D between the two round points on the screen uses the upper formula of Equation 4 and Equation 16 and Equation 15, and further, the magnification m of Equation 4 is an inverse (1 / m ) And replacing the reciprocal with (1 / m−1 / m ₀ ) as in Equation 14, the following equation is obtained.

  The middle expression of the equation 18 relating to the interval D is obtained by multiplying the constant a by the forward / backward factor f as a correction factor. This equation is a monotonically decreasing function with respect to the magnification m. When the index n is 1 (壱), Expression 18 is a hyperbolic expression.

(5) Role of practical fusion formula The role of practical fusion formula is the role of the ideal fusion formula described in (1) 1st stage to (3) 3rd stage. Basically the same. The roles are summarized below. The difference between the practical fusion type and the ideal fusion type is that the enlargement limit magnification m ₀ is introduced into the former. In a practical fusion equation, the fusion equation obtained by setting the reciprocal (1 / m ₀ ) of the magnification limit magnification m ₀ to 0 (zero) is an ideal fusion equation.

(A) The fusion constant b is responsible for preventing the generation of double images.
The constant b plays a role in preventing the generation of a double image. The value of the constant b is determined by visual experiment so that a double image is not generated at the enlargement limit magnification m ₀ . Therefore, this constant is called a “double image prevention constant”. The constant b is determined for each stereoscopic photograph, and the determination method will be described later.

(B) The perspective factor for adjusting the position of the three-dimensional image fusion is borne by the fusion constant a.
The practical fusion constant a in Expression 17 is a constant for adjusting the perspective of the stereoscopic image. From Expression 3 and Expression 18, when the magnification m is constant, for the depth position u of the stereoscopic image, the position u is a monotonically increasing function with respect to the interval D (Equation 3), and the interval D is a monotonically increasing function with respect to the constant a ( (Equation 18). Since the constant a is based on a practical deviation equation (Equation 16), the position u of the stereoscopic image becomes a monotonically increasing function with respect to the constant a in the practical deviation equation. From this, the value of the position u increases as the value of the constant a in the practical deviation equation increases, and the depth of the stereoscopic image increases accordingly. Further, as the value of the constant a decreases, the value of the position u decreases, and the depth of the stereoscopic image also decreases accordingly. A constant a plays a substantial role in the above-mentioned “perspective factor”. The value of the constant a is determined by visual experiment so that the stereoscopic image has the best depth feeling at the reduction limit magnification m ₁ . The constant a is determined for each stereoscopic photograph, and the determination method will be described later.

(C) The interval D is responsible for moving the stereoscopic image so as to match the human sense.
The amount d representing the deviation between the black and white circle points of the left and right images pasted on the flat plate corresponds to the distance D between the black and white circle points projected on the screen. As can be seen from Equation 18, the interval D between the black and white dots projected on the screen increases or decreases monotonously with respect to the magnification m, and the interval D is determined. Accordingly, using the practical fusion formula of Equation 17, the stereoscopic image smoothly advances toward the viewer (the value of D changes from a positive value to 0) when zooming in (m increases), and zooms. In the out operation (m decreases), the viewer smoothly moves backward (the value of D changes from 0 to a positive value), and the above-described unnatural movement is avoided. The mechanism for moving the position of the fusion of the stereoscopic image so as to match the human sense is the forward / backward factor f introduced in the upper stage of Expression 16, and the result is reflected in the interval D of Expression 18.

(D) The forward / backward adjustment of the three-dimensional image is performed by the exponent n of the forward / backward factor f.
As described above, it is the forward / backward factor f that advances / retreats the stereoscopic image. The effect of the forward / backward factor f on the viewer varies depending on the exponent n. Exponential n is a constant that takes a real number between 0 (zero) and 1 (、), but as the value of exponent n increases from 0 (zero) to 1 (壱), the effect of forward / reverse becomes significant. . When the index n is 0 (zero), the forward / backward factor f does not act on the fusion equation. Some viewers tend to feel that the force is insufficient when n = 0, and the effect is too strong when n = 1. Since the sense of stereoscopic vision has individual differences, the index n can also act as a constant that accepts the individual differences. However, the SPV developer should determine this value to be an average value that absorbs individual differences through prior visual experiments.

(E) The enlargement limit magnification m ₀ and the reduction limit magnification m ₁ can be regarded as SPV constants.
The enlargement limit magnification m ₀ and the reduction limit magnification m ₁ can be regarded as SPV constants. The former enlargement limit magnification m ₀ is a value related to the resolution of the camera, and the value depends on the content of the photographed stereoscopic photograph (for example, night scenery, fog scenery, summer coastal scenery, person, etc.). Will also change. Therefore, it is also conceivable to set the enlargement limit magnification m ₀ for each stereoscopic photograph. However, according to prior visual experiments, the same value can often be used among normal and commonly used digital cameras. Therefore, this value can also be set as a default value by the SPV developer. The latter reduction limit magnification m ₁ is a magnification at the limit of zoom-out. As described above, when zooming out, the stereoscopic image gradually becomes smaller, and there is a position immediately before the entire stereoscopic image becomes smaller than the screen size, but the zooming point is the reduction limit magnification m ₁ and viewing is possible. It can be said that this is the limit magnification. In order to specify the magnification, the reference magnification must be determined. In SPV, the reduction limit magnification m ₁ can be set as the reference magnification, and the magnification can be determined as 1 (壱). However, for the convenience of explanation, this reduction limit magnification will be referred to as m ₁ hereinafter. In this way, the range of magnification that can be zoomed is from the reduction limit magnification m ₁ to the enlargement limit magnification m _0. Normally, when a commonly used digital camera is used, the enlargement limit magnification m ₀ is reduced. It is about 10 to 20 times the limit magnification m ₁ .

(F) Graph Expression of Fusion Formula Considering the fusion formula of Equation 17 as a function of magnification m, the value of t at m = m ₀ is t _0, and the value of t at m = m ₁ is Assuming t ₁ , the following relationship is obtained.

FIG. 13 shows the state of the fusion type using the above enlargement limit magnification m ₀ , reduction limit magnification m ₁ , and the above t ₀ and t ₁ . FIG. 13 shows an example in the case of a = 1.0, b = 0.5, m ₁ = 1, m ₀ = 10. The fusion equation when the exponent n of the forward / backward factor f is 1 (壱) is a function that passes through the point (m ₀ , t ₀ ) and the point (m ₁ , t ₁ ). Further, the fusion equation when the exponent n of the forward / backward factor is 0 is a hyperbolic function.
Note that the range (zoomable range) of the magnification m of the image projected on the screen is m ₁ ≦ m ≦ m ₀ as described above.

Here, the operation of the practical fusion equation (Equation 17) will be described below with respect to the value d of the deviation equation when the double image prevention constant b is zero.
(1) Value d> 0 In this case, the constant a is a positive value. A three-dimensional image formed by fusion of the round dots projected on the screen by the left and right images is visually recognized behind the display screen as shown in FIG. When the magnification m is increased by the zooming operation, the value d of the deviation formula approaches 0 (zero) from the positive value, and accordingly, the interval D between the double dots projected from the left and right images pasted on the flat plate is According to Equation 18, the positive value approaches 0 (zero). In that case, since the number 3 u that determines the position of the stereoscopic image decreases, the stereoscopic image feels closer to the viewer. Conversely, when the magnification m is lowered, the value d of the deviation formula increases, and the distance D between the two round points projected on the display screen of the stereoscopic display device increases. In that case, since the number 3 u that determines the position of the stereoscopic image is increased, the stereoscopic image is felt to recede from the viewer.
(2) Value d <0 In this case, the constant a is a negative value. When the constant a is a negative value, the following inconvenience occurs. The positions of the black circle points and white circle points of the left and right images projected on the display screen are opposite to those in the case of (1) value d> 0, and the stereoscopic image is in front of the display screen as shown in FIG. A stereoscopic image by fusion is visually recognized. Further, when the magnification m is increased, the deviation value d approaches 0 (zero) from the negative value, and the distance D between the double dots projected from the left and right images pasted on the flat plate varies from the negative value to 0 (zero) from Equation 18. Get closer to. In this case, since the numerical value 3 u that determines the position of the fusion of the stereoscopic image approaches 0 (zero) from a negative value, the stereoscopic image feels like retreating from the viewer. Conversely, when the magnification m is lowered, the interval D becomes a negative value from 0 (zero). In this case, u in Formula 3 that determines the position of the fusion of the stereoscopic image is changed from 0 (zero) to a negative value, so that the stereoscopic image is felt toward the viewer. Such a stereoscopic image is inconsistent with human senses. Therefore, it is necessary to adopt a positive value for the constant a. (3) When the value d = 0, the right and left images just pasted on the flat plate In this case, there is no deviation between the two round points projected (black circle point and white circle point), and at this time, the magnification m is m _{0 and} the value d of the deviation equation is zero and is projected on the display screen. The distance D between the two round points is also 0 (zero). In this case, as shown in FIG. 12C, the stereoscopic image is visually recognized at the position of the display screen.

  The practical fusion formula constants a and b shown in the above equation 17 are constants determined by a visual experiment by the viewer's eyes. These determine the stereoscopic characteristics of each stereoscopic photo in SPV, and these values are determined so that the perspective stereoscopic effect of the viewer is optimized. However, the constant a is a positive value. The constant b takes one of positive, negative, and zero values.

  Next, an optimal perspective stereoscopic setting method using a fusion formula will be described. As described above, when the image is enlarged by increasing the magnification m, a double image may be generated. This is because the gap D between the round dots on the display screen of the stereoscopic display device is too large. It is. In order to prevent this, it is necessary to set the double image prevention constant b appropriately.

Further, as described above, when it is felt that the entire stereoscopic image is projected from the stereoscopic display device screen, the constant a is close to 0 (zero) even if the magnification m is close to the reduction limit magnification m _1. When the value (where a> 0) is set, the contribution of the value d of the deviation equation constituting the practical fusion equation (Equation 17) becomes small. As a result, a stereoscopic image with a perspective stereoscopic effect may be felt nearby, and the viewer may become tired easily. In that case, in order to prevent this, it is necessary to appropriately set the value of the constant a. Thus, the constants a and b are important for controlling the fusion of the three-dimensional image. Therefore, the method of setting these constants will be described in detail below.

  First, a method for setting each constant of the practical fusion equation (Equation 17) will be described. When the viewer performs a zooming operation and changes the magnification m, a mechanism in which the position of the right image flat plate is translated by a value t on the x axis according to the fusion equation (however, the position of the left image flat plate is fixed). It is assumed that the SPV has a mechanism for reading and storing the value at that time. As a method for determining the constants a and b, first, the constant b is determined, and then the constant a is determined. Each constant is set for each stereoscopic photograph.

(1) Determination of constant b A method for determining the constant b in accordance with the actual case will be described.
(A) For a certain stereoscopic photo A, a zoom-in area that is relatively distant from the image is selected, and the magnification of the image is zoomed in up to the enlargement limit magnification m ₀ for that area.
(B) When the viewer looks at the display screen of the stereoscopic display device and there is a deviation between the corresponding area of the left image and the area corresponding to the right image, the right image is to be eliminated. A parallel image is moved on the x-axis, the right image area and the left image area are overlapped without deviation, and a stereoscopic image is visually recognized at the position on the display screen as shown in FIG. So that The fusion is visually confirmed by a visual experiment by a viewer.
(C) That is, the value of the constant b is changed and adjusted so that the position of the three-dimensional image fusion is the position on the display screen.
(D) The value of the constant b when the above-mentioned fusion is achieved is b _0, and using this value, the value t ₀ for translating the right image flat plate on the x-axis is a practical value of Using a fusing formula,

(E) The position of the right image flat plate is rearranged using this value t ₀ .
(F) The value b _{0 of the} constant b is a specific constant related to the stereoscopic photograph A, and is stored in a predetermined storage portion of the SPV.
When the constant b is determined in this way, since the value d of Expression 17 relating to the left and right images is 0 (zero) at the image enlargement limit magnification m ₀ , the double position is obtained at that position as shown in FIG. A three-dimensional image by a fusion without image generation is visually recognized.

(2) Determination of constant a Next, how to determine the constant a will be described. The value of the constant a is set so that a three-dimensional image resulting from the fusion of a relatively close-up area of the three-dimensional photograph is visually recognized at a position on the display screen or a position behind it.
(A) Since the constant b is already determined as described above, a state without deviation is realized at the enlargement limit magnification m ₀ . From this state, zoom-out is performed to the limit range where the magnification m can be viewed. The limit position that can be browsed is that the stereoscopic image gradually becomes smaller by zooming out, and there is a boundary position where the stereoscopic image of the entire stereoscopic photograph becomes smaller than the display screen size, but the magnification at that time is This is the reduction limit magnification m ₁ and can be said to be the zoom-out limit magnification. The reduction limit magnification m ₁ may be set as a reference magnification to 1 (倍率).
(B) The value of the constant a is determined at the reduction limit magnification m ₁ . For this purpose, the viewer visually determines the perspective of the stereoscopic image due to the fusion of the left and right images. At this time, the viewer pays attention to a relatively foreground area, and a positive constant a value so as to visually recognize a three-dimensional image of the fusion area at a position on the display screen or a position behind it. Adjust. At this time, the stereoscopic effect felt by the viewer is the perspective stereoscopic effect described above, and the setting of the constant a is the setting of the perspective factor.
(C) The value of the adjusted constant a at that time is defined as a _1, and the value t ₁ used to translate the right image flat plate on the x-axis using the value is expressed by a practical fusion equation of Expression 17. Use it as follows.

(D) The position of the right image flat plate is rearranged using this value t ₁ .
(E) The value a _{1 of the} constant a is a specific constant related to the stereoscopic photograph A, and is stored in the SPV.
In particular, when the constant a is a small value, a stereoscopic image by fusion is visually recognized in front of the display screen. When the constant a is a certain size, the entire stereoscopic image is visually recognized at the rear of the display screen, so that an appropriate perspective stereoscopic effect that is easy to see can be obtained. This state depends on the constant a, and if a value that is extremely large is set, depending on the viewer, the perspective stereoscopic effect may be felt far away, resulting in a strange stereoscopic image. If the limit is exceeded, the difference between the left and right images becomes too large, resulting in a double image and the stereoscopic image does not fuse. As described above, it is important to appropriately determine the value of the constant a. However, in general, since the suitability value of a has a considerable range, its setting is easy.

  As a general rule, 3D photographs are taken using two cameras with the same performance. However, even when 3D photographs are taken using such a method, vertical and horizontal deviations between the two left and right images are not possible. As a result, it may not be suitable as a pair of stereoscopic photographs. Even if a camera dedicated to stereoscopic photography is used, when zooming is performed, a similar shift becomes apparent in the vicinity of the limit point of the resolution. The slightest installation error that occurs when two cameras are attached to the camera platform causes the deviation. This shift is not so much a problem when using the stereoscopic photograph for the purpose of viewing the whole. However, if you zoom in (zoom in) and look closely at a narrow area, the misalignment between the left and right images may become noticeable, and such misaligned stereoscopic photographs can be used as a pair of stereoscopic photographs. Absent. In that case, the SPV needs to incorporate a mechanism for correcting the shift between the left and right images.

  FIG. 14 shows an example of image shift in a set of stereoscopic photographs. As shown in FIG. 14 (a), if the left and right images are significantly shifted vertically, the stereoscopic image is not fused, so it is necessary to align it horizontally. The cause of such vertical misalignment occurs when two cameras are fixed to a camera platform with screws, or when one of them is loosely tightened. In this correction method, the flat plate on which the right image shown in FIG. 14A is pasted is translated along the y axis in the plus direction by p.

  In addition, the two images may be shifted from side to side. This occurs when the two cameras are attached to the camera platform when the distance between the lenses is too large, and is more likely to occur when a close object such as a person is photographed. As shown in FIG. 14B, when the shifted image is viewed on the SPV, the “perspective stereoscopic effect” is felt quite close, and the three-dimensional image formed by the fusion pops out and is perceived. Such popping out is a feature of stereoscopic photography, but when it exceeds a degree, it becomes a stereoscopic photography that places a burden on the eyes. As shown in FIG. 14B, this displacement translates the flat plate of the right image along the x axis by a value d or more. However, this correction can be dealt with by adjusting the constants a and b of the above fusion equation.

  Further, as shown in FIG. 14C, the right image or the left image may be slightly inclined (or rotated) around the center. In this case, when two cameras are attached to the camera platform, one of them is installed with a slight inclination with respect to the optical axis of the lens. An inclination of about 0.5 degrees may occur depending on how the screws are tightened. For this correction, as shown in FIG. 14C, the center of the plate of the right image is rotated by θ radians.

  Note that the inclination of the image by installing the camera may be an x-axis or y-axis inclination (rotation), but this inclination is a minute angle. It was also found that this correction is absorbed by other corrections by prior experiments. That is, the correction of the x-axis tilt is approximately regarded as the correction of the vertical deviation, and the correction of the y-axis tilt is approximately regarded as the correction of the horizontal shift. As a result, it is reflected in the correction operation of the deviation in the vertical direction and the horizontal direction, respectively. For this reason, in order to avoid the complexity of correction, it is also possible to exclude direct correction of displacement due to rotation around the x-axis and y-axis.

  In addition, when two cameras are installed along the edge of the camera platform, the two cameras may be arranged so as to be shifted from each other in the front-rear direction with respect to the subject direction. However, such a deviation value is usually extremely small compared to the distance from the camera to the subject (several meters to several tens of meters) (thousandths to tens of thousands of the distance to the subject). The deviation is below, and the deviation can be ignored. For this reason, in order to avoid the complexity of the correction, the correction of the deviation can be excluded.

  As described above, in order to correct the deviation generated between the left and right images, a known coordinate conversion function of 2DCG (two-dimensional computer graphics) can be used. The correction of the deviation is, for example, by rearranging the right image flat plate on which the right image photograph of a set of stereoscopic photographs is pasted, and can be performed by the above coordinate conversion. This coordinate transformation is an affine transformation that performs linear transformation and parallel movement of a two-dimensional vector. As shown in FIG. 15, this affine transformation is expressed as a product of a matrix of 3 rows and 3 columns using an expansion coefficient matrix (hereinafter, affine transformation matrix M).

FIG. 15 shows each element of the affine transformation.
Procedure 1: An affine transformation matrix M _{t1 that} moves p in the y-axis direction
Procedure 2: Affine transformation matrix M _t2 moving t in the x-axis direction
Step 3: An affine transformation matrix M _{r that} rotates θ radians around the origin of the coordinate axes
Step 4: An affine transformation matrix M _s that changes the aspect ratio in the x-axis and y-axis directions
A three-dimensional vector for the affine transformation matrix shown in FIG.
[X y 1]
And At this time, the point [x ₀ y ₀ 1] is represented by the affine transformation matrix M,
[X ₁ y ₁ 1] = [x ₀ y ₀ 1] M _t1 M _t2 M _r M _s
Is converted to

In the procedure 2, the value t is an element of the affine transformation matrix M _t2 . The deviation correction of the left and right images in the procedure 2 will be described below using the lowest equation of the practical fusion equation (17). The optimum rearrangement value t with respect to the x-axis of the right image flat plate is obtained by the practical fusion equation using the magnification m. When the viewer starts zooming, the value t is calculated from the practical fusion formula by the magnification m every moment. Based on this value t, an affine transformation matrix M _t2 is calculated, and a three-dimensional vector [x ₁ y ₁ 1] for the right image flat plate is calculated. Using this vector, the right image flat plate is rearranged to a position corresponding to the value t in real time.

Here, it can be seen that the difference in size between the left and right images can be corrected by applying the affine transformation matrix M _s in step 4 to the right image flat plate. However, in principle, a set of stereoscopic photographs is taken with two cameras having the same specifications and the same performance. In this case, generally, there is almost no difference in size. An identity transformation is used for the transformation matrix M _s . If two cameras having different performances are used for the above photographing, the difference in size between the left and right images can be corrected by applying the affine transformation matrix M _s .

Further, the affine transformation matrix M _s described above can be used for the aspect ratio conversion between the aspect ratio of the display screen of the stereoscopic display device and the aspect ratio of the stereoscopic photograph. The aspect ratio here is defined as the number of horizontal pixels / the number of vertical pixels on a display screen and the horizontal size / vertical size for a stereoscopic photograph. For example, a normal photograph has an aspect ratio of 1.333, and there is no aspect ratio problem when outputting to a stereoscopic display device of the same type screen using a notebook personal computer with an XGA (4 to 3 type) screen. . However, when the image is displayed in a horizontally wide recent wide area, the image spreads horizontally, so that coordinate conversion for adjusting the width is necessary. For example, when outputting to a wide-type stereoscopic display device with a WXGA (5-to-3 type) screen using an XGA (4-to-3 type) laptop computer, the stereoscopic image becomes wider as it is, so In order to return to the image width, it is necessary to increase the aspect ratio of the image on the personal computer side to 0.8 times. The corrected aspect ratio at this time is 1.333 × 0.8. If the ratio based on the aspect ratio (1.333) of the photograph is defined as the converted aspect ratio, the converted aspect ratio of the image at that time Becomes 0.8.

In addition, when a stereoscopic display device whose input format is a side-by-side format is used, the SPV stereoscopic format needs to be the same format. For this reason, the aspect ratio of the left and right images must be further compressed to 1/2. By the way, when outputting in side-by-side format from an XGA type personal computer to a WXGA type stereoscopic display device, the conversion aspect ratio of each of the left and right images is 0.5 × 0.8, that is, 0.4. become. The conversion aspect ratio is r / s, where r is horizontal and s is vertical, and the values r and s are obtained from the corresponding resolution of the stereoscopic display device and the size of the display screen. In the coordinate transformation for enlargement / reduction of the left and right flat plates for images, the values r and s are elements of the affine transformation matrix M _s .

  Since the above affine transformation matrix is not commutative, correct results cannot be obtained if the order is incorrect. The order of matrix operations for correction is performed in the order of procedure 1 → procedure 2 → procedure 3 → procedure 4. However, since the procedure 1 and the procedure 2 are commutative, the calculation order is not limited mathematically, but actually, since a visual confirmation experiment is initially performed with an unadjusted stereoscopic photograph, Make corrections first.

  How the affine transformation shown above is described in actual 2DCG (two-dimensional computer graphics) will be described using Java (registered trademark) 2D-API. The main part of the program is shown in FIG. The flow of description in other 2DCGs may be considered to be almost the same as the example shown here.

  The program shown in FIG. 16A fixes the left image flat plate, translates along the y-axis and x-axis with respect to the right image flat plate (step 1 and step 2), and the center of the left image flat plate. The right image flat plate is rearranged with respect to the left image flat plate by performing rotation (procedure 3) centering on. As a result, the shift between the left and right images can be adjusted. Further, in the part before the procedure 1, zoom conversion for zooming and parallel translation for shift are performed by the operation of the viewer. The program shown in FIG. 16B is a usage example of the coordinate conversion method of Java (registered trademark) 2D-API (Non-Patent Document 2) and an example of a fusion formula.

  As described above, when the coordinate conversion function of 2DCG (Java (registered trademark) 2D-API in FIG. 16) is used, the main problem of rearrangement of the right image flat plate can be simply described. In addition, the listed programs are organized and arranged for easy viewing, and some details have been omitted and are not complete. In the case of this Java (registered trademark) 2D-API program, it should be noted that since the origin is the upper left corner of the flat plate (image), the translation transformation is frequently used.

  FIG. 17 shows an example of the developed SPV. In this SPV, the above Java (registered trademark) 2D-API was adopted as 2DCG. Java (registered trademark) 2D is based on the Java (registered trademark) program language. The Java (registered trademark) 2D is responsible for image processing and coordinate transformation (affine transformation). Source code using the Java (registered trademark) 2D-API is translated by a Java (registered trademark) compiler and executed by a Java (registered trademark) virtual machine. The Java (registered trademark) program part generates left and right images, and synchronizes the zooming of the two images. Therefore, the side-by-side format is adopted as the three-dimensional format of SPV. FIG. 17 shows the cooperative relationship between Java (registered trademark) and Java (registered trademark) 2D-API. In the following, functions governed by languages and the like are listed. Since SPV employs Java (registered trademark), it operates without depending on the OS (operation system) of the computer to be used.

In FIG. 17, Java (registered trademark) 2D-API performs texture drawing and coordinate transformation, and rearranges the left and right images appropriately. That is, the following processing is performed.
-Utilization of texture drawing function-Setting of conversion matrix Java (registered trademark) works as a parent language of Java (registered trademark) 2D and performs the following processing.
・ Set up two screens for left-eye and right-eye images ・ Use scroll function for zooming ・ Synchronize left and right images during zooming ・ Use thread function for automatic zooming function ・ Store stereoscopic information

In the SPV, a plurality of operation buttons shown in FIG. 18 are prepared for the viewer to correct the deviation of each stereoscopic photograph. In this example, these operation buttons are provided at the lower part of the screen as in the example of the screen of FIG.
(1) Misalignment correction function button This button corrects misalignment to create a complete pair of stereoscopic photograph images as a stereoscopic photograph.
H button: Corrects the left / right misalignment (determines the constant b of the fusion equation).
・ V button: Corrects the vertical displacement.
• θ button: Corrects the deviation due to tilt.
These misregistration correction buttons are a group of buttons for correcting the misalignment of a pair of stereoscopic photographs that occur when taking a picture with a camera, and are used as a preliminary preparation before browsing. Every time the correction button is clicked by the viewer, a small value is created inside the SPV. Coordinate conversion is instantaneously performed for each click based on this value, and the right image flat plate is rearranged with reference to the left image flat plate. Each time the viewer visually recognizes a stereoscopic image by fusion of the left and right images with his / her own eyes. For example, when the plus button of the V button for performing vertical correction (switching between + and-with the SIGN button) is clicked, the right image flat plate is moved slightly upward. Click the plus or minus button of the V button until the optimal stereoscopic image is visible by fusion, and finely adjust the fusion state of the stereoscopic image.
(2) Depth determination function button A button for determining an optimal perspective perspective.
D button: determines perspective perspective (determines fusion constant a).
(3) Aspect ratio determination function button A button for setting an aspect ratio.
-A button: Adapts the three-dimensional image by fusion to the aspect ratio of the display screen.
(4) Double layer button This button is used to superimpose the left and right images in order to detect a shift between the left and right images.
-DBL-on button: The left image is translucently superimposed on the right image.
-DEL-off button ... Cancels the above function.
(5) Zoom function button A button for zooming a stereoscopic photograph.
・ ZM-on button: Zooms in an arbitrary part of a stereoscopic photo.
・ ZM-off button: Cancels the above function.
(6) Browsing function button This button is used when browsing stereoscopic photographs.
• NEXT button: View the next 3D photo.
-BACK button: View the stereoscopic photo displayed before.
-AUTO button: Automatic display like a photo frame and repeated viewing of stereoscopic photographs. (Also performs automatic zooming to any location.)
(7) Record button This button records the fusion type constants a and b.
-REC button ... Records and saves 3D photo data in the built-in file.

  The SPV has a function of superimposing left and right images. When this function is used, deviation correction can be easily performed without wearing stereoscopic glasses such as the polarizing glasses method, the anaglyph method, or the liquid crystal shutter glasses method. Since the 3D format of the SPV is a side-by-side format, when performing normal 2D display without using the 3D display function in the 3D display device, the left image on the left half of the display screen and the right image on the right half respectively , The image is compressed horizontally by a factor of 1/2. Accordingly, the DBL-on button is clicked to superimpose the semi-transparent image of the reference left image on the right image portion. Since the left and right images overlap, the position of the right image is adjusted according to the reference left image while observing the left and right overlap. When the adjustment operation is completed, the image overlay is canceled with the DEL-off button.

The deviation correction in the SPV is performed as follows.
(1) Vertical displacement correction This correction is a correction in which stereoscopic glasses are not worn. The photographer (also a viewer) of the stereoscopic photograph stores the left and right images of the photographed stereoscopic photograph with a file name in a folder held inside by the SPV. Photos are called with the NEXT button of the browsing function, and correction for each photo is performed as follows. Initially, by clicking the V button (+ and-) of the correction function, the left and right images are overlapped vertically by visual judgment so that there is no deviation. Then, the p value for correcting the deviation is recorded and saved in the viewer by clicking the REC button.

(2) Deviation correction by inclination Next, deviation correction by inclination is performed. This correction is correction without wearing stereoscopic glasses. By clicking the θ button (two types of + and −) of the correction function, the left and right images are overlapped by visual judgment so that there is no deviation due to tilt. The θ value for correcting the deviation is recorded and saved in the viewer by clicking the REC button. In prior visual experiments, there were very few stereoscopic photographs of objects that had to be corrected, depending on how the stereoscopic photograph was taken.

(3) Left / right deviation correction Next, right / left deviation correction is performed. This correction is first performed without wearing stereoscopic glasses, and then a stereoscopic image by fusion is confirmed using stereoscopic glasses. Enlarge the zooming operation of the localized portion of the distant view area of the stereoscopic photographs to the resolution limit (expansion limit magnification m ₀₎ to (zoom). Then, under this state, the correction function H button (two types, + and −) is clicked, and the left and right images are overlapped in the enlarged portion of the distant view so that there is no deviation. This determination is made visually. Then, in this state, wearing 3D glasses, it is confirmed whether or not a 3D image by fusion is visible at a position on the screen. If the stereoscopic image can be viewed at a position on the screen, the value of the constant b for correcting the left / right deviation is recorded and saved in the viewer by clicking the REC button.

(4) Adjustment of perspective stereoscopic effect Finally, the perspective stereoscopic effect is adjusted. This correction is performed by stereoscopic viewing with stereoscopic glasses. Click the D button (two types, + and-) to make it possible to appropriately obtain the perspective of the entire stereoscopic image by visual determination. First, the D button is clicked while zoomed out to the reduction limit magnification m ₁ . Then, if a proper perspective is obtained so that a three-dimensional image by fusion can be viewed on the display screen or at a position deeper than that, the value of the constant a that defines the perspective is clicked on the REC button to click inside the viewer. Save to record. The correction of the “perspective stereoscopic effect” is an adjustment rather than an image shift correction, and it is sufficient if an appropriate (preferred) perspective is obtained.

(5) Aspect ratio selection and browsing method At the start of browsing, in order to adapt the screen size of the stereoscopic display device and the resolution supported by the personal computer controlling the display screen, the A button (+ and-) When is clicked, the SPV automatically calculates the aspect ratio from the stereoscopic display screen or the like, so that the viewer selects an appropriate aspect ratio. When actually browsing, clicking on the AUTO button will cause the SPV to continue to display the stereoscopic photo while automatically zooming like a photo frame (many of the currently sold items are for non-stereoscopic photos). When the viewer clicks the NEXT button or the BACK button, an arbitrary zooming operation can be performed on a specific stereoscopic photograph.

  Actually, using two Fujifilm A220 digital cameras to create a pair of stereoscopic photographs with 12 million pixels and experimenting with SPV, it zoomed to about 12 times (about 150 times the area ratio). I was able to do it. The display stereoscopic display device used was a G170P polarized glasses display device manufactured by Dimen, and the resolution was XGA (1024 × 768).

  The stereoscopic photograph viewer SPV realized by using the principle of the present invention includes (a) a stereoscopic display device for a personal computer (both stereoscopic glasses and naked eyes are acceptable), and (b) a 3D stereoscopic television (commercially available household appliances and stereoscopic glasses). (3) Dual projector method (using two projectors) stereoscopic projection system, (d) Half mirror method (using two liquid crystal panels) high-definition stereoscopic display system It can be. The stereoscopic formats accepted by these stereoscopic display devices are the “side-by-side format” for the former two devices, and the “uncompressed two-image independent distribution format” (the left and right reproduced for stereoscopic viewing). The format in which the two images are distributed independently without being compressed). Therefore, the SPV can handle two three-dimensional formats: “side-by-side format” and “non-compressed two-image independent distribution format”.

Also, the above-described zooming mechanism in SPV can be applied to the technology of 3D virtual space of 3D graphics. 3DCG (three-dimensional computer graphics) employs a method in which a virtual person (generally called an avatar) having a viewpoint is placed in a virtual space, and a scene around the avatar is viewed on a display device. . This avatar can move around (walk through) in the virtual space. The avatar's viewpoint is looking at an image placed in a virtual space and pasted on a flat plate from a certain distance L. Therefore, in order to derive the fusion formula in the virtual world, the magnification m on the screen of the stereoscopic display device, which is the attribute (variable) of the 2DCG plane used in the fusion formula of Equation 17, is set to the 3DCG virtual space. What is necessary is just to replace with the distance L from the viewpoint of the avatar which is an attribute (variable) of this to the image. Since this distance L and the magnification m of the image projected on the display screen are inversely proportional, assuming that k is a constant inversely proportional to L and m, L ₀ × m ₀ = k, L ₁ × m ₁ = Since k is substituted for the magnification m in Equation 17, the following equation is obtained.

Further, the range of the magnification m in Expression 22 and m ₁ ≦ m ≦ m ₀ can be converted into the range of the distance L as follows.

Incidentally, L ₀ should be called “viewpoint approach limit distance”, and L ₁ should be called “viewpoint receding limit distance”. Here, t is replaced with X, a / k is replaced with α, and b is replaced with β.

  This is a coordinate conversion formula of the X coordinate regarding the distance L from the viewpoint of the avatar in the virtual space to the flat plate on which the image is pasted. Equation 24 is a fusion equation for walking through the virtual space in 3D graphics (approaching to the image or retreating from the image), and the constants α and β are determined by visual experiments. It is a constant determined by the same method.

  Note that a 3D photo viewer similar to the above SPV could be realized using the VRML language, which is 3D graphics, the JavaScript (registered trademark) language, and the HTML language. At that time, Equation 24 was used as the fusion formula for the virtual space. In terms of performance, it was equivalent to the above SPV.

The zoomable SPV of the present invention includes a stereoscopic display device that presents a left eye image and a right eye image to the viewer's left eye and right eye, respectively, and the left eye image on the stereoscopic display device. And a display control device that supplies a right-eye image, and an SPV that presents a viewer with a left-eye image and a right-eye image for displaying a stereoscopic image captured in advance ,
As for the gap D between the left-eye image and the right- eye image in the image displayed on the display surface so as to be fused in the back of the display surface of the stereoscopic display device, the distance D is monotonously decreased as the magnification is increased by zooming in. By zooming so that the position of the stereoscopic image by the fusion of the image for the left eye and the image for the right eye in the zoom-in moves to the front as viewed from the viewer,
Alternatively, with respect to the gap D between the left-eye image and the right- eye image in the image displayed on the display surface so as to be fused in the back of the display surface of the stereoscopic display device, the interval D is monotonously according to the reduction in magnification due to zoom-out. By zooming in, the position of the three-dimensional image resulting from the fusion of the left-eye image and the right-eye image during zoom-out is zoomed so as to move to the back as viewed from the viewer.

Claims (9)

A stereoscopic display device that presents a left-eye image and a right-eye image to the viewer's left eye and right eye, respectively, and a display control device that supplies the left-eye image and right-eye image to the stereoscopic display device. A stereoscopic photograph viewer for presenting a viewer with a left-eye image and a right-eye image for stereoscopic image display,
For the left eye in zooming in, the distance D between the same objects of the left image and the right image in the image displayed at the back of the display surface of the stereoscopic display device is monotonously decreased as the magnification increases due to zooming in. Zooming so that the position of the stereoscopic image by the fusion of the image and the image for the right eye moves to the front as viewed from the viewer,
Alternatively, the distance D between the same objects of the left image and the right image in the image displayed in the back of the display surface of the stereoscopic display device can be zoomed out by monotonically increasing the distance D as the magnification is reduced by zooming out. A stereoscopic photo viewer capable of zooming, wherein zooming is performed such that a position of a stereoscopic image resulting from the fusion of the left-eye image and the right-eye image in the camera moves backward as viewed from the viewer. The interval D is determined by changing D in proportion to α × (1 / m−β) ^p for p satisfying 0 ≦ p ≦ 1, where the magnification is m, α and β are constants,
Zoom in such a way that the position of the three-dimensional image resulting from the fusion in the zoom-in shifts forward when viewed from the viewer, or the position of the three-dimensional image resulting from the fusion in the zoom-out is moved to the back as viewed from the viewer. The stereoscopic photograph viewer capable of zooming according to claim 1, wherein the zooming is performed as follows. The left-eye image and right-eye image displayed on the stereoscopic display device are generated from a pair of stereoscopic photo images,
In the generation, the left-eye image and the right-eye image are generated by enlarging or reducing the pair of stereoscopic photograph images at a magnification m,
The zooming is based on a state in which the end points of the left-eye image and the right-eye image are aligned, and the distance t between the end-point of the left-eye image and the end-point of the right-eye image is used as a function of the magnification m. Done by changing,
The position of the three-dimensional image by the fusion in the zoom-in moves toward the viewer as viewed from the viewer, or
The zoomable three-dimensional photograph viewer according to claim 1, wherein zooming is performed so that a position of a three-dimensional image by the fusion in zoom-out moves to the back as viewed from the viewer. The distance t is a function of the magnification m, where m is the magnification, a, b, and k are constants, and a × (1 / m−k) ⁿ + b for ⁿ where 1 ≦ n ≦ 2. The zoomable three-dimensional photograph viewer according to claim 3, wherein: The constant b is obtained by zooming in on the relatively distant view area and relatively close view area selected in the zoom-in target area of the stereoscopic photograph image to the enlargement limit magnification m ₀ , and fusing the area. 5. The zoomable stereoscopic photograph viewer according to claim 4, wherein the three-dimensional image is determined by a viewer on the condition that the stereoscopic image is visually recognized at a position on the display screen of the stereoscopic display device. The constant a zooms out the stereoscopic photograph image to the reduction limit magnification m ₁ with respect to the relatively distant view area and the relatively close view area selected in the zoom-in target area of the stereo photograph image, and the relatively close view area. 5. The stereoscopic image obtained by the fusion of the above is determined by the viewer on the condition that the stereoscopic image is visually recognized at a position on the display screen of the stereoscopic display device or a position behind it. 3D photo viewer that can be zoomed. The constant k is characterized in that it is the inverse of the expansion limit magnification m _0, zoomable stereoscopic photograph viewer according to claim 6.   The stereoscopic display device is any one of an anaglyph method, a liquid crystal shutter method, a polarizing filter method, a parallax barrier method, or a lenticular lens method, according to any one of claims 1 to 7. 3D photo viewer that can be zoomed.   9. The left image and the right image in the image displayed on the stereoscopic display device are obtained by removing in advance a vertical shift and a tilt shift. A zooming stereoscopic viewer according to one of the above.

Images