JP4902012B1 - Zoomable stereo photo viewer - Google Patents

Abstract

A zoomable stereo photo viewer is provided that allows a natural image to be viewed without zooming in on a double image when zoomed in when viewing a stereoscopic photo.
A stereoscopic display device for presenting a left-eye image and a right-eye image to a viewer and an image display control device are provided, and the left-eye image and the right-eye image are presented to the viewer. The displayed distance D between the left and right images is changed according to the zoom-in (zoom-out) magnification m, so that the fusion position does not move or moves forward (backward). In particular, α and β are constants, 0 ≦ n, and D = α × (1−β × m) ^{n + 1} / ^mn . Further, zooming is performed by changing the distance t between the end points of the left and right stereoscopic photographs as a function of the magnification m. In addition, 0, n, c, b, k are constants, and the distance t is a function of the magnification m, which is c × (1 / m−k) ^{n + 1} + b, and is constant at the limit points of zoom-in and zoom-out. Determine a, b, k.
[Selection] Figure 6

Description

  The present invention relates to a zoomable stereo photo viewer that allows a viewer to view a natural fusion of a stereoscopic image even when a part of the stereoscopic image is enlarged.

Chapter 1 Introduction (1) The function of zooming stereoscopic photographs With the recent stereoscopic boom, stereoscopic techniques have begun to be applied in earnest to information-related technologies such as personal computers, cameras, televisions, game machines, mobile phones, and movies. Among them, stereo photography using binocular parallax that has been around for a long time still has many enthusiasts, and a group of like-minded people is also thriving. However, it can be said that the stereoscopic image browsing devices (including personal computer stereoscopic display software) used in the world are merely for viewing pictures three-dimensionally, and few have more functions. In view of such a situation, the inventor of the present invention has created a new mechanism of stereoscopic photo browsing software using a personal computer and a stereoscopic display device. This makes it possible to view stereoscopic photographs while smoothly zooming, and it can be said that this is a highly versatile method that can be applied when developing general stereoscopic photograph viewing software. The inventor of the present invention calls this method “stereo photo viewer”.

  In general, when using software that simply zooms in a 3D photo without special measures as shown in this specification, the 3D image is approaching in the process of enlarging (zooming in on) a fine area on the 3D display. The phenomenon of moving in a way that does not fit the human sense, such as retreating without coming, or the three-dimensional image that had been fused until then split into two images, the viewer However, there is a case where a three-dimensional image cannot be fused. The former is an undesirable phenomenon related to the essence of stereoscopic vision in stereoscopic zooming. The latter is a state in which the viewer's stereoscopic vision ability exceeds the limit and the fusion of the stereoscopic image is no longer possible.

  In particular, the latter inability to fuse a three-dimensional image is referred to as a “double image by fusion decay” (hereinafter simply referred to as “double image”). The double image at this time is simply an image in which two flat images are shifted and overlapped. (Note that there is a phenomenon similar to a double image called “ghost due to crosstalk” arising from the quality and performance of a stereoscopic display device, but this is not the subject of this specification).

  In view of this, the inventors of the present invention have examined the cause of the movement of a stereoscopic image and the generation of a double image using a two-dimensional computer graphics (2DCG) image processing technique in a stereo photo viewer. Then, we analyzed the mechanism of stereoscopic image fusion and defined a “fusion type” that controls zooming in a stereo photo viewer. The fusion formula is a mathematical formula that suppresses the generation of a double image and moves a three-dimensional image to fit a human sense.

  The inventor of the present invention has developed stereoscopic photo browsing software named “SPV” based on the stereo photo viewer system. Stereo photo viewer refers to a conceptual viewer, and SPV refers to software that implements the concept. The SPV has a zooming mechanism using 2DCG image processing technology and a fusion type that suppresses the generation of double images. Thus, a viewer using the SPV can view the desired region in a stereoscopic and clear manner while smoothly zooming the stereoscopic photograph.

(2) Function for editing a stereo photograph (stereo pair) Stereo photographs are obtained by attaching two cameras to a support base called a pan head and empirically determining the distance between the two lenses (stereo pictures in general). It has been enjoyed by enthusiasts for a long time by taking a picture of a stereo pair using a suitable 3D photo browsing device (made using lenses, mirrors, and prisms since the mid-19th century). ing. In this method, since the interval between the two lenses can be freely changed on the pan head, it is possible to take a stereoscopic photograph from a foreground to a distant view. Recently, a major Japanese camera company has begun selling a camera dedicated to stereo photography (two cameras that capture the left and right images are in one set). It is suitable for close-up shooting such as shooting. For this reason, the conventional method of using two cameras using a pan / tilt head is still attractive for enthusiasts who take a long-distance shot.

  By the way, when a stereoscopic display device is used, if a stereoscopic photograph is still taken, it cannot be an effective stereo pair. Due to a mistake in attaching the two cameras to the camera platform and the tilt of the camera at the time of shooting, it is normal that a slight deviation occurs in the superposition of the left and right images. Therefore, it is necessary to edit the left and right images so as to form an appropriate stereo pair by trimming the left and right images vertically and horizontally, or by rotating one image to correct the tilt. Since this is a rather troublesome task, recently, Adobe's premier image / video editing software is often used.

  Therefore, the inventor of the present invention thought that the stereo photo viewer must also have a mechanism for editing the left and right images to generate a complete stereo pair. The affine transformation (coordinate transformation) of 2DCG is applied to this mechanism. Using the stereo pair editing function of SPV that implements a stereo photo viewer as software, even a beginner can reproduce a stereoscopic image with a sense of depth from stereoscopic photographs taken with two ordinary cameras, not a stereoscopic photography dedicated camera. It can be done.

  SPV can reproduce a stereoscopic image on the screen of a stereoscopic display device (both glasses-type and naked-eye types), a stereoscopic television, or a stereoscopic projector.

  Chapter 2 analyzes the viewer's three-dimensional sensation, Chapters 3 to 10 show the concept and method of the stereo photo viewer, and Chapter 11 specifically introduces the SPV that has been realized.

Chapter 2 Stereoscopic Sense Analysis Section 2.1 Representation of Stereoscopic Sense by Model Expression The inventors of the present invention designed a stereo photo viewer and developed a prototype prototype viewer. I noticed that the sense of stereoscopic vision obtained from the stereoscopic image reproduced above changes considerably depending on the viewing environment of the stereoscopic photograph placed by the viewer. The browsing environment is created by the characteristics of the stereoscopic display device and the personal computer stereoscopic display software. (Depending on how the stereoscopic image is viewed, the stereoscopic effect of the viewer changes somewhat, but this is not the subject of this specification and will not be mentioned in this specification).

  The stereoscopic effect of the three-dimensional photographs taken by the two cameras varies depending on how the three-dimensional photographs are taken, the shooting scenery at that time, and so on, so each individual stereoscopic photograph has a “unique” stereoscopic effect. Therefore, this stereoscopic effect is referred to as “inherent stereoscopic effect” of the stereoscopic photograph. As is well known, the sense of depth of an individual stereoscopic photograph due to this inherent stereoscopic effect is determined by the situation at the time of shooting, and thus cannot be changed in principle. This will be reviewed in section 2.2.

  However, the stereoscopic effect due to the stereoscopic photograph browsing environment in which the viewer is placed (the characteristics of the stationary stereoscopic display device and the personal computer stereoscopic display software) is a slightly different stereoscopic effect from the inherent stereoscopic effect. This stereoscopic effect means that a stereoscopic image exists in the front of the stereoscopic display screen, in the rear, or in the vicinity of the screen if the stereoscopic image viewing environment is different even if the stereoscopic image is the same. It is a feeling. The stereoscopic effect that the viewer feels in this way is a stereoscopic effect in which some action is added to the inherent stereoscopic effect of the stereoscopic photograph.

  Therefore, if the stereoscopic feeling felt by the viewer is called “perspective stereoscopic effect”, and some action is called “perspective factor”, the inventor of the present invention can express the perspective stereoscopic effect by the following model expression. I noticed. (However, note that this model equation represents a conceptual image, not a strictly mathematical equation).

This number 1 represents that “the stereoscopic effect of the stereoscopic photograph felt by the viewer is the stereoscopic effect as a result of the action of a factor that causes some perspective on the stereoscopic effect unique to the stereoscopic photograph”. Therefore, the perspective stereoscopic effect felt by the viewer is analyzed by dividing it into the inherent stereoscopic effect and perspective factor of the stereoscopic photograph, as will be described in the following sections.
In this way, considering the stereoscopic effect of a three-dimensional photograph by dividing it into “perspective stereoscopic effect”, “inherent stereoscopic effect”, and “perspective factor” improves the perspective when designing a stereo photo viewer. This is because, for example, when zooming in on a stereoscopic photograph, the viewer sees the stereoscopic image by the left and right images (stereo pair) as a double image from a certain point in time, and a state in which the stereoscopic image cannot be fused occurs. The idea to suppress the generation of the double image can be realized from the analysis of the inherent stereoscopic effect and the perspective factor.

Section 2.2 Intrinsic stereoscopic effect Assume that there are three cars in the stereoscopic photo, and they are arranged in the order of Car A, Car B, and Car C from the front to the back. (In stereo photography, if there is a deviation in the vertical or tilt between the stereo pair, it is removed. This is an important requirement for the stereo pair to produce a stereo image. (How to adjust is described in Chapter 9). As described above, the depth of the stereoscopic image felt by viewers who look at it through a traditional peeping-type stereoscopic viewing device that uses lenses, mirrors, etc., when paying attention to the arrangement of the three cars. In the present specification, this is called “intrinsic stereoscopic effect”. The inherent stereoscopic effect of a stereoscopic photograph is already determined at the time of taking a picture and cannot be changed in principle.

  The factor that creates a unique stereoscopic effect in stereoscopic photography is the distance between the two lenses of the two cameras that photographed the scenery. Stereo pairs of stereoscopic photographs create a horizontal shift caused by the lens spacing of the left and right cameras. When the viewer appropriately views the stereo pair, the viewer's brain fuses the three-dimensional image based on the gap (that is, binocular parallax). As the interval increases, the depth of the stereoscopic image by the stereoscopic photograph is felt to increase. When the gap between the stereo pairs becomes larger than a certain value, it becomes difficult for the viewer to fuse the stereoscopic image, and when the limit is exceeded, the stereoscopic image collapses into a double image. When the gap is small, the sense of depth is reduced. When the gap is zero, the result is the same as when two identical images are viewed, and a three-dimensional feeling is not felt.

  Such shooting conditions for stereoscopic photography have been known for a long time. Therefore, in general, a stereoscopic photograph (stereo pair) to be viewed later using a stereoscopic display device and personal computer stereoscopic display software needs to be taken in a state where the two cameras have an appropriate lens interval in advance. There is.

  Note that the two cameras that take a three-dimensional photograph must be the same type and the same performance camera, and the two lenses are installed on the pan head so that the directions of the two lenses are parallel on the same plane. Shall be kept horizontal. (In addition to the distance between the lenses, there is a method called convergence as a method for generating a stereoscopic effect in a stereoscopic photograph. This method is a method in which the directions of two lenses are installed so as to cross each other. In 3D photography, the congestion is not adopted for the above reasons).

Section 2.3 Perspective stereoscopic effect The stereo photo viewer is executed on a personal computer, and the output stereoscopic image is reproduced on a stationary stereoscopic display device. The stereoscopic effect when a viewer views a stereoscopic photograph through the viewer, that is, the perspective stereoscopic effect, will be described using the example of three automobile stereoscopic photographs described in section 2.2. As a matter of course, the stereoscopic display device has a screen, and the viewer “sees through the screen” three cars in the stereoscopic image of the stereoscopic photograph. At this time, the viewer feels the sense of depth of the three vehicle arrays based on the inherent stereoscopic effect determined when the photograph is taken. When attention is paid only to the relative depth of each of the three vehicles, the stereoscopic effect at that time is almost the same as the stereoscopic effect that is felt in the case of the inherent stereoscopic effect.

  However, the inventor of the present invention noticed that not only that, but the viewer also felt a different three-dimensional feeling. Depending on the viewing environment, the three-dimensional effect is (a) “Scene I: all three cars appear to pop out closer to the screen” or (b) “Scene II: all three cars retract farther than the screen. Or (c) “Scene III: One or two of the three cars appear to be near the screen”. As described above, when the stereoscopic display device and the stereoscopic display software for a personal computer are used, a further perspective stereoscopic effect is added to the viewer in addition to the inherent stereoscopic effect. This stereoscopic effect is called “perspective stereoscopic effect” as described in section 2.1.

  As a prototype prototype viewer was developed, and it was found in prior experiments, the viewer sensed the entire 3D space seen from the 3D photo as a whole, and determined the location of the 3D space. It feels as a perspective based on the screen. The perspective stereoscopic effect of the viewer is that the perspective factor acts on the inherent perspective of the stereoscopic photograph, and what corresponds to the perspective factor is the function of the stereo photo viewer. A model formula for expressing such an image is shown in section 2.1 as a model formula of Formula 1.

  In the above-mentioned 3D viewing of a car, even if the viewer initially feels the stereoscopic effect of scene I, if the stereo photo viewer has a perspective filter function, the viewer can operate it and You can feel the three-dimensional effect of the scene or the three-dimensional effect of the scene III. The concrete image figure is shown in FIG. In the three drawings, the black frame means the screen, and the cube indicated by the dotted line means the three-dimensional space. On the left side of the figure, the three-dimensional space is placed in front of the screen, in the middle of the figure it is placed behind the screen, and on the right side of the figure it is placed near the screen, so it feels like that for viewers is there.

  Such a stereoscopic sensation varies depending on the situation, but is a sensation that clearly occurs when a stationary stereoscopic display device is used. That is, what is the “perspective” positional relationship with respect to the position of the stereoscopic display screen in which the entire stereoscopic space to be reproduced by the stereoscopic photograph is arranged. Depending on how the three-dimensional space is arranged, the three-dimensional effects of scene I, scene II, and scene III appear in the above-described vehicle arrangement. (This is an important issue related to the double image from the perspective of the viewer's perspective. This is the theme of this specification, and it is described in Chapters 5 to 8. ).

  The inventor of the present invention strongly felt this perspective stereoscopic effect when viewing a stereoscopic photograph using a stereoscopic display device and a trial prototype viewer. The perspective stereoscopic effect depends on the characteristics of a stereo photo viewer that reproduces a stereoscopic image from a stereoscopic photograph on a stereoscopic display. The visibility of the 3D image is subjective, but the 3D image is slightly behind the screen (slightly far from the screen) rather than popping out in front of the screen (visible near the screen). It is generally said that it is easier for viewers to see.

  In addition, there is a difference between individuals when viewing a stereoscopic photograph using a classic peeping-type stereoscopic photograph browsing device using a lens or a mirror or a stereoscopic HMD (head-mounted ultra-small stereoscopic display) The stereoscopic effect is weak, and only the intrinsic stereoscopic effect of a stereoscopic photograph can be felt strongly at first glance. It seems that the size of the viewing angle (a term used in the visual field rather than the display field) is related to the sense of perspective stereoscopic effect.

2.4 Attention to perspective factor When a stereoscopic photograph is reproduced as a stereoscopic image on a stereoscopic display, viewers who view the stereoscopic image are affected by the perspective factor on the inherent stereoscopic effect shown in Section 2.1. I feel the perspective of the perspective. However, when reproducing a three-dimensional image on a three-dimensional display, it has been usual to discuss the three-dimensional effect without fully grasping this perspective factor. From the engineering standpoint of designing a stereo photo viewer, the perspective factor can be discussed more clearly if the perspective factor is considered based on perspective perspective. Therefore, in the following chapters, we will focus on the perspective factor, and show that it is possible to design a stereo photo viewer that zooms stereoscopic images by handling it appropriately.

Chapter 3 Basic Functions of General 2DCG In order to develop a stereo photo viewer, 2DCG image processing technology is used, so the functions of general 2DCG will be described. In general, application software related to 2DCG is often developed by the following combinations.
(B) Combination of OpenCV-API and C / C ++ language
(B) Combination of Java 2D-API and Java language,
(C) Combination of HTML5 language and JavaScript language In the above (a) and (b), the C / C ++ language and the Java language are the parent languages of the respective APIs. In (c), the image processing function of the HTML5 language and the JavaScript language are used. However, for the most part, the latter corresponds to the parent language. Note that the SPV based on the stereo photo viewer developed this time is developed by a combination of Java 2D-API and Java language. This is described in Chapter 11.

Since these OpenCV-API and Java2D-API are basic software of 2DCG, they have common functions. The common functions necessary for developing SPV are as follows.
(A) A function for displaying an image (referred to as a texture) (b) A coordinate conversion function (affine transformation function) for rotating, translating and enlarging / reducing an image
From the standpoint of software development, user interfaces and zooming functions in SPV, automatic browsing such as photo frames, etc. are created using the functions of the parent language.

Chapter 4 Stereoscopicization in Stereo Photo Viewer Section 4.1 Relationship between Images and Screens First, in 2DCG, a set of stereoscopic photographs (left eye images and right eye images) called stereo pairs are displayed on a stereoscopic display screen. Let's briefly describe the concepts drawn in. FIG. 2 depicts a conceptual image of left and right images handled in a stereo photo viewer from above. Two thin flat plates are placed on top of each other, and two left and right images (stereo pairs) taken by two cameras are pasted to the two flat plates by the texture function of 2DCG. It shall be.

  In FIG. 2, the two flat plates are drawn in front and back for easy understanding, but conceptually the two flat plates are both the same thickness and overlapped along the x-axis. . An image for the left eye is pasted on the front plate, and an image for the right eye is pasted on the rear plate. However, an image taken with the left camera is used as an image for the left eye, and an image taken with the right camera is used as an image for the right eye. The center of the flat plate of the left-eye image is fixedly arranged at the origin of the xy plane coordinate system fixed to the stereo photo viewer. However, the flat plate of the right-eye image moves up and down, left and right, and rotates around the origin of the coordinate system around plus or minus. The plane coordinate system has a positive x-axis toward the right and a positive y-axis upward. The left and right images pasted on the two flat plates are superimposed and projected in parallel on the stereoscopic display screen.

  The viewer stereoscopically views the projected image. If the 3D display device is glasses-type, wear 3D glasses, and if it is naked-eye type, leave the naked eye and look at the left and right images projected on the projected display screen to create a 3D image. To fuse. At that time, the viewer looks at an arbitrary portion of the stereoscopic photograph (image) while zooming by a zooming operation using a mouse.

Section 4.2 Projecting Images to Screen The two left and right images that make up the stereo pair are projected and projected onto the display screen. In FIG. 2, a specific portion (a position indicated by a black dot in the drawing) of the left eye image (left image) as a stereo pair and a corresponding portion (a white circle in the drawing) of the right eye image (right image). A position indicated by a dot) is projected on the display screen as a round dot. In the left and right images, there are an infinite number of points corresponding to the left and right sides of the scene. Here, a set of them is taken up. Let d be the distance between the circle points of the left and right images attached to the flat plate (this is referred to as the “deviation value” in this specification). The interval between the circle points of the left and right images projected on the display screen is D. This distance D creates a perspective stereoscopic effect, which will be described later.

  By the way, when the magnification of the image projected on the screen is m, (A) the magnification m is proportional to the interval D between the round dots of the left and right images projected on the screen. If the magnification m is constant, the shift value d between the left and right images pasted on the flat plate is projected at a distance D proportional to the size of the display screen. That is, (B) the displacement value d of the image and the interval D between the round dots of the left and right images projected on the screen are in a proportional relationship with a constant magnification m at that time. The above two relationships (A) and (B) are important relationships used in Chapter 6.

Chapter 5 Problems in Fusion The basic structure of a stereo photo viewer that can be zoomed by operating the mouse is as described in chapter 4. However, with this configuration alone, the viewer senses from the inherent stereoscopic effect of stereoscopic photographs. The problem that the stereo photo viewer reproduces the perspective stereoscopic effect cannot be solved. This is because the perspective effect that the viewer feels is introduced by introducing the perspective factor action defined in Chapter 2 into the stereo photo viewer so that a double image does not occur even when zooming and increasing the screen magnification. It is necessary to give the perspective factor some kind of function that adjusts. This is a problem that leads to an expression called “Fusion Expression” in Chapter 6, but before that, the problems and solutions are pointed out below.

Section 5.1 Problem 1 (Generation of double images by zooming in)
As you zoom in on a 3D photo using a stereo photo viewer, the misalignment between the left and right images projected on the screen will increase (this is the same as magnifying glass showing fine details). When the magnification is a certain value or more, the deviation between the left and right images exceeds the limit, a double image appears on the screen, and the stereoscopic image does not fuse for the viewer. Since the three-dimensional sense is a subjective sense, strictly speaking, the limit value of the deviation between the left and right images varies from person to person, but the difference is small. In such a case, the perspective factor function to be introduced into the stereo photo viewer has a function to prevent the generation of a double image in the perspective stereoscopic effect by suppressing the expansion of the shift in the process of expanding the magnification of the screen. It is necessary to make it.

Section 5.2 Problem 2 (Strong pop-out effect)
In the past, there were many things that aimed for the pop-out effect when talking about stereoscopic photography, but in today's era when stereoscopic photography is viewed using a stereoscopic display device, there is a sense of depth that is retracted behind the screen. Eye-friendly 3D photography is preferred. However, when a stereoscopic image is reproduced on a stereoscopic display using a stereo photo viewer, the entire stereoscopic image may feel as if it protrudes from the screen even if the lens interval between the two cameras is appropriate. This is intense as a stereoscopic sensation, and it can be said that the eye is too burdensome for the viewer. When this becomes extreme, the stereoscopic image becomes a double image. In such a case, the perspective factor function to be introduced into the stereo photo viewer needs to have a function of suppressing the pop-out feeling over the entire stereoscopic image so that the perspective stereoscopic effect becomes an appropriate depth sensation.

Section 5.3 Problem 3 (stereoscopic image that moves away while zooming in)
During actual zooming, if the zoom-in operation is performed, the stereoscopic image that should be approached gradually moves away gradually, or if the zoom-out operation is performed, the stereoscopic image that should be moved away gradually approaches, There is a phenomenon that can be felt. This phenomenon seems to be affected by the scene (content) of a stereoscopic photograph, and may not be confirmed by a viewer without careful observation. However, such a movement of a three-dimensional image is unnatural from the viewpoint of people. This is caused by the fact that the deviation (value d described in section 4.2) on the screen (interval D described in section 4.2) in the left and right images pasted on the flat plate is enlarged when zooming in. In the case of zooming out, it is to reduce. If this is not resolved, stereoscopic zooming will be strange. It is necessary to give the perspective factor a function to prevent this.

  Japanese Patent Laid-Open No. 10-513019 discloses a method of viewing a stereoscopic image by shifting the positions of the left and right images to change the degree of mismatch and moving the perceived relative position of the object. It is disclosed. However, this Patent Document 1 does not disclose how to shift.

Japanese National Patent Publication No. 10-513019

Takashi Kawai, Miwa Tanaka, "Introduction to Next Generation Media Creator 1-Solid Expression", Cut Systems (2003) Shigeru Nakayama, "Java2-Introduction to Graphics Programming-", Gihodo Publishing (1999) By Barbad Mendibl, translated by B Sprout, Inc .: “3D video production”, Bone Digital (2010)

  When zooming in with a conventional stereoscopic photograph browsing device, the fused stereoscopic image moves away from the viewer in the process of enlarging a fine area of the image of the stereoscopic display device, and eventually becomes in a state where it cannot be fused. Further, when zooming out, it is felt that the stereoscopic image comes to the front.

  Therefore, in the present invention, even if zooming is performed, the fusion is not solved, and the viewer can perceive the natural fusion of the stereoscopic image with respect to the depth position of the fused stereoscopic image. SPV based on the concept of a simple stereo photo viewer.

Chapter 6 Proposal of fusion formula So far, the perspective stereoscopic feeling felt by the viewer has been explained only in a general way using only the conceptual word of perspective factor. Therefore, in this chapter, it is shown that "It is possible to place a stereoscopic image at the target position when the meaning of the perspective factor is defined as a mathematical formula and that formula is used as the perspective factor." I will decide. This equation is referred to as “fusion equation” in this specification, and this equation allows the generation of the double image (Section 5.1), the strong pop-up effect (Section 5.2), and the stereoscopic image shown in Chapter 5. It is possible to control the reverse movement (section 5.3). First, the deviation value, which is the first stage of the fusion formula, will be described.

Section 6.1 Analysis of Fusion (1) Position of 3D image to be fused First, “When the distance D between the black and white circles projected on the screen increases as a positive value, the 3D image becomes a positive image at the back of the screen. A phenomenon is known that when the distance D moves backward and the absolute value increases as the interval D is a negative value, the stereoscopic image moves forward in the negative direction in front of the screen (the coordinate system will be described later). Based on this phenomenon, the position of the stereoscopic image visually recognized by the viewer will be qualitatively explained.

  FIG. 3 is a view of FIG. 2 as viewed from the top in the negative direction of the y-axis. In the scenes shown in the left image and the right image, two points corresponding to each other in each scene are doubled and projected on the display screen. In this way, in the left and right images, there are an infinite number of points corresponding to the left and right sides of the scene. Here, only one of them is taken up. The left and right images are affixed to the respective flat plates, and the value of the shift between the two points in the corresponding image is d.

  In FIG. 3, with respect to the projection on the screen, the projected point by the point of the left image is indicated by a black circle point, and the projected point by the right image is indicated by a white circle point. Let D be the distance between the circle points on the screen. The viewer views the white circle point of the projected point on the display screen with the right eye and the black circle point with the left eye, and the viewer's brain fuses the three-dimensional image based on the interval D between the two round points.

  The determination of the deviation value d and the interval D is performed in a one-dimensional coordinate system (x-axis) described below. As shown in FIG. 3 (1), the black circle point of the left image is fixed as the origin (starting point), and the distance from the corresponding white circle point of the right image is set as a deviation value d between the left and right images. If the white circle point of the image moves to the right side, the movement is positive, and the deviation value d is a positive value. If the white dot moves to the left, the deviation value d is negative, and the deviation value d is negative. Accordingly, the deviation value d between the left and right images takes positive, zero, and negative values. Then, the deviation value d of the image is projected on the screen as an interval D between the two round points. As described in section 4.2, since D and d are in a proportional relationship, the one-dimensional coordinate system (x-axis) of the interval D on the screen has the projected black dot as the origin, and the positive and negative directions are left and right. This is the same as the x-axis coordinate of the image shift value d.

  The viewer feels the perspective of whether the stereoscopic image looks close or far, depending on the value of the distance D between the black and white circles projected on the screen. The position at which the three-dimensional image is fused can be expressed by the following expression from the geometric proportional relationship using the amount shown in FIG. 4 (Non-patent Document 1). The position of the stereoscopic image is determined by setting the position u on the display screen as a reference position, with the direction toward the display device being positive and the direction toward the viewer being negative.

Here, the value e is the pupil distance of the viewer (generally 6.5 cm for humans), h is the distance between the viewer and the display screen, and D is the distance between the black and white circle points projected on the screen. The expression for the position u is a monotonically increasing function with respect to the distance D. This number 2 can be easily derived by establishing a proportional expression in the triangle relating to the stereoscopic view of FIG.

  It can be seen from Equation 2 that the position u of the stereoscopic image that the viewer fuses can be obtained quantitatively from the interval D if the viewer's distance h and pupil interval e are fixed. If the interval D increases from 0 to a positive value, u increases, and the stereoscopic image becomes deeper from the viewer side. If the distance D decreases from a positive value to 0, u decreases and the stereoscopic image approaches the viewer side. Further, if the absolute value of the interval D increases from 0 to a negative value, u increases its absolute value in the negative direction, and the stereoscopic image approaches the viewer side. If the distance D decreases from the negative value to the zero direction, u decreases the absolute value from the negative value to the zero direction, and the stereoscopic image is deepened from the viewer side.

(2) Size of stereoscopic image to be fused The size of the stereoscopic image to be visually recognized can be obtained by a geometric procedure in the same manner as Equation 2. In FIG. 5, black circles, white circles, black circles, and white circles are arranged from the left on the screen, but the image portion between two black circle points and the image portion between two white circle points are for the left eye and the right eye, respectively. And r is the size of the partial real image on the screen. The real image of this part is visually recognized as a stereoscopic image by the gap D of the deviation. u is the depth of the stereoscopic image. When the size of the stereoscopic image to be visually recognized is v and a simple geometric proportional relationship is calculated, v is expressed as follows.

  The depth and size of the stereoscopic image visually recognized from the two right and left stereoscopic photographs can be determined from Equations 2 and 3.

  In Section 7.4, the effect of the gap D on the stereoscopic image will be discussed by simulating the position and size at which the stereoscopic image is fused using Equations 2 and 3. Equations 2 and 3 are derived from the drawing and the proportional expression. However, there are individual differences when people view stereoscopically, so in fact it is necessary to consider the movement of the eyeball-convergence-etc. It is. The application of Equations 2 and 3 shown in this paper should be considered as a conceptual one.

(3) Relationship between deviation value and perspective sensation As shown in FIG. 3 (1), the image deviation value d is set to a positive value, and the right image is moved in the x-axis positive direction so as to increase the value. Since the white circle point projected on the screen similarly moves in the positive x-axis direction, the distance D between the two round points on the screen increases as a positive value. Therefore, the position u of the stereoscopic image to be fused increases as a positive value from Equation 2, so that the viewer feels that the stereoscopic image is retracted. Further, as shown in FIG. 3B, when the image shift value d is a negative value and the right image is moved in the x-axis negative direction so as to increase the absolute value, the image is projected on the screen. Since the white circle point similarly moves in the negative direction of the x-axis, the distance D between the two circle points on the screen is negative and increases its absolute value. Therefore, since the absolute value of the position u of the stereoscopic image to be fused is increased from Equation 2 as a negative value, the viewer feels that the stereoscopic image approaches. Also, as shown in FIG. 3 (3), when the black and white dots of the left and right images overlap at a single point with no deviation, the image deviation value d is 0, so the interval between the two round points on the screen. The value of D is 0. Therefore, since the position u of the stereoscopic image to be fused becomes zero from Equation 2, the viewer feels that the stereoscopic image exists at the position on the screen. This is confirmed by experiments and is a well-known fact (Non-Patent Document 1).
As shown here, the stereoscopic image moves backward and forward by increasing / decreasing the image shift, so this phenomenon can be used to construct a mathematical expression that controls the movement of the stereoscopic image. If possible, it becomes a fusion formula, which is a mathematical formula of the perspective factor.

Section 6.2 Derivation of Fusion Equation Based on the above, the quantitative ratio between “magnification m of the image projected on the display screen” and “deviation value d between the two left and right images stuck on the flat plate” The relationship will be discussed in four stages. Then, the fusion formula that is the subject of this specification will be derived.

(1) First stage (inversely proportional relationship between d and m)
First, in order to deal with the problem 3 in Section 5.3, an attempt is made to express the displacement value d of the left and right images pasted on the flat plate with a quantitative expression using the magnification m of the projected image. As described in the conclusions (A) and (B) of Section 4.2, when zooming in and the magnification m becomes n times, the gap D of the deviation becomes a magnification of n times from the principle of parallel projection in 2DCG. . The interval D between the round dots projected on the screen is enlarged n times in proportion to the deviation value d under the magnification m at that time. As a result, as described in section 6.1, while zooming in, the viewer feels as if the stereoscopic image originally moves forward, as if moving backward. In order to prevent this, first, consider stopping the stereoscopic image at a specific position. For this purpose, it is necessary to keep the distance D between the round dots on the screen shown in FIG. 3 constant even if the magnification m of the image changes due to zooming in. Also, when zooming out, it is necessary to keep the interval D between the round dots constant.

The distance D between the black circle point and the white circle point projected on the screen is proportional to the deviation value d when the magnification m is constant, and the sign of positive and negative is also the same. In order to make D constant regardless of the magnification m, d and m may be in an inversely proportional relationship. Let k ₁ be a constant value and let it be an inversely proportional constant. If the proportionality constant is k ₂ , it becomes as follows.

When the above equation is rewritten with k ₁ / k ₂ as a, the following inverse proportional equation for d and m is obtained.

D becomes a constant value regardless of the magnification m as follows.

  In addition, although the position of the black circle point of the left image in FIG. 3 is assumed to be fixed, the reason that “a” is a positive value in this case is to fuse the stereoscopic image in the back of the screen. If the value is negative, the entire stereoscopic image pops out of the screen and is generally difficult to see. Conversely, when the position of the white circle point in the right image is fixed, a is a negative value.

(2) Second stage (forward / backward factor)
Since the distance D between the round dots from Expression 6 is kept constant regardless of the magnification m of the screen, the stereoscopic image does not move backward or forward by zooming. Even in such a state, the viewer does not feel much unnatural movement of the stereoscopic image on the entire screen. However, we will make another improvement. If the three-dimensional image approaches the viewer in the case of zoom-in and moves away in the case of zoom-out, the viewer should feel as a more natural movement. Therefore, the following “forward / backward factor” is defined.

This factor f is a function that takes a value of 0 when the screen magnification m is ∞ (infinity) and ₁ when the magnification m is m ₁ . m ₁ should be called “reduction limit magnification”, and this magnification m ₁ will be described in Chapter 7. The exponent n of the power is for adjusting the forward / backward movement of the stereoscopic image and takes a real value of 0 ≦ n ≦ 2. This index n is also described in Chapter 7. When zooming out to the limit magnification that can be browsed, the stereoscopic image gradually becomes smaller, and there is a magnification that makes the entire stereoscopic image smaller than the screen size. This is the limit magnification for zooming out that can be browsed. The magnification m and the reduction limit magnification m ₁ have the following relationship.

  The forward / backward factor acts on the constant a in the equation as a correction factor. This effect will be described later. Using this factor, Equation 5 is corrected as follows.

  This is called an “ideal deviation formula”. Further, by substituting Equation 9 into Equation 4 for the interval D, the following equation regarding the interval D is obtained.

From Equation 10, it can be seen that the interval D when the index n is 1 is inversely proportional to the magnification m (proportional to the inverse of the magnification m). Using Equation 10, in the case of a zoom-in operation (when m increases from m ₁ to ∞), the distance D between the projected round points goes from a positive value to 0, and the stereoscopic image advances toward the viewer Come on. Further, in the case of a zoom-out operation (when m decreases from ∞ to m ₁ ), the interval D goes from 0 to a positive value, and the stereoscopic image moves backward from the viewer. By introducing an advancing / retreating factor and multiplying it by the constant a in Equation 5, it is possible to control the advancing and retreating of the entire stereoscopic image by zooming so that it becomes a natural movement. The index n is an artificial parameter introduced to make the movement of the stereoscopic image non-linear with respect to the reciprocal of the magnification m. The effect will be described in Chapter 7.

  At the end of this stage, it will be shown that the constant a in the ideal deviation equation of Equation 9 can be a parameter for adjusting the perspective of a stereoscopic image. From Expressions 2 and 10, when the magnification m of the image is constant, for the depth position u of the stereoscopic image, the position u is a monotonically increasing function with respect to the interval D (Equation 2), and the interval D is proportional to the constant a ( 10). Since the constant a is based on an ideal deviation equation (Equation 9), the stereoscopic image position u is a monotonically increasing function with respect to the constant a in the ideal deviation equation. From this, it can be seen that the depth of the stereoscopic image increases as the value of the constant a in the ideal deviation equation increases, and the depth of the stereoscopic image decreases as the value of the constant a decreases. The value of the constant a is determined from a visual experiment, which will be described later.

(3) Third stage (ideal fusion type)
Using the ideal deviation equation (Equation 9), the following equation is further determined.

Specifically, it is the following formula.

  This equation is expanded by adding a constant b called “double image prevention constant” to the ideal deviation equation, and this equation will be called “ideal fusion equation”. (This formula is a function that fuses a stereoscopic image that prevents the generation of a double image and obtains an optimal perspective stereoscopic effect for the viewer. Therefore, it was named “Fusion Formula”) . This fusion formula is a mathematical expression of the conceptual perspective factor introduced in Chapter 2.

  The double image prevention constant b is a parameter related to prevention of generation of a double image. The constant b is the value of the deviation between the left image flat plate (position is fixed) and the right image flat plate itself when the value d of the ideal deviation formula of Equation 9 is zero. . If the value of the constant b is appropriately determined, stereoscopic viewing can be achieved without generating a double image even at the zoom-in limit, but the value is determined from a visual experiment. This constant b is related to the lens interval between the two cameras when an image is taken. The value d calculated from the ideal deviation equation is a deviation value in the left and right images corresponding to the distance D between the two round points on the screen that can be visually confirmed by the viewer, but the constant b is the right and left images. Is the absolute deviation value of the right image flat plate itself with respect to the left image flat plate (position is fixed) when the deviation value d between the round points in FIG. Therefore, the fusion equation t including the constant b is used as an equation for coordinate transformation described in Chapter 10. (Note that the lens interval of the photographing camera is an attribute that is absorbed when the constant b is determined by visual experiment, and is not particularly required).

  If the right image flat plate is arranged according to the optimal shift value obtained from the fusion formula, the generation of a double image is suppressed, and a stereoscopic image that allows the viewer to feel the optimal perspective stereoscopic effect is fused. It becomes possible to do.

The following two methods are conceivable for determining the ideal fusion constants a and b derived in the third stage.
(A) Individual determination method: The magnification m of the image projected on the screen is brought to an extremely large value (zoomed in) (m is a positive value → ∞), and the projected left and right images (black dots and white dots) The value of the constant b is determined by visual experiment so that the points overlap without deviation. Thereafter, the constant a is zoomed out (m is a positive value → 0), and is determined by visual experiment so that the depth of the stereoscopic image is the best. This determination method is described in detail in Chapter 8.
(B) Cooperation Determination: For the magnification m of the fusion expression defines a certain magnification m _a, determines the variable t _a as stereoscopic image is generated best by visual experiments. Another magnification m _b is determined again, and a variable t _{b at} which a best stereoscopic image is similarly generated is determined by a similar visual experiment. Then, a binary simultaneous equation with the constants a and b of the fusion equation as variables is established and the equation is solved. However, for this determination method, if the magnification m _a and the magnification m _b are arbitrary magnifications, even if the best stereoscopic image can be confirmed at those magnifications, the value of the constant b that does not generate a double image from the combination is obtained. Whether the value of the constant a that produces an optimal perspective stereoscopic image is obtained is a trial and error.

However, the above two methods both have problems. With regard to the individual determination method of (A), if the magnification m is increased as much as possible (m is a positive value → ∞), the magnification becomes theoretically infinite beyond the resolution limit of the image. Fusion becomes difficult. With regard to the linkage determination method of (B), even if the equation solving method is automatically processed inside the stereo photo viewer, the collaborative work of the viewer's experimental operation and the solution procedure becomes troublesome. This is because the procedure of finding the solutions of simultaneous equations after performing visual experiments for determining the two constants a and b, respectively, imposes restrictions on the viewer, and is convenient for stereoscopic viewing. It will not be a tool. In this case, it becomes difficult to give a meaning to the role of the constants a and b in the fusion equation. If the magnification m _a and the magnification m _b are defined as an enlargement limit magnification m ₀ and a reduction limit magnification m _1, which will be described later, they become the same as the individual determination method, and the meaning of the linkage determination method is lost.

(4) Fourth stage (practical fusion type)
In order to avoid the problems described in the third stage, the fusion formula that allows the viewer to easily determine the constants a and b will be improved. This fusion equation is an improvement of the ideal fusion equation, but if it is used, the individual determination method can be easily applied to the determination of the two constants a and b. The reason for this is that (a) the problem of infinite image magnification can be avoided, (b) the constants a and b can be determined independently in a visual experiment, and (c) the role of the constants a and b in the determination. The meaning of is also clarified. In addition, there is an advantage that the viewer can reduce the work to one pattern without any experimental restrictions. Therefore, in order to be able to apply the individual determination method, an ideal fusion formula will be improved.

First, let's introduce a magnification m ₀ that maximizes the enlargement of the image on the screen. This is called “enlargement limit magnification” and is a magnification determined by the resolution of the image. This magnification m ₀ will be described in Chapter 7. The enlargement limit magnification m ₀ and the magnification m have the following relationship.

The reason is that if the magnification m exceeds the enlargement limit magnification m ₀ , it exceeds the resolution limit of the image, so that the generated stereoscopic image has no meaning. The enlargement limit magnification m ₀ depends on the resolution characteristics of the stereoscopic camera.

The enlargement limit magnification m ₀ is determined as a value about 10 to 20 times the reduction limit magnification m ₁ in visual experiments. Using this enlargement limit magnification m ₀ , let's transform the ideal fusion equation (Equation 9) into a fusion equation that is easy to handle experimentally.

In Equation 5 determined in the first stage, the reciprocal 1 / m of the magnification m is replaced with (1 / m−1 / m ₀ ) as follows. This replacement is made so that the magnification m is the enlargement limit magnification m ₀ , the value d of the deviation equation is 0, and the situation shown in FIG. 3 (3) in Section 6.1 is created. As a result, the black and white circles overlap on the screen.

  Accordingly, the forward / backward factor f (Equation 7) is changed as follows.

The modified factor f, when the magnification m is m ₀ 0, the magnification m is in the range is a function that takes a value of 1 (m is m ₁ ≦ m ≦ m ₀ when at m _1, f is 0 ≦ f ≦ 1). m ₁ is the reduction limit magnification shown in Section 6.2 (2). The exponent n of the power is for adjusting the forward / backward movement of the stereoscopic image and takes a real value of 0 ≦ n ≦ 2. The effect of this index will be discussed in Chapter 7.

  Then, the forward / backward factor f of Formula 15 is applied to the constant a of Formula 14 as a correction factor as follows.

  This formula is referred to as a “practical misalignment formula”, but hereinafter, this formula is simply referred to as the misalignment formula.

  Then, a new fusion equation is obtained as follows by the same method as that obtained in the third stage.

The right-hand side of this equation is that c, b, and k are constants.
c × (1 / m−k) ^{n + 1} + b.

The newly defined fusion equation (17) approaches the ideal fusion equation of equation (12) as a function when the magnification limit magnification m ₀ is considerably large. Equation 17 is hereinafter referred to as “practical fusion type”. Hereinafter, simply speaking, the fusion type refers to this practical fusion type.

  When a practical fusion equation is used, the following equation can be obtained by using the upper equation and the equation 15 of the equations 4 and 16 as the distance D between the two round points on the screen.

This equation is obtained when α and β are constants.
α × (1−β × m) ^{n + 1} / m ⁿ .

  The middle expression of Equation 18 regarding the interval D is obtained by multiplying the constant a by the forward / backward factor as a correction factor. The interval D obtained by Equation 18 is a monotonically decreasing function with respect to the magnification m. In addition, when the index n is 0, it is a linear expression regarding the magnification m.

Chapter 7 Features of Practical Fusion Equations Section 7.1 Role of Fusion Equations The role of practical fusion equations is the ideal described in the first to third stages of section 6.2. The role in the fusion formula is basically the same. The role is summarized below. The difference between the practical fusion type and the ideal fusion type is the introduction of the enlargement limit magnification m ₀ into the former. If the reciprocal (1 / m ₀ ) of the enlargement limit magnification m ₀ is set to ₀ in a practical fusion equation, the fusion equation becomes an ideal fusion equation.

(1) The role of preventing the generation of the double image is played by the fusion constant b. The role of preventing the generation of the double image is played by the practical fusion constant b. The value of the constant b is determined by visual experiment so that a double image is not generated at the enlargement limit magnification m ₀ . Therefore, this constant is called a “double image prevention constant”. As a result, Problem 1 in Chapter 5 can be solved. The constant b as a parameter is determined for each stereoscopic photograph, and the determination method will be described in section 8.2 (1).

(2) The perspective factor for adjusting the position of the stereoscopic image is assumed by the fusion constant a. The practical fusion constant a of Equation 17 is a parameter for adjusting the perspective of the stereoscopic image. From Expression 2 and Expression 18, when the magnification m is constant, for the depth position u of the stereoscopic image, the position u is a monotonically increasing function with respect to the interval D (Equation 2), and the interval D is a monotonically increasing function with respect to the constant a ( (Equation 18). Since the constant a is based on a practical deviation equation (Equation 16), the position u of the stereoscopic image becomes a monotonically increasing function with respect to the constant a in the practical deviation equation. From this, the value of the position u increases as the value of the constant a in the practical deviation equation increases, and the depth of the stereoscopic image increases accordingly. Further, as the value of the constant a decreases, the value of the position u decreases, and the depth of the stereoscopic image also decreases accordingly. The actual role of the perspective factor described in Chapter 2 is played by the constant a. The constant a is determined as a parameter by visual experiment so that the depth of the stereoscopic image is best at the reduction limit magnification m ₁ . Therefore, this constant is called “perspective control constant”. This can solve Problem 2 in Chapter 5. The constant “a” as a parameter is determined for each stereoscopic photograph, and the determination method is described in section 8.2 (2).

(3) The movement of the stereoscopic image visually recognized by the viewer is based on the interval D. The best rearrangement regarding the shift between the left and right images is carried out by the fusion formula t, but the movement of the stereoscopic image visually recognized by the viewer is The interval D of the number 18 linked to the fusion equation t is carried. Therefore, when the interval D is used, it is possible to simulate the depth of the stereoscopic image visually recognized by the viewer and the magnification of the stereoscopic image. This is discussed in detail in section 7.4.

(4) The forward / backward adjustment of the stereoscopic image is performed by the exponent n of the power of the forward / backward factor f. It is the forward / backward factor that advances / retreats the stereoscopic image. The index n is a constant taking a real number between 0 and 2 (n is an empirically introduced parameter, so it may be 2 or more, but the upper limit is 2 from a practical viewpoint) . The effect on the viewer of this factor using a practical fusion formula depends on its exponent n. As the value of the index n increases from 0 to 2, the effect of forward / reverse becomes stronger as m decreases, as it is inversely proportional to the magnification m. However, there are individual differences in stereoscopic vision, and some people may feel that the forward / backward effect is strong or weak. Since the index n is an arbitrary parameter, it is preferable to determine an average value that absorbs individual differences through prior visual experiments by a stereo photo viewer developer. The introduction of the index n can solve the problem 3 in Chapter 5.

(5) The enlargement limit magnification m ₀ and the reduction limit magnification m ₁ can be regarded as constants of the stereo photo viewer. The enlargement limit magnification m ₀ and the reduction limit magnification m ₁ can also be regarded as constants of the stereo photo viewer. The former is the magnification at the limit of zooming in, and is a value related to the resolution of the camera, but the value is the content of the taken stereoscopic photograph (for example, night landscape, foggy landscape, summer coastal landscape, person Etc.). Therefore, it is also conceivable to set the enlargement limit magnification m ₀ for each stereoscopic photograph. However, according to prior visual experiments, it is often safe to use the same value among commonly used digital cameras. Therefore, this value may be a value determined by the developer of the stereo photo viewer. The latter is the magnification at the limit of zoom-out. By zooming out, the stereoscopic image gradually becomes smaller, and there is a position where the entire stereoscopic image becomes smaller than the screen size. This is the reduction limit magnification m ₁ , and the zoom-out limit magnification that can be viewed and I can say that. In order to specify the magnification, the reference magnification must be determined. However, in the stereo viewer, the reduction limit magnification m ₁ is set as the reference magnification, and the magnification is set to 1. However, the minimum limit magnification is referred to as m ₁ for convenience of explanation.

The range of magnification that can be zoomed is a magnification between the reduction limit magnification m ₁ and the enlargement limit magnification m ₀ , and according to prior visual experiments, when using a commonly used digital camera, The enlargement limit magnification m ₀ is approximately 10 to 20 times the reduction limit magnification m ₁ .

Section 7.2 Graphical representation of the fusion formula Let's show the state of the practical fusion formula of Equation 17 in a graph. When the magnification m of the image on the screen takes the value of the enlargement limit magnification m ₀ , the constant b is determined from a visual experiment, and the value at that time is b _{0 and} the value of _the fusion equation t is t ₀ .
t ₀ = b ₀ .

When the magnification m takes the value of the reduction limit magnification m ₁ , the constant a is determined from a visual experiment, and when the value at that time is a ₁ and the value of the fusion equation t is t ₁ ,
t ₁ = a ₁ × (1 / m ₁ −1 / m ₀ ) + b ₀ .

FIG. 6 is a graph showing an example of a fusion type using the enlargement limit magnification m ₀ , the reduction limit magnification m ₁ , and the above t ₀ and t ₁ . The fusion equation having the exponent n of the forward / backward factor is a function passing through the point (m ₀ , t ₀ ) and the point (m ₁ , t ₁ ). Further, the fusion equation when the exponent n of the forward / backward factor is 0 is a hyperbolic function.

  It should be noted that the range that can be taken by the magnification m of the image projected on the screen (the range in which zooming is possible) has the following relationship from Equation 8 and Equation 13.

Section 7.3 Working of the fusion formula Let us examine the practical operation of the fusion formula when the double image prevention constant b is zero. That is, the operation of the lower equation of Equation 16 will be described as the value d of the practical deviation equation.

(1) Value d> 0 This is the case where the constant a is a positive value. The three-dimensional image generated by the round dots projected on the screen by the left and right images is fused behind the display screen as shown in FIG. When the magnification is increased by the zooming operation with the mouse (m is changed from m _{1 to} m ₀ ), the value d of the deviation formula approaches 0 from the positive value, and is accordingly projected from the left and right images attached to the flat plate. The distance D between the two round points approaches 0 from the positive value according to Equation 18. In that case, since the number 2 u that determines the position of the stereoscopic image decreases, the stereoscopic image feels closer to the viewer. Conversely, when the magnification is reduced (m is changed from m _{0 to} m ₁ ), the value d of the deviation equation increases, and the distance D between the double dots projected from the left and right images pasted on the flat plate increases. In that case, since the number 2 u that determines the position of the stereoscopic image increases, the stereoscopic image feels like retreating from the viewer.

(2) When the value d <0 The deviation value d in the fusion equation is a negative value when the constant a is a negative value. Up to now, it has been described that the constant a of the fusion formula adopts a positive value. However, when the constant a is a negative value, the following inconvenience occurs. The positions of the black and white circle points in the left and right images projected on the screen are opposite to those in Section 7.3 (1). In this case, as shown in FIG. It melts in the foreground and is hard to see for general viewers. The movement of the stereoscopic image by the zooming operation of the mouse is opposite to that in section 7.3 (1), and when the magnification is increased (m is changed from m _{1 to} m ₀ ), the shift value d is changed from a negative value to 0. The distance D between the two round points projected from the left and right images pasted on the flat plate approaches from 0 to 0 from the negative value. In that case, since the number 2 u that determines the position of the stereoscopic image approaches 0 from a negative value, the stereoscopic image feels like retreating from the viewer. Conversely, when the magnification is lowered (m is changed from m _{0 to} m ₁ ), the interval D becomes a negative value from 0. In that case, u in Equation 2 that determines the position of the stereoscopic image changes from 0 to a negative value, so that the stereoscopic image is felt toward the viewer. Such a stereoscopic image is inconsistent with human senses. Therefore, the constant a must be a positive value (however, the left image is fixed).

(3) In the case where the value d = 0, it is a case where both the round points (black circle point and white circle point) projected on the left and right images which are pasted on the flat plate overlap with each other and the magnification m is m ₀ and the deviation occurs. The value d of the above formula is zero, and the distance D between the two round points projected on the screen is also zero. In this case, the stereoscopic image is fused on the display screen as shown in FIG.

Section 7.4 Three-dimensional image simulation analysis (1) Depth and size analysis of a three-dimensional image Using the expressions considered so far, the depth position and the size of a three-dimensional image visually recognized by a viewer are obtained quantitatively. be able to. For the calculation of the depth position of the stereoscopic image, the gap D between the left and right images on the screen (the lower equation of Equation 18) and the position u of the stereoscopic image (Equation 2) are used.
For the size v of the three-dimensional image, the lower equations of Equations 3 and 18 and the three equations of Equation 2 are used.

However, since the stereo photo viewer performs zooming, it is necessary to consider the change in the magnification m of the actual image as the size v of the stereoscopic image. Therefore, how the size v of the three-dimensional image changes with respect to the change in the magnification m of the real image will be examined. When the magnification of the real image is m ₁ and the size (reference length) of the real image portion is r ₁ , when the magnification is m, the size of the real image portion is enlarged to the size r as follows. The

  For this reason, when the magnification of the real image is m, the three-dimensional image size v can be obtained by rewriting Equation 3 as follows.

If the magnification of the stereoscopic image with respect to the magnification of the actual image is b and the magnification reference is the magnification m ₁ of the actual image, and the size (reference length) of the actual image is r ₁ , then the magnification b of the stereoscopic image is As in the equation (Equation 22), the size v of the stereoscopic image obtained from Equation 21 may be divided by (r ₁ / m ₁ ).

The calculation parameters are: viewing distance h is 50.0 cm, pupil distance e is 6.5 cm, reference length r ₁ is 1.0 cm, reduction limit magnification m ₁ is 1.0, and enlargement limit magnification m ₀ is 10.0. did. The constant k ₂ × a in Equation 18 is 1.2 cm so that the gap D on the screen is 1 cm when the magnification is m ₁ when a 17-type stereoscopic display (Dimen-G170P) is used. . Further, the exponent n of the power of the forward / backward factor was set to four types of 0.0, 0.5, 1.0, and 2.0. The calculation results are shown in FIG. The solid line is a calculation graph of the magnification b of the stereoscopic image (when n is 0.0), and the broken line is a calculation graph of the depth u (n is 0.0, 0.5, 1.0 and 2,0) In the case of In addition to the simulation, a visual experiment was also performed while performing automatic zooming.

  It can be said that the calculated magnification b of the stereoscopic image roughly matches the magnification m of the actual image. A close examination reveals that when n is 0.0, the change increases monotonically to such a degree that non-linearity is slightly observed. However, as n approaches 1.0, the non-linearity becomes very weak and the magnification b of the stereoscopic image is increased. Is close to the actual image magnification m (when the actual image magnification is 1.0, the stereoscopic image magnification is slightly larger 1.20, but as the actual image magnification becomes 10.0, the stereoscopic image magnification b is a smooth monotonic increase to 10.0, and the same tendency is observed when n is 0.5, 1.0, and 2.0). Since the four calculated values are close to each other when viewed from the whole figure and the four solid lines are attached, FIG. 7 shows only the case where n is 0.0. According to the simulation, the calculated value of the magnification of the stereoscopic image is slightly larger than the magnification of 1.0 of the actual image (a stereoscopic image having a size close to the calculation is visually recognized in a visual experiment). However, since the magnification of the stereoscopic image changes smoothly with the progress of zooming, the viewer is particularly uncomfortable from visual experiments that the magnification is calculated slightly larger and the stereoscopic image is visually recognized at that size. I never felt.

  Next, as for the depth u, as shown in FIG. 7, it has a remarkable nonlinearity with respect to a change in the magnification m. When n is 0.0, the outline is linear, and the movement of the depth change of the stereoscopic image proceeds smoothly (however, there is a difficulty that the zoom speed becomes slower as the magnification becomes higher). As n approaches 2.0, the curve becomes larger and the non-linearity becomes remarkable. That is, when zooming in, the viewer feels that the stereoscopic image suddenly approaches while the magnification m is between 1.0 and 4.0, and then approaches gradually.

  By the way, the relationship between the magnification m of the stereoscopic image and its depth position u must be in an inversely proportional relationship such that if the magnification is doubled, the depth position is halved. It has been found from the detailed simulation analysis that the index n that almost satisfies such a relationship is a value near 0.7. As described above, this value can be used consistently for estimating the magnification of a stereoscopic image. Therefore, 0.7 is used as the index n of the forward / backward factor. However, since it is generally said that there are individual differences in the visual recognition of a stereoscopic image, an average index n value that promotes comfortable visual recognition may be obtained from the results of visual experiments by a large number of viewers. It is.

  For reference, the calculated magnification b and depth position u of the stereoscopic image are listed in Table 1. In the calculation, the index n is set to 0.2, 0.7, and 1.5. Capital letters M and L in the table are the magnification M of the reference actual image and the distance (object distance) L to the object in the actual image at that time. The object distance is a calculated virtual distance obtained from an inversely proportional relationship of M × L = a constant value. The magnification b of the stereoscopic image is based on the magnification of the stereoscopic image (10.0 in the calculation) corresponding to the magnification of the real image of 10 times, and the depth position u of the stereoscopic image is 1 times the magnification of the real image. In this case, the depth position of the stereoscopic image is converted to 1.0, and the value is used as a comparison reference. The calculation was performed so that the value of L × M was 10.0, and when the index n was 0.7, the value that u × b was about 10 was obtained. According to the calculation model (method) shown in this specification, it can be said that the result is almost satisfactory.

(2) Functionalization of index n of forward / backward factor As described with reference to FIG. 7, the depth position (stereoscopically) of the stereoscopic image visually recognized by the viewer is determined according to the method of determining exponent n of the forward / backward factor. The object distance (note that it is not the object distance estimated in section 7.4 (1)) is changed. In particular, when performing automatic zooming, the manner in which the viewer approaches and retracts the stereoscopic image is important. So, let's be able to express the index n as a function of the magnification m of the real image so that it can be handled easily. This function is not derived from theory, but is determined empirically.

7 shows that (a) when the value of n is small, when the zoom-in is performed while changing the magnification m of the real image from m ₁ to m ₀ , the depth position of the stereoscopic image is initially gentle. The viewer approaches the viewer and then approaches it quickly (the opposite is true for zooming out). (B) When the value of n is large, when zooming in while changing the magnification m of the real image from m ₁ to m ₀ , the depth position of the stereoscopic image quickly approaches the viewer at first, After that, it is visually recognized as approaching slowly (the opposite is true for zooming out).

  From this, the following formula can be introduced in order to change how the stereoscopic image approaches and retreats during zooming. The state of the formula is shown in FIG.

Increase function of index n in the case of (a)

Decrease function of index n in the case of (b)

p is a parameter for adjusting the shape of the function, and takes a real number in the range of m ₁ ≦ p ≦ m ₀ . As shown in FIG. 8, the index n in the case of (a) changes from 0 to 2 according to the change in the magnification m of the actual image, and the expression n in the case of (b) indicates that the index n is an actual image. It changes from 2 to 0 according to the change of the magnification m.

  Using these formulas for changing the index n, unlike the case where n is fixed, it is possible to forcibly change how the stereoscopic image is approached or retracted during the automatic zooming process (parameter p is set to p). It can be fine-tuned by changing it). Prior visual experiments confirmed the effect. If such a formula is implemented in a stereo photo viewer, for example, it is possible to produce an effect that a stereoscopic image suddenly approaches in front of the viewer's eyes. Note that the above formula is an example, and a function other than the linear function used here (such as a sine function, a cosine function, or a quadratic function) may be used.

  Therefore, the zoomable stereo photo viewer of the present invention has the following characteristics.

A stereoscopic display device that presents a left-eye image and a right-eye image to the viewer's left eye and right eye, respectively, and supplies the left-eye image and right-eye image to the stereoscopic display device. A stereo photo viewer that provides a viewer with a left-eye image and a right-eye image for stereoscopic image display,
Whether the distance D between the same objects of the left-eye image and the right-eye image in the image displayed on the stereoscopic display device is constant according to an increase (or decrease) in magnification due to zoom-in (or zoom-out) By monotonously decreasing (or increasing), the position of the three-dimensional image resulting from the fusion of the left-eye image and the right-eye image during zoom-in (or zoom-out) does not move or is in front of the viewer (or Zooming to move to the back)
The distance D is the magnification m, the the alpha and beta is a constant, for 0 ≦ n becomes n, by changing in proportion to D to α × (1-β × m ) n + 1 / m n ,
Zooming is performed so that the position of the three-dimensional image resulting from the fusion in zooming in (or zooming out) moves forward (or back) as viewed from the viewer.

The left-eye image and the right-eye image displayed on the stereoscopic display device are generated from a pair of stereoscopic photograph images,
In the generation, the left eye image and the right eye image are generated by enlarging or reducing the pair of stereoscopic photograph images at a magnification m,
The zooming is based on a state in which the end points of the left-eye image and the right-eye image are aligned, and the end-point of the left-eye image and the end-point of the right-eye image as a function of the magnification m. By changing the distance t of
Zooming is performed so that the position of the three-dimensional image by the fusion in zooming in (or zooming out) does not move or moves to the front (or back) as viewed from the viewer.

The distance t is a function of the magnification m such that c × (1 / m−k) ^{n + 1} + b for n where 0 ≦ n, where m is the magnification, and c, b, and k are constants. is there.

  Further, the constant b is a condition that a three-dimensional image formed by a fusion based on the zoomed-in region is visually recognized at a position on the display screen of the three-dimensional display device for an arbitrary region selected in the three-dimensional photo image. It is the one requested by

Here, the feature is that a relatively distant view region and a relatively close view region are selected in the zoom-in target region of the stereoscopic photograph image, and the constant b is a relatively distant view zoom-in target region in the stereoscopic photograph image. This is roughly equivalent to the feature that the zoom-in is performed up to the enlargement limit magnification m ₀ and the stereoscopic image obtained by fusion of the region is obtained under the condition that the stereoscopic image is viewed at a position on the display screen of the stereoscopic display device.

  Further, the constant c is obtained by using the obtained constant b to obtain a three-dimensional image formed by a fusion based on the zoomed-out region of the selected arbitrary region of the stereoscopic photograph image. It is obtained under the condition that the image is visually recognized on the display screen or at a position behind it.

Here, the above feature is that a relatively distant view region and a relatively close view region are selected in the zoom out target region of the stereoscopic photograph image, and the constant c zooms out the stereoscopic photograph image to the reduction limit magnification m _1. This is roughly equivalent to the feature of obtaining a stereoscopic image by fusion of a relatively close-up area in the image under the condition that the stereoscopic image is viewed at a position on the display screen of the stereoscopic display device or a position behind it. is there.

  The constant k is equal to β.

  Further, the value of n is monotonously increased or monotonously decreased according to the change of the magnification m.

  The stereoscopic display device is any one of an anaglyph method, a liquid crystal shutter method, a polarizing filter method, a parallax barrier method, or a lenticular lens method.

  In addition, the left-eye image and the right-eye image in the image displayed on the stereoscopic display device are obtained by removing in advance vertical and horizontal deviations.

Depending on the viewing environment, "Scene I: All three cars appear to pop out closer to the screen", or "Scene II: All three cars appear to retract farther than the screen" or "Scene III: Three cars There are three cases in which one or two of the cars appear to be near the screen. However, even if the viewer initially feels the stereoscopic effect of scene I, the stereo photo viewer will If a function is provided, it is a diagram showing a specific image that a viewer can feel the stereoscopic effect of scene II or the stereoscopic effect of scene III by operating it. A black frame means a screen, and a cube indicated by a dotted line means a three-dimensional space. In the left side of the figure, the three-dimensional space is arranged in front of the screen, in the middle of the figure, it is arranged behind the screen, and in the right side of the figure, it is arranged near the screen. A conceptual image of left and right images handled in a stereo photo viewer is drawn from above. Two flat plates placed so that two thin flat plates overlap each other, and two left and right images (stereo pairs) taken with two cameras are pasted by the texture function of 2DCG, respectively To do. In the drawing, two flat plates are drawn in front and back for easy understanding, but conceptually the two flat plates are both of the same thickness and overlapped along the x-axis. An image for the left eye is pasted on the front plate, and an image for the right eye is pasted on the rear plate. However, an image taken by the left (or right) camera is used as a left (or right) eye image. The center of the flat plate of the left-eye image is fixedly arranged at the origin of the xy plane coordinate system fixed to the stereo viewer. The flat plate of the image for the right eye moves up and down, left and right, and rotates around the origin of the coordinate system around plus or minus. The plane coordinate system has a positive x-axis toward the right and a positive y-axis upward. The left and right images attached to the two flat plates are superimposed and projected in parallel on the stereoscopic display screen. The viewer stereoscopically views the projected image. At that time, the viewer views an arbitrary portion of the stereoscopic photograph (image) while zooming by a zooming operation using a mouse. FIG. 2 is a view of FIG. 2 as viewed from the top toward the negative direction of the y-axis. In the scenes shown in the left image and the right image, two points corresponding to each other in each scene are doubled and projected on the display screen. In this way, in the left and right images, there are an infinite number of points corresponding to the left and right sides of the scene. Here, only one of them is taken up. The left and right images are affixed to the respective flat plates, and the value of the shift between the two points in the corresponding image is d. The projected point by the point of the left image is indicated by a black circle point, and the projected point by the right image is indicated by a white circle point. Let D be the distance between the circle points on the screen. The viewer views the white circle point of the projected point on the display screen with the right eye and the black circle point with the left eye, and the viewer's brain fuses the three-dimensional image based on the interval D between the two round points. The viewer feels the perspective of whether the stereoscopic image looks close or far, depending on the value of the distance D between the black and white dots projected on the screen, but the position where the stereoscopic image is fused is From the geometric proportional relationship, it can be obtained as in Equation 2. The position of the stereoscopic image is determined by setting the position u on the display screen as a reference position, with the direction toward the display device being positive and the direction toward the viewer being negative. The value e is the viewer's pupil interval (generally 6.5 cm for humans), h is the distance between the viewer and the display screen, and D is the interval between the black and white circle points projected on the screen. The expression for the position u is a monotonically increasing function with respect to the distance D. It is a figure for calculating | requiring the magnitude | size of the stereoscopic image to visually recognize by a geometric procedure. Black circles, white circles, black circles, and white circles are lined up from the left on the screen, but the image part between the two black circle points and the image part between the two white circle points correspond to the left eye and right eye, respectively. Let r be the size of the partial real image on the screen. The real image of this part is visually recognized as a stereoscopic image by the gap D of the deviation. u is the depth of the stereoscopic image. When the size of the stereoscopic image to be visually recognized is v, and simple geometric proportionality is calculated, the following equation 3 is derived for v. Expansion limit magnification m _0, is a diagram showing a state of fusion type using the reduction limit magnification m _1, and the above t _0, t _1. The fusion equation having the exponent n of the forward / backward factor is a function passing through the point (m ₀ , t ₀ ) and the point (m ₁ , t ₁ ). Further, the fusion equation when the exponent n of the forward / backward factor is 0 is a hyperbolic function. When the magnification of the stereoscopic image with respect to the magnification of the real image is b and the magnification reference is the magnification m ₁ of the real image, and the size (reference length) of the real image is r ₁ , the magnification b of the stereoscopic image is several 22, Equation 2 and Equation 18 are used, but the viewing distance h is 50.0 cm, the pupil interval e is 6.5 cm, the reference length r ₁ is 1.0 cm, the reduction limit magnification m ₁ is 1.0, and the magnification is increased. in the case of the limit magnification m ₀ 10.0 is a diagram showing a magnification of the stereoscopic image with respect to the magnification of the real image. For the constant k ₂ × a in Equation 18, 1.2 cm was adopted. Further, the index n of the forward / backward factor was set to four types of 0.0, 0.5, 1.0, and 2.0. The solid line is a calculation graph of the magnification b of the stereoscopic image (when n is 0.0), and the broken line is a calculation graph of the depth u (n is 0.0, 0.5, 1.0 and 2,0) In the case of It is a figure which shows the outline | summary of the type | formula introduce | transduced in order to change the method of the approach of three-dimensional images, and the method of retreating in zooming. (1) is a diagram showing correction of vertical misalignment. When the vertical misalignment is remarkably misaligned, the three-dimensional image is not fused and must be aligned horizontally. In this correction method, the flat plate on which the right image is pasted is translated by p along the y axis in the plus direction, and is aligned with the left image. (2) is a diagram illustrating the correction of the shift in the left-right direction, and is a case where the two images are too far left and right. If you browse with a stereo photo viewer, you can feel the perspective three-dimensionality, and the three-dimensional image appears to pop out. As a correction method, the flat plate of the right image is translated in the plus direction by the value d along the x-axis. (3) is a diagram illustrating correction of deviation due to inclination, and is correction when the right image or the left image is inclined (rotated) around the origin. In this correction method, the flat plate on which the right image is pasted is rotationally converted in the plus direction by θ radians around the origin. This is an example of coordinate conversion using SPV (stereo photo browsing software developed based on a stereo photo viewer) developed using Java 2D-API as an example. The program of (a) fixes the left image flat plate, translates along the x-axis and y-axis with respect to the right image flat plate (procedure 1 and procedure 2), and centers the left image flat plate. By performing the rotation (procedure 3), the right image flat plate is rearranged with respect to the left image flat plate. In the part before the procedure 1, zoom conversion for zooming and translational conversion for shift are performed by the user's mouse operation. The program (b) shows an example of a coordinate conversion method (Non-Patent Document 2) of Java 2D-API and an example of a fusion method. It is a figure which shows the cooperation structure of the program in SPV which is the stereoscopic photograph browsing software developed based on the stereo photo viewer, Especially when the side by side format is adopted as the stereoscopic format of SPV, Java (registered trademark) It is a figure which shows the cooperation relationship of Java2D-API. It is a figure which shows the example of the screen of SPV actually developed based on the stereo photo viewer. It is an enlarged view of the operation button of SPV set in the lower part of the screen of FIG.

Chapter 8 Optimum perspective stereoscopic setting with a practical fusion equation The constants a and b of a practical fusion equation are parameters determined by simple visual experiments (judgment experiments by viewers' visual observations). is there. These determine the stereoscopic characteristics of each stereoscopic photo in the stereo photo viewer, and the values are determined so as to optimize the perspective stereoscopic effect of the viewer. However, the parameter a is a positive value. The parameter b takes one of positive, negative, and zero values. The determination method is described in this chapter.

Section 8.1 Parameters for Determining Characteristics of Practical Fusion Equations When the image is enlarged by increasing the magnification m as in Problem 1 shown in Section 5.1, a double image is generated. Since the gap D between the projected left and right images is greatly enlarged (this is the same as when a fine portion is seen enlarged with a magnifying glass), the fusion collapses. In order to prevent this, first, it is necessary to appropriately set the value of the parameter b of the practical fusion formula. Therefore, this parameter b is called “double image prevention constant”.

Also, as in Problem 2 of Chapter 5, if the entire stereoscopic image is projected from the display screen, the parameter a is set to 0 even if the magnification m is close to the reduction limit magnification m _1. When a close value (however, a> 0) is set, the contribution of the value d of the deviation equation constituting the fusion equation becomes small. As a result, a stereoscopic image due to a perspective stereoscopic effect is perceived nearby, resulting in a tight stereoscopic image on the eyes. In order to prevent this, it is necessary to set the value of the parameter a appropriately. Therefore, this constant is called “perspective control constant”.

  Thus, the parameters a and b are important for controlling the fusion. Therefore, in the following, how these parameters should be set will be described in detail.

Section 8.2 Setting Practical Fusion Equation Parameters When the viewer zooms with the mouse and changes the magnification m of the image on the screen, the right image flat plate is applied according to the practical fusion equation. The stereo photo viewer has a mechanism that translates the position of the image by a value t on the x-axis (provided that the position of the left image plate is fixed) and a mechanism that reads and stores the value at that time. Shall. As the parameter determination method, the individual determination method described in the section 6.2 (3), in which the parameter b is first determined and then the parameter a is determined, is adopted for a practical fusion formula. Parameters are set for each stereoscopic photo.

Note that when determining the parameters, the area of the image and the perspective of the landscape shown in the image are involved. Therefore, the following terms to be referred to in this specification are defined for the region and perspective. The terms are “entire view area”, “partial area”, “near view area”, and “far view area”. “Overview area” refers to the entire area of the image that is zoomed out to the reduction limit magnification m ₁ and displayed on the entire screen (the magnification m ₁ is the zoom-out limit, and the magnification is 1.0) Is described in section 7.1 (5)). The “partial area” is a part of the whole, and means a partial area of the image displayed on the screen. Of course, for example, a landscape of a distant mountainous landscape, a landscape of nearby trees, a landscape of a house in the middle of the photo, etc. are mixed, and a landscape of the entire photo is configured. An area in the image where a distant view is shown is called “distant view area”, and an area where a close landscape is shown is called “near view area”. The “distant view region” and the “near view region” constitute a part of the entire photograph, and are “partial regions” with respect to the “entire view region”. In addition, when it is referred to as “recent view area”, it indicates a partial area that is the most foreground among the parts that are regarded as a foreground. For example, in a human photograph where three people are lined up from the back with the cityscape in the background, in the image on the screen, the background of the cityscape is in the distant view area, the mass of three people is in the foreground area, and in the foreground area The part where the foreground person appears is the most recent scene area. Similarly, a “farthest view area” also refers to a partial area that is the farthest in the part that is regarded as a distant view.

  In the present invention, attention is paid to each of the distant view region and the foreground region when determining the parameters of the fusion equation. Here, when focusing on the farthest view area and the most recent view area, good zooming can be performed on the entire screen. However, when simply focusing on the distant view area and the foreground area relatively, the limited screen magnification m is set. Zooming effective in the range can be performed. In the present invention, since it can be applied in any of the above cases, these are indicated as “relatively distant view region” and “relatively close view region”, and are relatively distant view regions or close view regions in a relative sense. It means that.

(1) Determination of parameter b A method of determining the parameter b will be described. For a certain stereoscopic photo A, the viewer selects an arbitrary area to be zoomed in the image displayed on the screen, and zooms in on the area to the magnification up to the enlargement limit magnification m ₀ . The arbitrary area may be a near-field area, a far-field area, or an intermediate area in the image, but if you want to zoom in on a distant landscape that is particularly small, you can view the desired far-field area. select. Then, the viewer looks at the display screen, and if there is a deviation between the corresponding area of the left image and the corresponding area of the right image, the right image flat plate is placed on the x-axis to eliminate the deviation. By parallel translation, the right image area and the left image area are overlapped without deviation so that a stereoscopic image can be visually recognized at a position on the screen as shown in FIG. Countermeasure to Problem 1). This three-dimensional image is confirmed by a visual experiment by a viewer. Visual adjustment is performed while changing the value of the parameter b so that a stereoscopic image is generated at a position on the screen. The adjusted value of the parameter b at that time is b _0, and the value t ₀ for translating the right image flat plate on the x-axis using the value is as follows using the fusion equation.

Using this value t ₀ , the position of the right image flat plate is rearranged.

The value b _{0 of the} parameter b is a parameter specific to the stereoscopic photo A and is stored inside the stereo photo viewer. When the parameter b is determined in this way, the value d of the deviation equation (in the fusion equation) for the left and right images is ₀ at the enlargement limit magnification m ₀ of the image. As shown, a three-dimensional image without the occurrence of a double image is fused.

(2) Determination of parameter a Next, how to determine the parameter a will be described. In order to solve the problem 2 in Chapter 5, it is necessary to set the entire stereoscopic image so that it is positioned on the screen or behind it. Since the parameter b is already determined, a state without deviation is realized at the enlargement limit magnification m ₀ . The viewer zooms out from this state to the reduction limit magnification m ₁ . That is, the entire view area of the image is displayed on the screen.

A visual experiment for determining the value of the parameter a is performed at the reduction limit magnification m ₁ . That is, the viewer determines the degree of the three-dimensional image fused by the left and right images by a visual experiment. In this experiment, a three-dimensional image in which an optimal three-dimensional feeling is felt for the viewer is fused while changing the value of the parameter a in the fusion formula in which the parameter b is determined (Problem 2 in Chapter 5). The value of parameter a (a> 0) is visually adjusted. The stereoscopic effect felt by the viewer at this time is the perspective stereoscopic effect, and the setting of the parameter a is the setting of the perspective factor.

Specifically, this is performed as follows. With respect to the stereoscopic photograph A, in the entire view area zoomed out to the reduction limit magnification m ₁ , the viewer looks at the stereoscopic display screen and changes the value of the parameter a (provided that a> 0), but the corresponding area of the left image The entire stereoscopic image reproduced from the corresponding area of the right image is made visible at a position on the stereoscopic display screen or a position behind it. This is to make the three-dimensional space of the three-dimensional image a “three-dimensional space behind the screen” as shown in the middle of FIG. This work is as follows: “The viewer changes the value of the parameter a while paying attention to the partial area that is the closest view in the entire view area that is reduced and displayed at the reduction limit magnification m ₁ , that is, the latest view area. It can also be realized by superimposing the left and right images on each other without deviation. Changing the value of the parameter a is to translate the right image flat plate on the x-axis.

The adjusted value of the parameter a at that time is a _1, and the value t ₁ used to translate the right image flat plate on the x-axis using the value is as follows using the fusion equation.

Using this value t ₁ , the position of the right image flat plate is rearranged. The value a _{1 of the} parameter a is a unique parameter related to the stereoscopic photo A, and is stored inside the stereo photo viewer.

  The parameter a will be described in more detail. When the parameter a is a small value, the image is fused in front of the screen. However, if the parameter a is a value of a certain size, the entire stereoscopic image is fused to the rear of the display screen. A feeling is obtained. This state depends on the parameter a, and if a value that is extremely large is set, the perspective stereoscopic effect can be felt far away depending on the viewer, resulting in a stereoscopic image with a sense of incongruity. When the limit is exceeded, the difference between the two images becomes too large, and the stereoscopic image becomes a double image and does not melt. Although it is important to appropriately determine the value of the parameter a, it is easy to set the aptitude value of a because there is a considerable range according to previous experiments.

  The mathematical background of the individual determination method in practical fusion formulas is explained above, but the cooperative determination method is trial and error, so it is difficult to make such a clean explanation. It is. Specific examples of how to determine the constants a and b in the individual determination method are described in Sections 11.2 and 11 based on SPV (stereophoto browsing software developed based on a stereo photo viewer) developed by the inventors of the present invention. It will be described in Section 3.

Chapter 9 Correction of stereo pair misalignment caused by camera shooting When taking 3D photographs using two normal cameras (same manufacturer and same model), even if careful work is performed, the distance between the two left and right images In this case, a vertical shift or a tilt shift occurs, and it is not suitable as a stereo pair as it is. (Even if a stereoscopic camera is used, zooming often causes a similar shift near the resolution limit). The slightest installation error that occurs when two cameras are attached to the camera platform causes the deviation. This shift is not so much a problem when using the 3D photo for the purpose of zooming out and viewing the entire photo, but for the purpose of zooming in and examining a narrow area, Since the gap between the two becomes remarkable, the stereoscopic photograph cannot be used as a stereo pair. Therefore, it is necessary for the stereo photo viewer to incorporate a mechanism for correcting the shift between the left and right images that prevents the formation of a stereo pair.

  Hereinafter, a method for correcting a shift in a stereo pair will be described with reference to FIG. It should be noted that these corrections are easier to perform in a non-stereoscopic state without wearing stereoscopic glasses, except in the case of Section 9.2. The deviation is corrected visually so that the left and right images are completely overlapped.

Section 9.1 Correction of Up / Down Shifts As shown in FIG. 9 (1), when there is a significant vertical shift between the left and right images, the three-dimensional image is not fused, so it must be aligned horizontally. The cause of such vertical misalignment occurs when two cameras are fixed to a camera platform with screws, or when one of them is loosely tightened. In this correction method, the flat plate on which the right image shown in FIG. 9A is pasted is translated along the y-axis in the plus direction by p to match the left image.

Section 9.2 Correction of Misalignment in the Left-Right Direction As shown in FIG. 9 (2), this is a case where the two images are too far left and right. This occurs when the two cameras are attached to the camera platform when the distance between the lenses is too large, and is more likely to occur when a close object such as a person is photographed. If you browse with a stereo photo viewer, you can feel the perspective three-dimensionality, and the three-dimensional image appears to pop out. As a correction method, the flat plate of the right image shown in FIG. 9B is translated in the plus direction along the x axis by a value d. This correction method is already included in the method described in section 8.2 (1), and can be dealt with by adjusting the fusion parameter b.

Section 9.3 Correction of Deviation by Inclination As shown in FIG. 9 (3), the right image or the left image may be inclined (rotated) around the origin. In this case, when two cameras are attached to the camera platform, one of them is installed with a slight inclination to the camera platform. An inclination of about 0.5 degrees may occur depending on how the screws are tightened. In this correction method, as shown in FIG. 9 (3), the flat plate on which the right image is pasted is rotationally converted in the plus direction by θ radians around the origin.

Section 9.4 Other misalignments (1) Misalignment due to x-axis or y-axis tilt As for the tilt of the image due to the installation of the camera, the tilt (rotation) of the x-axis or y-axis can be considered. It was found that the work was within a small angle, and this correction was almost absorbed by other corrections by prior experiments. This is because correction of the tilt of the x-axis is roughly regarded as correction of the vertical deviation, and correction of the tilt of the y-axis is roughly regarded as correction of the horizontal deviation. This is because it is reflected in the direction deviation correction operation. Therefore, in order to avoid the complexity of the correction, the stereo photo viewer does not incorporate a direct shift correction due to the inclination of the x axis and the y axis.

(2) Deviation along the z-axis When two cameras are installed on the camera platform, one camera is slightly closer to the subject to be photographed than the other, or Misalignment may occur. With respect to the camera position, if a “three-dimensional coordinate system in which the center of the film surface of one camera is the origin and the surface is the xy plane” is set, this deviation can be said to be a deviation along the z-axis direction. If this correction is performed, the right image is enlarged or reduced by an appropriate value. However, since the displacement in the z-axis direction when two cameras are attached to the camera platform is generally within 1 mm, the error is usually negligible compared to the distance from the camera to the subject (several tens of meters). I can do it. Therefore, the stereo photo viewer does not incorporate a direct shift correction regarding the z-axis as in the case of the inclination of the x-axis and the y-axis described in Section 9.4 (1).

Chapter 10. Image Rearrangement Method Section 10.1 Affine Transformation A practical means of correcting the deviation between the left and right images is the 2DCG coordinate transformation function used by the stereo photo viewer. is there. The shift correction is to rearrange the right image flat plate, but this can be done by simple coordinate transformation. The coordinate transformation is affine transformation of enlargement / reduction, rotation, and translation. A general 2DCG has a 3 × 3 affine transformation matrix therein, and the transformation is performed in real time by matrix operation.

A method of appropriately converting (rearranging) the position of the right image flat plate with respect to the position of the reference left image using an affine transformation matrix will be shown (Non-Patent Document 2). [0 0 1] is the three-dimensional row vector of the right image plate centered at the origin of the coordinate system, and [x y 1] is the row vector of the plate when coordinate transformation (rearrangement) is performed. To do. Note that 1 in the third element of the vector is 1 in the homogeneous coordinate expression. The deviation correction procedures shown in Sections 9.1, 9.2, and 9.3 are referred to as Procedure 1, Procedure 2, and Procedure 3. The elements of the conversion matrix for correction in these procedures are shown in Table 2. Is shown in (The transformation matrix described in this table is conceptual and may require further transformations depending on how the origin is determined). In this table, the aspect ratio correction procedure, which will be described later, is called procedure 4, and its conversion matrix is also described. This correction will be described in section 10.3. The conversion order between them is described in Section 10.4. However, it is necessary to perform conversion (rearrangement) in the order from Procedure 1 to Procedure 4. Then, the coordinate transformation (rearrangement) of the three-dimensional row vector [x y 1] of the right image flat plate at that time is performed by the following equation (27). However, since it is fixed for the reference flat plate for the left image, the transformation matrix M _t1 (element p is zero), M _t2 (element t is zero), M _r (element trigonometric θ Is a unit matrix, and as a result, only the aspect ratio correction conversion in the procedure 4 is performed.

  Prior to procedure 1, coordinate conversion for browsing, that is, enlargement / reduction conversion by zooming and parallel movement conversion by moving in a plane are performed, but the description thereof will be omitted.

Section 10.2 Explanation of Image Rearrangement Using Fusion Formula Regarding the correction of the left and right images in Procedure 2, using the practical fusion formula of Equation 17 described in Section 6.2 (4), “How the coordinate conversion function of 2DCG corrects the left / right displacement of the left and right images” will be described.

The optimum rearrangement value t with respect to the x-axis of the right image flat plate is obtained by a practical fusion equation using the magnification m. Incidentally, the value t is an element of the affine transformation matrix M _t2 as shown in Table 2. When the viewer performs a zooming operation for correcting the deviation, the value t is calculated from the fusion equation using the magnification m. Then, the affine transformation matrix M _t2 is calculated based on this value t, and the three-dimensional row vector [x y 1] of the right image flat plate is calculated. Then, using this row vector, the coordinate conversion function of 2DCG rearranges the right image flat plate to an optimal position.

Section 10.3 Aspect Ratio Conversion In addition to the correction from Procedure 1 to Procedure 3, it is necessary to correct the aspect ratio (horizontal / vertical value) of Procedure 4 for images. A normal photograph has an aspect ratio of 1.333, and there is no aspect ratio problem when outputting to a stereoscopic display of the same type screen using a personal computer compatible with XGA (4 to 3 type) resolution. However, when the image is displayed in a horizontally wide recent wide area, the image spreads horizontally, so that coordinate conversion for adjusting the width is necessary. For example, when outputting to a wide 3D display having a 5 to 3 type screen using a personal computer compatible with XGA (4 to 3 type) resolution, the width of the 3D image becomes thick as it is, so that the width of the original 3D image is restored. Therefore, it is necessary to increase the aspect ratio of the image on the personal computer side by 0.8. The corrected aspect ratio at this time is 1.333 × 0.8. If the ratio based on the aspect ratio (1.333) of the photograph is defined as the converted aspect ratio, the converted aspect ratio of the image at that time Becomes 0.8.

  When a stereoscopic display whose stereoscopic format is the side-by-side format is used as the display device, the stereoscopic format of the stereo photo viewer needs to be the same side-by-side format. Therefore, the converted aspect ratio of each of the left and right images must be further halved. By the way, when outputting in a side-by-side format from a XGA resolution compatible personal computer to a wide 3D display with a 5 to 3 screen, the converted aspect ratio of each of the left and right images is 0.5 × 0.8, that is, 0. .4.

  The converted aspect ratio is r / s, where r is horizontal and s is vertical, and the values r and s can be calculated from the corresponding resolution of the personal computer and the screen size of the stereoscopic display. In the coordinate transformation for the enlargement / reduction of the left and right flat plates for images, the values r and s are elements of the affine transformation matrix Ms.

Section 10.4 Coordinate transformation order In matrix computation, the computation order is important. If you make a mistake, you cannot get the correct result. The order of matrix operations for correction described in Section 10.1 is performed in the order of procedure 1 → procedure 2 → procedure 3 → procedure 4. However, since procedure 1 and procedure 2 do not depend on the matrix operation order, either one may be mathematically performed first, but in reality, a visual confirmation experiment is initially performed with an unadjusted stereoscopic photograph. It is necessary to perform the correction of 1 above and below (a stereoscopic image may not be fused unless this correction is performed first).

Section 10.5 Example of coordinate transformation How the affine transformation shown above is described in an actual 2DGG is developed based on the SPV (stereo photo viewer) developed using Java2D-API. Let's take 3D photo browsing software as an example. The main part of the program is shown in FIG. The flow of description in other 2DCGs may be considered to be almost the same as the example shown here. The program in FIG. 10A fixes the left image flat plate, translates along the x-axis and y-axis with respect to the right image flat plate (procedure 1 and procedure 2), and the center of the left image flat plate. By rotating about the center (procedure 3), the right image flat plate is rearranged with respect to the left image flat plate. As a result, the shift between the left and right images can be adjusted. Further, in the part before the procedure 1, zoom conversion for zooming and parallel translation for shift are performed by the user's mouse operation. The program shown in FIG. 10B shows a usage example of the coordinate conversion method of Java2D-API (Non-Patent Document 2) and an example of a practical fusion formula.

  As described above, when the coordinate conversion function of 2DCG (in FIG. 10, Java2D-API) is used, the main problem of rearrangement of the right image flat plate can be simply described. The listed programs are organized and arranged for easy viewing. Some details are omitted, and are not complete. In the case of this Java2D-API program, the origin is flat. Note that the translation transformation is frequently used because the upper left corner of the (image) is the origin.

Chapter 11 SPV Development Based on Stereo Photo Viewer Section 11.1 SPV Implementation Method and Program Structure The inventor of the present invention conceived of utilizing the 2DGC image processing function when designing a stereo photo viewer. The general concept is shown in Chapter 10. The rest of this chapter is a description of the SPV developed by the inventors of the present invention based on the design concept.

  In SPV, Java2D-API was adopted as 2DCG (Non-patent Document 2). Java2D is based on the Java (registered trademark) programming language. Java2D is responsible for image processing and coordinate transformation (affine transformation). Source code using Java2D-API is translated by a Java (registered trademark) compiler and executed by a Java (registered trademark) virtual machine. The Java (registered trademark) program part generates left and right images, and synchronizes the zooming of the two images. Therefore, the side-by-side format is adopted as the three-dimensional format of SPV. FIG. 11 shows the linkage relationship between Java (registered trademark) and Java2D-API, and the functions governed by languages and the like are listed below. Since SPV employs Java (registered trademark), it operates independently of the OS.

Java2D-API
Perform coordinate transformation and rearrange the left and right images appropriately.
-Setting of transformation matrix-Utilization of texture function-Java (registered trademark) language
Work as the parent language of Java2D.
・ Setting of left and right two screens ・ Utilization of scroll function for zooming ・ Synchronization of left and right images during zooming ・ Utilization of thread function for automatic zooming function ・ Storing of stereoscopic photo information

Section 11.2 SPV practice Various operation buttons are prepared for SPV. The main buttons and their functions are shown below.
● Misalignment correction button (button to correct the misalignment and create a complete stereo pair as a stereoscopic photo)
H button: Corrects the left / right misalignment (determines the constant b of the fusion equation).
・ V button: Corrects the vertical displacement.
• θ button: Corrects the deviation due to tilt.
● Depth setting button (button to determine the best perspective perspective)
D button: determines perspective perspective (determines fusion constant a).
● Aspect ratio setting button (Aspect ratio setting button)
-A button: Adapts the stereoscopic image to the aspect ratio of the window display screen.
● Double layer button (button for overlapping left and right images to detect misalignment between left and right images)
・ DBL-on button ... The left image is translucently superimposed on the right image.
-DBL-off button ... Cancels the above function.
● Zoom function button (button for zooming stereoscopic photos)
・ ZM-on button ... Zooms an arbitrary part of the stereoscopic photo.
・ ZM-off button ... Cancels the above function.
● Zooming area setting button (button for setting the area (position) for zooming)
-TG-on button: After setting, specify the zoom target position by mouse click.
・ TG-off button: Cancels the setting.
● Browsing function button (button used when viewing stereoscopic photos)
・ NEXT button: View the next 3D photo.
-BACK button ... View the previously displayed stereoscopic photo.
・ AUTO button ・ ・ ・ Automatically display like a photo frame, and repeatedly view multiple 3D photographs. (Also performs automatic zooming.)
● Record button (button to record fusion parameters a, b, etc.)
・ REC button ... Records and saves 3D photo data in the built-in file.
● Magnification setting bar (scroll bar that determines image magnification)
-S bar: Sets the image to an arbitrary magnification.

  FIG. 12 shows an actually developed SPV screen, and FIG. 13 shows an enlarged view of operation buttons set at the bottom of the screen.

  The usage of the lowermost misalignment correction button in FIG. 13 will be described a little. This is a group of buttons for correcting the displacement of the stereo pair that occurs when shooting with the camera, and is used as a preliminary preparation before browsing. Every time the correction button is clicked by the viewer, a minute value is created inside the SPV. Coordinate conversion is instantaneously performed for each click based on this value, and the right image flat plate is rearranged with reference to the left image flat plate. Each time the viewer checks the stereoscopic image generated from the left and right images with his own eyes. For example, when the plus button of the V button for performing vertical correction (switching between + and-with the SIGN button) is clicked, the right image flat plate moves slightly upward. Click on the V button plus or minus button to fine-tune the state of the 3D image until the optimal 3D image is generated.

Section 11.3 Operation such as misalignment correction in SPV SPV has a function to superimpose left and right images for a display device whose stereoscopic format is a side-by-side format. By using this function, it is possible to easily execute the deviation correction performed without wearing the stereoscopic glasses. In the case of using this display device, the left image is displayed on the left half of the screen and the right image is displayed on the right half, respectively, and the image is displayed ½ times horizontally. Therefore, click the DBL-on button and overlay the semi-transparent image of the reference left image on the right image. Since the left and right images overlap, the position of the right image is adjusted according to the reference left image while observing the left and right overlap. Click the DBL-off button when the overlay operation is complete.

(1) Vertical displacement correction This correction is a correction in which stereoscopic glasses are not worn. A photographer of a stereoscopic photograph (also a viewer) stores the photographed stereoscopic photograph with a file name in a folder held inside by the SPV. Photos are called with the NEXT button of the browsing function, and correction for each photo is performed as follows. Initially, by clicking the V button (+ and-) of the correction function, the left and right images are overlapped vertically by visual judgment so that there is no deviation. Then, the p value for correcting the deviation is recorded and saved in the viewer by clicking the REC button.

(2) Deviation correction by inclination Next, deviation correction by inclination is performed. This correction is correction without wearing stereoscopic glasses. By clicking the θ button (two types of + and −) of the correction function, the left and right images are overlapped by visual judgment so that there is no deviation due to tilt. The θ value for correcting the deviation is recorded and saved in the viewer by clicking the REC button. In prior visual experiments, there were very few stereoscopic photographs of objects that had to be corrected, depending on how the stereoscopic photograph was taken.

(3) Left / right deviation correction Next, right / left deviation correction is performed. This correction is first performed without wearing stereoscopic glasses, and then a stereoscopic image to be fused is confirmed using stereoscopic glasses. As shown in section 8.2 (1), a part of the far-field region of the stereoscopic photograph is enlarged (zoomed in) to the resolution limit (magnification limit magnification m ₀ ) by zooming operation with the mouse. In this state, the H button (+ and-) of the correction function is clicked so that the left and right images overlap in the enlarged portion of the distant view so that there is no deviation. This determination is made visually. Then, in this state, wearing 3D glasses, it is confirmed whether a 3D image is generated at a position on the screen. If the stereoscopic image can be fused at the position of the screen, the value of the parameter b for correcting the left / right deviation is recorded and saved in the viewer by clicking the REC button.

(4) Adjustment of perspective stereoscopic effect Finally, the perspective stereoscopic effect is adjusted. This correction is performed by stereoscopic viewing with stereoscopic glasses. Click the D button (two types, + and-) to make it possible to appropriately obtain the perspective of the entire stereoscopic image by visual determination. First, the D button is clicked while zoomed out to the reduction limit magnification m ₁ shown in section 8.2 (2). If the stereoscopic image can be fused with an appropriate perspective, the value of the parameter a that defines the perspective is recorded and saved in the viewer by clicking the REC button. The correction of the “perspective stereoscopic effect” is an adjustment rather than an image shift correction, and it is sufficient if an appropriate (or favorite) perspective is obtained.

  If the deviation correction for the above items (1), (2), and (3) is not performed correctly, it is not possible to perform complete stereoscopic viewing. The time required for the three misalignment corrections and the adjustment in the above item (4) is about several minutes with respect to one stereoscopic photograph. In the SPV, a table for recording and saving the values of a, b, p, and θ for an arbitrary number of stereoscopic photographs is prepared. These table values are automatically used for subsequent browsing.

(5) Aspect ratio selection and browsing method At the start of browsing, click the A button (two types, + and-) to match the screen size of the stereoscopic display device and the aspect ratio of the stereoscopic photo. Since the aspect ratio is automatically calculated from a stereoscopic display screen or the like, the viewer selects an appropriate aspect ratio. When you click on the AUTO button for actual viewing, SPV will continue to display multiple 3D photographs automatically zoomed like a photo frame (many of the products currently sold are for non-stereo photography). . When the viewer clicks the NECT button or the BACK button, an arbitrary zooming operation can be performed on a specific stereoscopic photograph.

Chapter 12 Conclusion (1) Supplement (Application to 3D graphics)
The zooming mechanism in the stereoscopic vision described in this specification can be applied to the technology of the three-dimensional virtual space of 3D graphics, and it will be shown below that it is simple. In 3DCG, a virtual person (usually called an avatar) having a viewpoint in a virtual space is placed, and a scene around the avatar moving around (walking through) in the virtual space is displayed on a display device. Is adopted. The avatar has two eyes, and the viewpoint is looking at an image pasted on a flat plate placed in the virtual space from a certain distance z. The magnification m on the screen, which is the attribute (variable) of the 2DCG plane used in the practical fusion formula of Equation 17, is the distance z from the viewpoint of the avatar, which is the attribute (variable) of the 3DCG space, to the image. Should be replaced.

  The distance z and the magnification m of the image projected on the display screen are inversely related,

(K is a constant inversely proportional to z and m), z ₀ × m ₀ = k and z ₁ × m ₁ = k.

Is obtained. Further, the range of the magnification m in Expression 19 and m ₁ ≦ m ≦ m ₀ can be converted into the range of the distance z as follows.

Incidentally, z ₀ should be called “approach limit distance” and z ₁ should be called “retreat limit distance”. In this way, the coordinate conversion formula of the t coordinate regarding the distance z from the viewpoint of the avatar in the virtual space to the flat plate on which the image is pasted is obtained. Equation 29 is a practical fusion equation when walking through (returning to or from an image) a virtual space in 3D graphics. Parameters a and b when the constant k is 1 are It is a parameter determined by the visual experiment described in the chapter.

  In addition, the inventor of the present invention has developed 3D graphics viewing software similar to SPV using VRML language, JavaScript language, and HTML language, which are 3D graphics. The fusion formula for the virtual space uses Equation 29, and the same performance as SPV can be made in terms of performance.

(2) Conclusion (Usefulness of SPV based on stereo viewer)
In this specification, we proposed a new method of stereo photo viewer using 2DCG technology that allows you to zoom in and view stereoscopic photographs, and introduced SPV that realized it. If the 2DCG image processing technique is simply used, a minute shift in the left and right images (stereo pair) of the stereoscopic photograph is enlarged during zooming, and as a result, stereoscopic viewing cannot be performed. In order to solve this problem, a practical fusion formula was derived from the principle of stereoscopic vision, and a method for determining the parameters of the formula from a judgment experiment by visual observation of the stereoscopic image was devised. With this fusion formula, even if an arbitrary area is zoomed to the resolution limit of stereoscopic photography, it is possible to reproduce a stereoscopic image of a fine area neatly without generating a double image. (Depending on how the 3D photograph is taken, the distant view and the close view may be mixed and the distant view may be mixed. For such a special 3D photograph, SPV completely generates all double images. However, the development of an SPV that suppresses the generation of a double image as described in this specification by a fusion method has not been present in the world of stereoscopic systems).

  Actually, a stereo pair of 12 million pixels was created using two Fujifilm A220 digital cameras, and an experiment using SPV was performed. If allowed, you could zoom in at a higher magnification). The stereoscopic display device used was a G170P polarized glasses display device manufactured by Dimen, and the resolution was XGA (1024 × 768).

  In addition, if you do not use the fusion formula, if you zoom in from the zoomed out state, the 3D image should expand toward the viewer as it progresses. In the state where the image is enlarged, the stereoscopic image seems to gradually move back its position. Such a phenomenon also occurs in zooming shooting of 3D video images (Non-Patent Document 3). However, by using the fusion formula, it has become possible to suppress the occurrence of such a phenomenon.

  SPV makes full use of 2DCG technology, and correction of stereoscopic photograph displacement, that is, rearrangement of left and right images, is easily performed by causing the affine transformation matrix of 2DCG to perform the calculation. Further, since a mechanism capable of performing automatic zooming at an arbitrary position of a stereoscopic photograph using the animation function of 2DCG is provided, the SPV can be used as a software product of a stereoscopic photo frame.

  The SPVs developed this time are: (a) 3D display device for personal computers (both 3D glasses and naked eyes), (b) 3D 3D television (commercially available consumer electronics, 3D glasses and naked eyes), (c) A dual projector type (using two projectors) stereoscopic projection system and (d) a half mirror type (using two liquid crystal panels) high-definition stereoscopic display system can be displayed. The stereoscopic formats that these stereoscopic display devices accept are “side-by-side format” for the former two devices, and “dual channel format for the latter two devices (the left and right two images are not compressed, and each has independent channels). The format of the system used for distribution via Therefore, SPV enables the handling of two stereoscopic formats, “side-by-side format” and “dual channel format”.

Claims (9)

A stereoscopic display device that presents a left-eye image and a right-eye image to the viewer's left eye and right eye, respectively, and supplies the left-eye image and right-eye image to the stereoscopic display device. A stereo photo viewer that provides a viewer with a left-eye image and a right-eye image for stereoscopic image display,
Whether the distance D between the same objects of the left-eye image and the right-eye image in the image displayed on the stereoscopic display device is constant according to an increase (or decrease) in magnification due to zoom-in (or zoom-out) By monotonously decreasing (or increasing), the position of the three-dimensional image by the fusion based on the image for the left eye and the image for the right eye in the zoom-in (or zoom-out) does not move or is in front of the viewer (or Zooming to move to the back)
The distance D is the magnification m, the the alpha and beta is a constant, for 0 ≦ n becomes n, by changing in proportion to D to α × (1-β × m ) n + 1 / m n ,
A stereo photo viewer capable of zooming , wherein zooming is performed so that a position of a three-dimensional image resulting from the fusion in zooming in (or zooming out) moves forward (or back) as viewed from the viewer. The left-eye image and right-eye image displayed on the stereoscopic display device are generated from a pair of stereoscopic photograph images,
In the generation, the left eye image and the right eye image are generated by enlarging or reducing the pair of stereoscopic photograph images at a magnification m,
The zooming is based on a state in which the end points of the left-eye image and the right-eye image are aligned, and the end-point of the left-eye image and the end-point of the right-eye image as a function of the magnification m. By changing the distance t of
The zooming according to claim 1, wherein zooming is performed so that a position of the stereoscopic image by the fusion in zooming in (or zooming out) does not move or moves to the front (or back) as viewed from the viewer. Possible stereo photo viewer. The distance t is a function of the magnification m, where m is the magnification, c, b, k are constants, and c × (1 / m−k) ^{n + 1} + b for n where 0 ≦ n. The zoomable stereo photo viewer according to claim 2, characterized in that:   The constant b is obtained on the condition that a three-dimensional image by a fusion based on the zoomed-in region is visually recognized at a position on the display screen of the three-dimensional display device for an arbitrary region selected in the three-dimensional photo image. The zoomable stereo photo viewer according to claim 3, wherein the stereo photo viewer is capable of zooming.   As the constant c, a three-dimensional image formed by fusion based on the zoomed-out region of the selected arbitrary region of the stereoscopic photograph image using the obtained constant b is displayed on the display screen of the stereoscopic display device. 5. The zoomable stereo photo viewer according to claim 4, wherein the stereo photo viewer is obtained under the condition that it is visually recognized at a position above or behind it.   6. The zoomable stereo photo viewer according to claim 1, wherein the constant k is equal to the β.   The zoomable value according to any one of claims 1 to 6, wherein the value of n is monotonously increased or monotonously decreased in accordance with a change in the magnification m. Stereo photo viewer.   The stereoscopic display device is any one of an anaglyph method, a liquid crystal shutter method, a polarizing filter method, a parallax barrier method, or a lenticular lens method, according to any one of claims 1 to 7. Stereo photo viewer capable of zooming as described in.   The left-eye image and the right-eye image in the image displayed on the stereoscopic display device are obtained by removing in advance a vertical shift and a tilt shift. The zoomable stereo photo viewer according to any one of 8.

Images