JP4413203B2 - Image presentation device - Google Patents

Description

  The present invention relates to an image presentation apparatus that presents a two-dimensional image using a display whose position and orientation are variable.

  Various interactive user interface technologies have been devised that construct a virtual three-dimensional world space and make the user appear to exist inside the virtual world.

  “Head-mounted display” has been devised as a technology for changing the viewpoint position and line-of-sight direction in the virtual world. This detects the position and orientation of the user's head in the real world, projects the virtual world on an image according to the detection result, and presents the image on a display attached in front of the user's eyes. is there. In addition, in order to realize the same effect more simply, the position and orientation of the user's head and the orientation of the eyeball are detected, an image is generated according to the detection result, and displayed on the conventional “stationary display” Techniques to make it happen are also being studied.

On the other hand, “data glove” has been devised as a technique for manipulating objects in a virtual world. In this method, a user wears a glove having a position / orientation sensor disposed at a corresponding part of each joint, monitors the movement of the hand and fingers, and uses the movement and shape as input information. As a result, it is possible to operate an object in the virtual world with the same feeling as in the real world. Also, a method has been devised in which a user operates an input means (for example, a three-dimensional mouse) specialized in position and orientation, detects the position and orientation of the input means, and uses the detection result as input information. Further, conventional input methods such as an existing keyboard and mouse are also used.
Japanese Patent Laid-Open No. 4-52869 JP 7-306955 A

  However, each method described above has the following problems. The “head-mounted display” is problematic in that it is difficult to put on and take off and feels uncomfortable while being worn, and is not suitable for frequent use and long-lasting use. Moreover, since the user's field of vision is obscured, the danger caused by obstacles in the real world cannot be ignored. Furthermore, in order to simulate the entire field of view with sufficient image quality, a very high resolution display is required, which is limited by the resolution of the display device and the calculation cost for image generation. The method of generating an image according to the position and orientation of the head and the orientation of the eyeball and displaying the image on the “stationary display” is not an effective method because the range of the view is limited.

  “Data glove” is also problematic due to the complexity of attaching and detaching and the discomfort during wearing. In addition, for each individual object, it is necessary to calculate how the force is transmitted at the contact point between the hand and the object. When using an input means specialized for the position and orientation, the operation must be performed at a place different from the place where the image is displayed, so it cannot be said to be an intuitive operation. A method using a conventional input method such as a keyboard and a mouse is an operation lacking a sense of reality because the user's intention must be transmitted to the computer by an abstract expression such as input of coordinate values.

  The construction of an interactive interface based on a three-dimensional space using a simple device has various applications. For example, in the virtual reality field, intuitive space recognition and interactive object manipulation are essential in a virtual three-dimensional world, and that technology can be realized with a small-scale system such as a portable terminal. It is desired. In addition, if it is possible to accurately extract information required from a complicated three-dimensional structure without requiring acquisition of special skills, it can be applied to fields such as medical image analysis. Under such circumstances, it is important to construct a spatial user interface that can be used easily and is characterized by natural and effective space formation and motion response.

  The present invention has been made in view of such circumstances, and it is not necessary to attach a device such as a sensor or a display to the user's body, and it is not necessary to use a high-resolution display. An object of the present invention is to provide an image presenting apparatus.

  The image presenting apparatus of the present invention uses a movable display having a normal resolution that is directly held by a user's hand or supported by an indicator, and a viewpoint and a projection plane according to the position and orientation of the movable display. The three-dimensional virtual world is projected onto a two-dimensional image while changing the positional relationship between the two, and the obtained two-dimensional image is displayed on the movable display. Hereinafter, the outline of the present invention will be described more specifically with reference to the drawings.

  FIG. 13 is a schematic diagram showing a usage state of the image presentation device of the present invention. In FIG. 13, a movable display in the real world is indicated by a black frame, a projected portion of the virtual world is indicated by a solid line, and a non-projected portion is indicated by a dotted line. By using the image presenting apparatus of the present invention, for example, it is possible to “look around”, “move viewpoint”, and “manipulate an object” in a virtual world.

  First, in the state of FIG. 13A, a two-dimensional image obtained by projecting a three-dimensional virtual world according to a certain position and orientation of the movable display is presented on the movable display. From this state, it is considered that the viewpoint position is fixed, and if the position and orientation of the projection plane are changed according to the movement of the movable display, it looks as if the virtual world is being looked through the movable display. “Look around” is possible (FIG. 13B). In addition, from the state of FIG. 13A, when the viewpoint position is also changed in accordance with the movement of the movable display, “movement of viewpoint” can be performed in the three-dimensional virtual world (FIG. 13C). . Further, if the position and orientation of a specific object in the virtual world are also moved in accordance with the movement of the movable display, “object operation” becomes possible (FIG. 13D). As described above, the image presenting apparatus according to the present invention uses a movable display, and changes the position and orientation of the projection plane, the viewpoint position, and / or the position and orientation of the object according to the movement of the movable display. A two-dimensional image obtained by projecting a three-dimensional virtual world is displayed on a movable display.

The present invention is based on visual psychology theory as described below.
(Information extraction theory)
In the human visual system, it is said that invariant terms such as the three-dimensional structure of the object are extracted from the flow pattern of the surrounding light array in the environment caused by “look around” and “movement of the viewpoint”. That is, based on the muscle contraction movement for the movement of the eyeball, head, and trunk, the inertial force applied to the body, the position and orientation of the eyeball is obtained from moment to moment, and the obtained result is obtained on the retina. It is thought that a three-dimensional internal representation of the object is constructed in the head in comparison with the image.

  In the present invention, when a user receives visual information of a virtual world, a display system full of realism is realized by preventing a natural change in the position and orientation of the eyeball. This is also considered in the conventional virtual reality technology using the “head mounted display”.

(Affordance)
When a certain person (A) can perform a certain operation (C) on a certain object (B), “the object (B) affords the operation (C) to the person (A)” Or, “operation (C) is affordance of object (B) for person (A)”. In addition, when the person (A) actually performs the operation (C) on the object (B), “the person (A) perceives the affordance of the operation (C) from the object (B) and uses the affordance. It can be said that

  The present invention makes it easy to use affordances of objects in the virtual world by matching the operation of the object in the virtual world to the user as closely as possible to the operation of the movable display in the real world. ing. What is important when a person manipulates an object is what affordance to use, and the individual forces acting between the finger and the object surface are less important. In that respect, the conventional virtual reality technology using “data glove” or the like that calculates the force one by one costs more calculation cost than necessary. In the present invention, the operations that can be executed are limited to basic operations such as parallel movement and rotational movement, so that a sufficiently practical interactive operation is possible while suppressing the calculation cost. In addition, by making the movable display itself an operation target, it is possible to provide visual feedback in a natural manner as to whether or not an appropriate operation is being performed.

(Visuospatial attention)
In order to make the best use of limited cognitive information, humans are thought to preferentially accept only part of the information in the retinal image, rather than adopting all the visual field information uniformly ( Caution capacity limit theory). When manipulating an object, attention is directed to the movement of the object with the eyes (active attention), and attention is directed to the object when the eyes are drawn (passive attention). In the present invention, instead of simulating the entire field of view as seen in the prior art using a “head-mounted display”, a display with a normal resolution is obtained by simulating only the area where attention is directed. Even if it is used, it does not impair the sense of reality.

  An image presentation apparatus according to claim 1 is an image presentation apparatus that presents a two-dimensional image obtained by projecting a three-dimensional image onto a two-dimensional plane, and a storage unit that stores the three-dimensional image, and a user holds or supports A display whose position and orientation are variable, a projection means for projecting a three-dimensional image stored in the storage means onto a two-dimensional plane, and the projection means according to the position and orientation of the display Determining means for determining the position of the viewpoint and the position and orientation of the projection plane at the time of projection with the projection means, based on the position of the viewpoint determined by the determination means and the position and orientation of the projection plane, A two-dimensional image obtained by projecting a three-dimensional image is displayed on the display, and the determining means includes a viewpoint position and a projection plane position so as to keep the viewpoint position constant. Characterized by being configured to determine the way back.

  An image presentation apparatus according to a second aspect is the image presenting device according to the first aspect, wherein the determining means maintains the relative positional relationship between the position of the viewpoint and the position and orientation of the projection plane constant. The position and orientation are determined.

  According to a third aspect of the present invention, there is provided the image presenting apparatus according to the first or second aspect, further comprising detection means for detecting the position and orientation of the display, and the determining means is a position of the viewpoint according to a detection result of the detection means. And the position and orientation of the projection plane are determined.

  According to a fourth aspect of the present invention, there is provided the image presenting apparatus according to the first or second aspect, further comprising a control unit that controls a position and an orientation of the display, and the determining unit is configured to perform a viewpoint according to a control signal for the display of the control unit. And the position and orientation of the projection plane are determined.

  An image presentation apparatus according to a fifth aspect includes the setting means for setting the position and orientation of the display according to the first or second aspect, and the determination means includes the position and orientation of the display set by the setting means. The position of the viewpoint and the position and orientation of the projection plane are determined according to the above.

  An image presentation apparatus according to a sixth aspect includes the movable vehicle on which the display is mounted according to any one of the first to fifth aspects, and the determining means includes the position and orientation of the display, the position of the movable vehicle, and A feature is that the position of the viewpoint and the position and orientation of the projection plane are determined in accordance with the orientation.

  The image presentation device according to a seventh aspect of the present invention is the image presentation device according to any one of the first to sixth aspects, wherein the three-dimensional image includes a plurality of objects having positions and orientations, and selects one of the objects. Comprising: selecting means; and means for determining the position and orientation of the object selected by the selecting means according to the position of the viewpoint determined by the determining means and the position and orientation of the projection plane. To do.

  FIG. 1 is a block diagram showing a basic configuration of an image presentation apparatus (claim 1) according to the present invention. This image presentation apparatus includes a movable display 1, a three-dimensional storage unit 2, a viewpoint projection plane determination unit 3, and a projection unit 4. The movable display 1 is held by a user or supported by a support, and its position and orientation are variable. The three-dimensional storage unit 2 stores an image of a three-dimensional virtual world. The viewpoint projection plane determination means 3 determines the position of the viewpoint and the position and orientation of the projection plane in the projection means 4 according to the position and orientation of the movable display 1, and determines the determined parameters (viewpoint position and projection plane position). Position and orientation) is output to the projection means 4. The projection unit 4 projects the image of the three-dimensional virtual world stored in the three-dimensional storage unit 2 on the two-dimensional image based on the parameters determined by the viewpoint projection plane determination unit 3, and the two-dimensional image is Presented on the movable display 1.

  The concept of the projection means 4 in the three-dimensional virtual world is shown in FIG. In FIG. 2, A and B are the determined viewpoint and projection plane, respectively, and C indicates a projection target. On the straight line connecting the pixel on the projection plane B and the viewpoint A, the viewpoint equilateral area D on the viewpoint A side from the projection plane B, the viewpoint opposite area E on the opposite side of the viewpoint A from the projection plane B, and the projection plane B itself A point closest to the viewpoint A in the projection target area, which is one or more areas, is projected onto the pixel.

  With such a configuration, a 2D image obtained by projecting a 3D virtual world onto a 2D image while changing the positional relationship between the viewpoint A and the projection plane B according to the position and orientation of the movable display 1 is obtained. The dimensional image can be presented on the movable display 1 and provided to the user. Since a two-dimensional image that changes according to the position and orientation of the movable display 1 is presented, the user can easily and easily enter the three-dimensional virtual world without wearing a device such as a sensor or a display on the body. It can be recognized on the 3D screen.

  Since the viewpoint position determined by the viewpoint projection plane determination means 3 is kept constant, as described above, the three-dimensional virtual world can be viewed through the movable display 1 (from FIG. 13 (a) ( Transition to b)) The situation of looking around from a fixed point in a three-dimensional virtual world can be simulated.

  Further, when determining the parameters by the viewpoint projection plane determination means 3, when the relative positional relationship between the viewpoint position and the position and orientation of the projection plane is kept constant (claim 2), As described above, the viewpoint can be moved in the three-dimensional virtual world (transition from FIG. 13 (a) to (c)), and the viewpoint is changed while fixing the size and shape of the visuospatial attention area. Can simulate the situation.

  FIG. 3 is a block diagram showing a basic configuration of another image presenting apparatus (claim 3) of the present invention. This image presentation apparatus includes a movable display 1, a three-dimensional storage unit 2, a viewpoint projection plane determining unit 3 and a projection unit 4 similar to those in FIG. 1, and a display position / orientation detection for detecting the position and orientation of the movable display 1. Means 5. The viewpoint projection plane determining unit 3 determines the position of the viewpoint and the position and direction of the projection plane when projected by the projection unit 4 according to the position and orientation of the movable display 1 input from the display position / orientation detection unit 5. decide. Thus, for example, when the user operates the movable display 1 with his / her hand, the user's operation on the movable display 1 (direct change of the position and orientation to the movable display 1) is the viewpoint position and the projection plane. The position and orientation of the presented image can be finely adjusted.

  The display position / orientation detecting means 5 is a detecting means using a magnetic sensor or an ultrasonic sensor, and a detecting means using a multi-joint arm device having a function of detecting the angle of each joint of the support of the movable display 1. Any detection means that can detect the position and orientation, such as the gyroscope, the angular velocity measured by the accelerometer, and the means for updating the angle, position, and speed stored in the internal memory according to the acceleration can be used. is there.

  FIG. 4 is a block diagram showing a basic configuration of still another image presentation apparatus according to the present invention. This image presentation apparatus includes a movable display 1, a three-dimensional storage unit 2, a viewpoint projection plane determining unit 3 and a projection unit 4 similar to those in FIG. 1, and a display position / orientation control for controlling the position and orientation of the movable display 1. Means 6. From the display position / orientation control means 6, a control signal for controlling the position and orientation of the movable display 1 is sent to the viewpoint projection plane determination means 3, and according to the control signal, the viewpoint projection plane determination means 3 The position of the viewpoint at the time of projection by the projection means 4 and the position and orientation of the projection plane are determined. Therefore, for example, when the video provider other than the user moves the movable display 1 in accordance with the video story of the three-dimensional virtual world, the viewpoint position and the viewpoint direction of the video recipient who is the user can be guided to the optimum one. .

  FIG. 5 is a block diagram showing a basic configuration of still another image presentation apparatus according to the present invention (claim 5). This image presentation apparatus includes a movable display 1, a three-dimensional storage unit 2, a perspective projection plane determination unit 3 and a projection unit 4 similar to those in FIG. 1, and a display position / orientation setting for setting the position and orientation of the movable display 1 Means 7. A setting pattern for the position and orientation of the movable display 1 is sent from the display position / orientation setting means 7 to the viewpoint projection plane determination means 3, and the viewpoint projection plane determination means 3 uses the projection means 4 according to the setting pattern. The position of the viewpoint at the time of projection and the position and orientation of the projection plane are determined. Therefore, for example, when the movable display 1 is installed on a moving object for which a movement pattern is set, an image that changes according to the movement of the moving object can be presented to the user by the movable display 1.

  FIG. 6 is a block diagram showing a basic configuration of still another image presenting apparatus according to the present invention (claim 6). This image presentation apparatus includes a movable display 1, a three-dimensional storage unit 2, a viewpoint projection plane determination unit 3 and a projection unit 4 similar to those in FIG. 1, and a movable vehicle 8 on which a user is mounted. Information on the position and orientation of the movable vehicle 8 is sent to the viewpoint projection plane determination means 3, and the viewpoint projection plane determination means 3 not only determines the position and orientation of the movable display 1 but also the position and orientation of the movable vehicle 8. Also referring to the information, the position of the viewpoint at the time of projection by the projection unit 4 and the position and orientation of the projection plane are determined. Therefore, for example, it is possible to simulate a situation where the user is on a vehicle in a three-dimensional virtual world.

  FIG. 7 is a block diagram showing a basic configuration of still another image presenting apparatus according to the present invention (claim 7). This image presenting apparatus includes a movable display 1, a three-dimensional storage unit 2, a viewpoint projection plane determination unit 3 and a projection unit 4 similar to FIG. 1, and a virtual object selection unit that selects a specific virtual object in a three-dimensional virtual world. 9 and a virtual object position / orientation determining means 10 for determining the position and orientation of a specific virtual object according to the viewpoint position determined by the viewpoint projection plane determining means 3 and the position and orientation of the projection plane; Have

  When a specific virtual object is selected by the virtual object selection means 9 from among a plurality of virtual objects constituting the 3D world, the selection information is sent to the 3D storage means 2. Further, the position and orientation of the selected specific virtual object are determined by the virtual object position / orientation determining means 10 as the viewpoint position determined by the viewpoint projection plane determining means 3 and the position and orientation of the projection plane. The position / orientation information is sent to the three-dimensional storage means 2. In view of these pieces of information, the three-dimensional virtual world of the three-dimensional storage unit 2 is projected onto the two-dimensional image by the projection unit 4. Therefore, a specific virtual object also moves in accordance with the movement of the movable display 1, and as described above, the object can be operated in the three-dimensional virtual world (from FIG. 13 (a) ( Transition to d)) When the user moves the movable display 1, it is possible to simulate a situation in which the position and orientation of an object in the three-dimensional virtual world are changed.

  In the image presentation apparatus of the present invention, a three-dimensional virtual world is projected onto a two-dimensional image while changing the relationship between the viewpoint and the projection plane according to the position and orientation of the movable display, and the obtained two-dimensional image is presented. As a result, it is possible to provide an intuitive spatial user interface that does not require a device such as a sensor or a display to be mounted on the user's body and does not require the use of a high-resolution display. Moreover, since the position of the viewpoint is kept constant, it is possible to simulate the situation of looking around the virtual world from a fixed point.

  In the image presenting apparatus of the present invention, the relative positional relationship between the position of the viewpoint and the position and orientation of the projection plane is kept constant, so that the viewpoint can be determined while fixing the size and shape of the visuospatial attention area. Can be changed.

  In the image presentation device of the present invention, the position and orientation of the movable display are detected, and the position of the viewpoint and the position and orientation of the projection plane are determined according to the detection result, so that the user can move directly. An image corresponding to the change of the position and orientation of the expression display can be presented.

  In the image presenting apparatus of the present invention, the position of the viewpoint and the position and orientation of the projection plane are determined according to the control signal when controlling the position and orientation of the movable display. It is possible to change the position and orientation of the movable display and to change the position and orientation of the movable display according to the story of the virtual world.

  In the image presenting apparatus of the present invention, the position of the viewpoint and the position and orientation of the projection plane are determined according to the set position and orientation of the movable display, so that the user can follow the movement of the movable display. The position and orientation of the movable display can be set.

  In the image presenting apparatus of the present invention, since the image on the movable display is presented to the user on the vehicle, it is possible to simulate riding on the vehicle in the virtual world.

  In the image presenting apparatus of the present invention, a specific object in the virtual world is selected and its position and orientation are determined, so that the position and orientation of the specific virtual object can be changed by moving the movable display. .

  Hereinafter, the present invention will be described in detail with reference to the drawings showing embodiments thereof.

(Embodiment 1)
FIG. 8 is a front view of a small virtual reality terminal as an example of the present invention that presents a three-dimensional virtual world as a two-dimensional image. As the movable display 1, a flat liquid crystal display 11 having a certain size that can be freely operated by a user's hand is used. Further, as the display position / orientation detection means 5, a magnetic sensor 12 installed in the virtual reality terminal and a magnetic generator 13 fixed at one point in the real world are used. Based on the detection result of the sensor 12, the position and orientation of the liquid crystal display 11 (relative position and orientation with respect to the magnetism generator 13) are detected. Further, the virtual reality terminal is provided with a later-described viewpoint change button 14 and an object operation button 15 as auxiliary input means.

  The projection target areas in the projection unit 4 are all areas of the viewpoint same-side area, the viewpoint opposite-side area, and the projection surface itself (see FIG. 2). In consideration of the hardware characteristics of the liquid crystal display 11, the luminance of the image is converted according to the position of the viewpoint and the position and orientation of the projection plane. That is, the display surface of the liquid crystal display 11 becomes dark when viewed from an oblique direction. In such a case, the display surface is slightly brightened. The viewpoint projection plane determination means 3 makes the position and orientation of the projection plane equal to the position and orientation of the liquid crystal display 11 in the real world.

  When the viewpoint change button 14 is not pressed, the viewpoint position is kept constant. In this case, the position of the fixed viewpoint is a position away from a predetermined value (several tens of centimeters) in front of the liquid crystal display 11 at the start of use of the virtual reality terminal, and when the viewpoint change button 14 is finally released otherwise. The position of the viewpoint. The user fixes his eyes (viewpoint) at such a fixed viewpoint position. The user looks around the virtual world of the part through the peephole by moving the liquid crystal display 11 to a position where attention is directed while keeping the viewpoint change button 14 released and keeping the viewpoint unchanged. I can simulate that.

  When the viewpoint change button 14 is pressed, the relative positional relationship between the position of the viewpoint and the position and orientation of the projection plane is set to be kept constant. While pressing the viewpoint change button 14, the user moves the liquid crystal display 11 to a position where attention is to be paid while fixing the relative positional relationship between the liquid crystal display 11 and his / her viewpoint. You can change the position of the viewpoint in the world.

  When the object operation button 15 is pressed, the virtual object selecting unit 9 selects a virtual object on which the pointer displayed at a fixed position on the liquid crystal display 11 overlaps. The virtual object position / orientation determination means 10 projects the center of gravity of a preset virtual object onto the projection plane, and if the projected point on the projection plane is used as the center of gravity projection point, the center of gravity projection point moves on the projection plane. The position and orientation of the virtual object so that the distance from the viewpoint to the center of gravity is proportional to the distance from the viewpoint to the center of gravity projection, and the relative relationship between the orientation of the virtual object and the orientation of the projection plane is kept constant. To decide. With the object operation button 15 released, the user moves the liquid crystal display 11 so that the virtual object to be operated is projected at the position where the pointer is, and then presses the object operation button 15 to select the virtual object. By moving the liquid crystal display 11 while the operation button 15 is pressed, the position and orientation of the virtual object can be changed.

  As a specific example, an operation of directing the document S placed on the desk in the virtual world to this side will be described with reference to FIG. 9 showing a user operation procedure during the operation. 9A to 9C show changes in the user's visual field over time. First, without pressing the object operation button 15, the liquid crystal display 11 is moved to find out where the desired document S is. If the document S is found, the pointer on the liquid crystal display 11 is set to the position of the document S (FIG. 9A). Thereafter, the orientation of the liquid crystal display 11 is adjusted so that the orientation of the liquid crystal display 11 and the orientation of the document S presented on the screen are equal in appearance, and here, the object operation button 15 is pressed to select the document (FIG. 9 (b)). Then, if the liquid crystal display 11 is turned to the front with the object operation button 15 kept pressed, the document S in the virtual world is also presented facing the front (FIG. 9C). Conventionally, complicated calculation is required by detecting the angle of the document, but in the present invention, the operation on the document S can be performed very intuitively.

  For each object in the virtual world, restrictions on movement such as “can move only along a certain surface” and “a certain axis always faces a certain direction” are designated in advance. Therefore, when the user tries to move the virtual object beyond the specified limit, the virtual object is moved by reinterpreting the movement within the limit range. For example, when a cup is placed on a table, even if the user tries to move the cup below the table surface, the cup is not embedded in the table surface, but is interpreted as a two-dimensional movement on the table surface. Move on the table surface. Further, by limiting the movement of the automobile to two-dimensional movement on the ground and limiting the movement of the train to one-dimensional movement on the track, a sense of reality can be expressed more.

  By the way, when the position and orientation of an object in the virtual world change, the following secondary changes are also presented to further emphasize the sense of reality. For example, when tilting a glass containing water, if it is tilted slowly, water will flow out of the glass, but if it is tilted quickly, water will flow out of the glass vigorously. When the mirror is moved, a scene reflected in the mirror is dynamically generated and presented according to the movement of the mirror. Further, when moving the luminaire, the state of the surrounding scene is changed according to the movement of the luminaire.

  Also, by providing a consistent secondary effect to a virtual object provided by the virtual world provider, the user can use the virtual object as a tool. For example, the user can perform engraving in the virtual world by selecting the engraving sword with the object operation button 15 and moving the liquid crystal display 11 to operate the virtual engraving sword.

(Embodiment 2)
Next, an example of a terminal suitable for medical image processing will be described. The apparatus configuration and basic usage are the same as those in the first embodiment. However, the projection target area in the projection unit 4 is limited to the viewpoint opposite side area and the projection plane, and the portion of the viewpoint same side area is not projected. As a result, when the projection plane passes through the target virtual object, an image obtained by cutting the virtual object on the plane can be obtained. FIG. 10 shows a presentation example in which a cubic virtual object is cut. By doing so, for example, when the target object is the brain, the appearance of the brain is presented when the liquid crystal display 11 is separated, and the cross section of the brain can be seen when the liquid crystal display 11 is brought closer.

  In the conventional method for presenting the cut surface, it is necessary to independently adjust the parameters related to the position and orientation of the cut surface, but in the second embodiment, many parameters can be intuitively fine-tuned simultaneously. . In addition, when this method is used, the internal structure of an object having a complicated three-dimensional shape can be expressed even outside the medical field.

(Embodiment 3)
Next, an example using a wide screen two-dimensional GUI in a portable terminal will be described. As a result, a standard GUI can be used instead of a user interface specialized for portable terminals. In other words, considering a vast two-dimensional GUI plane in a virtual three-dimensional world and “looking into” it using a movable display, the GUI plane can be scrolled and enlarged / reduced intuitively. .

  As the movable display 1, a flat liquid crystal display having a size that can be freely operated by a user by hand is used. The display position / orientation detection means 5 captures an image with a CCD camera fixed to the display, and uses image analysis techniques such as optical flow calculation, tracking of a standard object or a user's face, and so on. Detect the position and orientation. In the virtual object selection means 9, the mouse pointer on the GUI screen is always selected. The virtual object position / orientation determining means 10 determines the position and orientation so that the mouse pointer always moves on the two-dimensional GUI plane and the projection point does not move on the projection plane. Note that the movable display 1 is provided with a button having the same function as the mouse button, and when it is necessary to press the mouse button, the button is pressed.

(Embodiment 4)
Next, an example in which the movement of the movable display is limited to rotation will be described. The movable range of the movable display is limited to horizontal rotation and vertical rotation. FIG. 11 is a schematic diagram of the terminal device according to the fourth embodiment. This terminal device is like a notebook personal computer, and includes a liquid crystal display 11 as a movable display 1, and rotation angle detectors 16a and 16b such as a volume for detecting the rotation angle of the liquid crystal display 11 in the horizontal and vertical directions. And motors 17a and 17b for controlling the rotation angles of the liquid crystal display 11 in the horizontal direction and the vertical direction, respectively. The rotation angle detectors 16a and 16b are used as the display position / orientation detection means 5, and the motors 17a and 17b are used as the display position / orientation control means 6.

  In the viewpoint projection plane determining means 3, the viewpoint position is always kept at a position separated by a predetermined value in front of the liquid crystal display 11. When the user directly operates the liquid crystal display 11, the position and orientation of the projection plane are determined according to the position and orientation detected by the display position / orientation detection means 5 (rotation angle detectors 16a and 16b). When the user indirectly operates the liquid crystal display 11 through the auxiliary input device or when the liquid crystal display 11 is moved in accordance with a virtual world event, the display position / orientation control means 6 (motors 17a, 17b The position and orientation of the projection plane are determined in accordance with the control signal used in ().

  In the fourth embodiment, it is possible to display a panoramic view of a bird's eye view and look around a virtual object existing in the vicinity of the projection plane. For example, it is possible to present an image obtained by looking around a camellia, a vase or the like from various angles.

(Embodiment 5)
Next, an example of a globe-type terminal device will be described. In this terminal device, an image as if a specific virtual object was rotated can be presented.

  In the movable display 1, a spherical screen is arranged instead of the globe, and an image is presented there by a liquid crystal projector. In the projection unit 4, the position of the liquid crystal projector and the position and shape of the spherical screen are taken into consideration, and an image is generated so that a part of the virtual world can be seen without distortion from the position of the viewpoint. The viewpoint projection plane determination unit 3 keeps the viewpoint position constant. The display position / orientation detection means 5 detects the rotation angle of the spherical screen in the horizontal and vertical directions using a rotation angle detector such as a volume. The virtual object selection means 9 always selects one virtual object. In the virtual object position / orientation determining means 10, the center of gravity of the preset virtual object is positioned at the center of the spherical projection plane, and the relative relationship between the orientation of the virtual object and the spherical projection plane is kept constant. Thus, the position and orientation of the virtual object are determined.

(Embodiment 6)
Next, an example of a virtual spotlight type terminal device will be described. In this terminal device, it is possible to present an image as if the virtual world was illuminated with a spotlight. FIG. 12 is a schematic diagram of the terminal device according to the sixth embodiment. As the movable display 1, a liquid crystal projector 18 having a projection unit 19 and a size that can be freely operated by a user by hand, and a screen (not shown) fixed in the real world are used. As the display position / orientation detection means 5, a magnetic sensor 12 installed in the liquid crystal projector 18 and a magnetism generator (not shown) fixed at one point in the real world are used. A viewpoint change button 14 is provided on the liquid crystal projector 18.

  The projection target areas in the projection unit 4 are all the areas of the viewpoint same side area, the viewpoint opposite side area, and the projection plane itself. The viewpoint projection plane determining means 3 calculates the position and orientation at which the image from the liquid crystal projector 18 is projected based on the screen position, and determines the position and orientation of the projection plane to be equal to the calculation result.

  Then, in the viewpoint projection plane determination means 3, when the viewpoint change button 14 is not pressed, the viewpoint position is kept constant. In this case, the fixed viewpoint position is set to a position that is a predetermined distance above the liquid crystal projector 18 at the start of use of the terminal device, and otherwise to the position of the viewpoint when the viewpoint change button 14 is finally released. . With the viewpoint change button 14 released, the user operates the liquid crystal projector 18 so as to illuminate the part to which attention is to be directed while keeping his viewpoint unchanged. You can simulate looking around through a hole.

  On the other hand, in the viewpoint projection plane determination means 3, when the viewpoint change button 14 is pressed, the relative positional relationship between the viewpoint position and the projection plane position and orientation is set to be constant. The user operates the liquid crystal projector 18 so as to illuminate the part to which attention is directed while fixing the relative positional relationship between the liquid crystal projector 18 and his / her viewpoint while pressing the viewpoint change button 14. Thus, the position of the viewpoint can be changed in the virtual world.

  In the virtual object selection means 9, a spotlight in the virtual space is always selected instead of a specific virtual object. The virtual object position / orientation determining means 10 matches the position and orientation of the spotlight with the position and orientation of the liquid crystal projector 18. In the illumination state, light is applied from a virtual spotlight.

(Embodiment 7)
Next, application examples of image presentation to the user will be shown. By moving the movable display 1 according to the story of the image on the image providing side, the position of the viewpoint and the direction of the line of sight on the image receiving side (user) can be guided to an optimum one.

  As the display position / orientation control means 6, an arm device is used in which the angle of each joint can be controlled by a motor. The viewpoint projection plane determining means 3 keeps the relative positional relationship between the viewpoint position and the projection plane position and orientation constant. When the user accepts an image, the user can optimize the viewpoint position and the line-of-sight direction for the image by holding his / her viewpoint at a position that is a predetermined value away from the front of the movable display 1. .

(Embodiment 8)
Next, an example of a virtual elevator scene using the elevator as a movable vehicle will be described. Users on the elevator can enjoy the scenery by presenting the scenery of various parts of the world or a fantastic virtual world according to the vertical movement of the elevator on the movable display 1 installed on the wall of the elevator. Can do. Further, even in a high-speed elevator of a skyscraper in which windows cannot be installed, the speed of the elevator can be realized from the change in the scenery presented on the movable display 1.

  Information on the direction and position of the movable display 1 required in the viewpoint projection plane determining means 3 is calculated by measuring the acceleration of the elevator with an accelerometer as the display position / orientation detecting means 5, or the guide role of the elevator is Depending on the operation, it is directly input by an input device as the display position / orientation setting means 7 or an elevator control signal in the elevator drive system as the display position / orientation control means 6 is used. In the viewpoint projection plane determining means 3, the position of the viewpoint is set to a point separated from the front of the movable display 1 by an appropriate distance, and the position and orientation of the projection plane are made to coincide with the position and orientation of the movable display 1.

  Further, similar to the example of the virtual landscape presentation in the elevator as described above, the movable display 1 is arranged on the amusement park vehicle or the train wall, and the landscape image is presented in accordance with the movement of the vehicle. It is possible to make the vehicle feel as if it is moving in a virtual world.

(Embodiment 9)
Next, an example of a newspaper browsing terminal is shown. The system configuration is obtained by adding the function of saving the position of the viewpoint and the position and orientation of the projection plane every moment to the configuration of the small virtual reality terminal shown in the first embodiment. These pieces of information are stored in the internal memory of the viewpoint projection plane determination unit 3.

  It is assumed that a virtual newspaper in which each article is arranged so that the characters become smaller in the order of a headline, a summary, and a body text is displayed in the virtual three-dimensional world. The user peeks at the virtual newspaper through the newspaper browsing terminal. By moving the liquid crystal display 11, the user can browse the headlines and summaries if the projection plane is separated from the newspaper and the viewpoint is close, and the headline and summary are read. If you look in, you can read the text in high resolution.

  The trajectory of the viewpoint position and the position and orientation of the projection plane saved during browsing execution represents a history of recording where the user tried to pay attention in the virtual world. By comparing this trajectory with the placement of each newspaper article in the virtual world, it is possible to calculate to what level and how much time the user has read each article. Based on this value, the user's interest in each article is quantitatively estimated, and the interest value is stored in association with each article. By using this interest value, it is possible to select an article with high interest and display a detailed explanation, or to search only articles in a field with high interest.

(Embodiment 10)
Next, an example of a virtual reality terminal with a binocular stereoscopic function will be described. A binocular stereoscopic video presentation method such as a liquid crystal shutter method or an anaglyph method is added to the configuration of the virtual reality terminal shown in the first embodiment.

  The viewpoint projection plane determination means 3 has an internal memory for storing the positions of two viewpoints for the right eye and for the left eye. For each of the right eye and the left eye, the position of the viewpoint is determined every moment in the same manner as in the first embodiment, and the virtual world viewed from the position of each viewpoint is projected onto the projection plane, and two images obtained Is presented as a stereo image on the movable display 1. The user uses this terminal while moving his / her head to bring his / her eyes to the positions of the two viewpoints. The position of the viewpoint when the terminal starts to be used is a position that is a certain distance away from the position that is a predetermined value away from the front of the movable display 1 along the straight line parallel to the upper and lower sides of the movable display 1. It should be noted that the distance between the left and right viewpoints needs to be adjusted to match the distance between the user's own left and right eyes.

  When such a virtual reality terminal with a binocular stereoscopic function is used, it is expected that the stereoscopic effect of objects in the virtual world will be further improved by a synergistic effect of motion parallax and binocular stereoscopic vision.

It is a block diagram which shows the basic composition of the image presentation apparatus of this invention. It is a schematic diagram which shows the projection concept of a projection means. It is a block diagram which shows the basic composition of the other image presentation apparatus of this invention. It is a block diagram which shows the basic composition of the further another image presentation apparatus of this invention. It is a block diagram which shows the basic composition of the further another image presentation apparatus of this invention. It is a block diagram which shows the basic composition of the further another image presentation apparatus of this invention. It is a block diagram which shows the basic composition of the further another image presentation apparatus of this invention. It is a front view of the small virtual reality terminal by this invention. It is a schematic diagram which shows an example of the user's operation procedure in this invention. It is a figure which shows the example of presentation of the cut surface of an object. It is a schematic diagram of a rotation limited type terminal device according to the present invention. It is a schematic diagram of a virtual spotlight type terminal device according to the present invention. It is a figure for demonstrating the outline | summary of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Movable display 2 3D memory | storage means 3 View point projection surface determination means 4 Projection means 5 Display position / orientation detection means 6 Display position / orientation control means 7 Display position / orientation setting means 8 Movable vehicle 9 Virtual object selection means
10 Virtual object position / orientation determination means
11 LCD display
12 Magnetic sensor
13 Magnetic generator
14 Viewpoint change button
15 Object operation button
16a, 16b Rotation angle detector
17a, 17b motor
18 LCD projector
19 Projector

Claims (7)

  In an image presentation apparatus that presents a two-dimensional image obtained by projecting a three-dimensional image onto a two-dimensional plane, the storage means for storing the three-dimensional image, and the position and orientation held by the user or supported by the support are variable. A projection unit that projects a three-dimensional image stored in the storage unit onto a two-dimensional plane, and a viewpoint position and projection at the time of projection by the projection unit according to the position and orientation of the display 2 which is obtained by projecting a three-dimensional image by the projecting means based on the position of the viewpoint and the position and orientation of the projection plane determined by the determining means. The dimensional image is configured to be displayed on the display, and the determination unit is configured to determine the position of the viewpoint and the position and orientation of the projection plane so as to keep the position of the viewpoint constant. Image displaying apparatus according to claim and.   The determining means is configured to determine the position of the viewpoint and the position and orientation of the projection plane so that the relative positional relationship between the position of the viewpoint and the position and orientation of the projection plane is kept constant. The image presentation apparatus according to claim 1.   A detection unit configured to detect the position and orientation of the display, wherein the determination unit is configured to determine the position of the viewpoint and the position and orientation of the projection surface according to a detection result of the detection unit; The image display device according to claim 1, wherein the image display device is an image display device.   Control means for controlling the position and orientation of the display is provided, and the determining means is configured to determine the position of the viewpoint and the position and orientation of the projection surface in accordance with a control signal for the display of the control means. The image presenting apparatus according to claim 1, wherein the image presenting apparatus is an image presenting apparatus.   Setting means for setting the position and orientation of the display; and the determining means determines the position of the viewpoint and the position and orientation of the projection plane according to the position and orientation of the display set by the setting means. The image presenting apparatus according to claim 1, wherein the image presenting apparatus is configured as described above.   A movable vehicle on which the display is mounted is provided, and the determining means determines the position of the viewpoint and the position and orientation of the projection plane according to the position and orientation of the display and the position and orientation of the movable vehicle. The image presenting apparatus according to claim 1, wherein the image presenting apparatus is configured as described above.   The three-dimensional image includes a plurality of objects having positions and orientations, and a selection unit that selects one of the objects, and the position and orientation of the object selected by the selection unit are determined. The image presentation apparatus according to claim 1, further comprising: a unit that determines the position of the viewpoint determined by the unit and the position and orientation of the projection plane.

Images