JP2004062755A - Image providing device and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable a user to limit a traveling path in walk-through in a virtual space using a panoramic real image. <P>SOLUTION: An image providing device is provided with: a panoramic real image storing means holding a plurality of panoramic real images; a movable path setting means for setting movable path information; a movable path storing means holding a plurality of pieces of the movable path information; an operating part; a movable path selecting means for selecting desired movable path information from the movable path storing means; a viewpoint calculating means for calculating viewpoint information consisting of viewpoint position information and visual line direction information on the basis of operation information sent from the operating means and the movable path information; an image generating means for generating a segmented image from the panoramic real images on the basis of the viewpoint information, generating a path drawing representing the movable path on the basis of the movable path information and combining the segmented image and the path drawing to generate a display image; and an image display part for displaying the display image. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention enables a user of a walk-through system using a panoramic photographed image to switch or set a movable route.
[0002]
[Prior art]
Attempts have been made to photograph a real space with a photographing device mounted on a moving body and to represent the photographed real space as a virtual space using a computer based on photographed real image data (for example, Endo, Katayama, Tamura, Hirose, Watanabe, Tanigawa: "On Cyber-Electronic Imaging of Urban Space Using a Camera Mounted on a Mobile Vehicle" (Research Society, PA-3-4, pp. 276-277, 1997), or Hirose, Watanabe, Tanigawa, Endo, Katayama, Tamura: "Construction of a cyberspace urban space using a camera mounted on a mobile vehicle (2)-Creation of a wide-area virtual space using real images-" Pp. 67-70, 1997).
[0003]
As a method of expressing the photographed real space as a virtual space based on the real image data captured by the imaging device mounted on the moving object, a geometric model of the real space is reproduced based on the real image data. However, there is a method of expressing by a conventional CG technique, but there is a limit in terms of accuracy, precision, and realism of the model. On the other hand, Image-Based Rendering (IBR) technology for expressing a virtual space using a real image without performing reproduction using a model has recently attracted attention. The IBR technology is a technology for generating an image viewed from an arbitrary viewpoint based on a plurality of real images. Since the IBR technology is based on a real image, it is possible to represent a realistic virtual space.
[0004]
In order to construct a walkable virtual space using such an IBR technology, it is necessary to generate and present an image according to the position of the user in the virtual space. Therefore, in this type of system, each frame of the real image data and the position in the virtual space are stored in association with each other, and the corresponding frame is obtained based on the position in the virtual space and the gaze direction of the user. Play this.
[0005]
As a method of obtaining position data in the real space, it is general to use a positioning system using an artificial satellite represented by GPS (Global Positioning System) which is also used in a car navigation system and the like. . As a method of associating position data obtained from a GPS or the like with actual image data, a method of associating the data with a time code has been proposed (Japanese Patent Laid-Open No. 11-168754). In this method, the time data included in the position data is associated with the time code added to each frame of the actually shot image data, so that each frame of the actually shot image data is associated with the position data.
[0006]
In such a walkthrough in the virtual space, the user can see a desired direction at each viewpoint position. For this reason, the image at each viewpoint position is stored as a panoramic actual image covering a wider range than the angle of view at the time of reproduction, and the panoramic actual image is stored based on the viewpoint position and the viewing direction in the user's virtual space. It is conceivable to cut out a partial image to be reproduced and display it.
[0007]
[Problems to be solved by the invention]
However, a route that can be moved when walking through such a virtual space is fixed to a route set in advance by a designer, and the walkthrough in that route is not limited. Therefore, for example, when both the outdoor space and the indoor space are photographed and both routes are included in the movable route, a user who desires a walk-through only outdoors enters the room by mistake. There was such a case. In some cases, a user who wishes to walk through only the main street accidentally enters a narrow alley.
[0008]
The present invention has been made in view of the above circumstances, and in a walk-through in a virtual space using a panoramic real image, a user can switch or set a movable path to limit a traveling path. The purpose is to do.
[0009]
Another object of the present invention is to enable a user to easily determine a route that cannot be moved.
[0010]
[Means for Solving the Problems]
In order to achieve at least one of the above objects, for example, an image presentation device of the present invention has the following configuration.
[0011]
The invention according to claim 1 of the present application provides a panoramic real image storage unit that stores a plurality of real panoramic images, a movable path setting unit that sets movable path information, and a movable path that stores a plurality of the movable path information. Storage means, an operation unit capable of reflecting the user's intention, a movable path selecting means for selecting desired movable path information from the movable path storage means, Viewpoint calculation means for calculating viewpoint information comprising viewpoint position information and gaze direction information based on the operation information and the movable route information, and generating and displaying a cutout image from the panoramic real image based on the viewpoint information It is characterized by comprising an image generating means for generating an image, and an image display unit for displaying the display image.
[0012]
In the invention according to claim 10 of the present application, the image generating means generates a route graphic representing a movable route based on the movable route information, and generates a display image by combining the cut image and the route graphic. It is characterized by doing.
[0013]
The invention according to claim 15 of the present application inputs a user's instruction using an operation unit, calculates viewpoint information including viewpoint position information and gaze direction information based on the input instruction and movable path information, An information processing method for generating a cut-out image from a plurality of held panoramic real images based on viewpoint information based on viewpoint information, wherein the movable path information can be edited. I do.
[0014]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.
[0015]
[First Embodiment]
In the present embodiment, an image presenting apparatus that allows a user of a walk-through system using a panoramic photographed image to switch a movable path will be described.
[0016]
FIG. 1 is a block diagram showing a schematic configuration of the image presentation device according to the first embodiment of the present invention.
[0017]
Reference numeral 1 denotes a panoramic photographed image storage unit that stores panoramic photographed images at a plurality of points along a route such as a road. The panoramic photographed image is stored in a medium such as a hard disk, tape, or memory. The group of panoramic photographed images is assigned a frame number, for example, in order from the top. Each real panoramic image may be divided into a plurality of divided images and held. The real panoramic image stored in the real panoramic image storage means 1 is synthesized from a video taken along a route such as a road by one or more video cameras arranged so as to be able to take pictures in all directions, for example, It is associated with the position information and the posture information of the point.
[0018]
Reference numeral 2 denotes a photographing path storage unit, which stores photographing path information indicating a path at the time of photographing the group of panoramic photographed images stored in the panoramic photographed image storage unit 1 (hereinafter, referred to as a photographing path). The photographing route information is stored in a medium such as a hard disk, tape, or memory. The photographing route information includes, for example, a point that separates the photographing route (hereinafter, referred to as a branch point) and a line segment (hereinafter, referred to as a route) connecting the branch point. At this time, the branch point and the route are assigned IDs that can uniquely specify each of them. The line connecting the branch points is a straight line or a curve between two branch points, that is, a section.
[0019]
Various methods are conceivable for describing the branch point and the route. In the present embodiment, it is assumed that the branch point and the route are described in a form as shown in FIGS. 2A and 2B. FIG. 2A is a diagram illustrating a description example of a branch point. FIG. 2B is a diagram illustrating a description example of a route. As shown in FIG. 2A, in the branch point list 200, for each branch point, a branch point ID 201 for specifying the branch point, coordinate data 202 representing the position, and a name 203 given to the branch point are described. . The name data may or may not be given. Further, as shown in FIG. 2B, in the route list 210, for each route, a route ID 211, a start branch point 212, an end branch point 213 indicating branch points at both ends of the route, and a route name 214 are described. Is done. The name data may or may not be given. Note that the start frame number 215 and the end frame number 216 are the panoramic real shot images respectively corresponding to the start branch point 212 and the end branch point 213 specified based on the position and orientation information of the shooting point associated with the panoramic real shot image. Is the frame number.
[0020]
Note that the association between the branch point and the frame number is automatically performed according to the position and orientation information of the shooting point, but the user can manually associate the frame point with the frame number. In this case, if the position information of each of the panoramic real images in the panoramic real image group sandwiched between the branch points is assigned, for example, at equal intervals on a line connecting the branch points, the position and orientation information of the shooting point can be obtained. It is possible to generate and hold the association data without using it.
[0021]
Note that the shooting route information may further describe attribute information of each route. The attribute information can describe, for example, the width of the shooting route, information indicating whether the shooting route is indoor or outdoor, and the like.
[0022]
Reference numeral 3 denotes a movable path setting unit that sets a movable path based on the photographing path information held in the photographing path storage unit 2 and sends the path to the movable path storage unit 4. The movable route information is set, for example, by extracting all or a part of the photographing route information. In the movable route information, for example, a branch point and a route are described in a form as shown in FIGS. 2A and 2B.
[0023]
The extraction of the movable route information can be realized by, for example, editing data in the above-described form representing the photographing route information using a text editor or the like. In addition to the above, for example, a map image storage unit (not shown) is further provided, a route graphic representing a photographing route is synthesized on a map image, and a partial graphic to be set as a movable route on the image is converted by a device such as a mouse. By specifying, it is also possible to extract the movable route information. When attribute information of each route is described in the photographing route information, it is also possible to extract movable route information by automatic processing based on the attribute information.
[0024]
It is also possible to set new movable route information based on the movable route information already stored in the movable route storage means 4.
[0025]
Reference numeral 4 denotes a movable route storage unit which stores a plurality of movable route information set by the movable route setting unit 3. The movable path information is stored in a medium such as a hard disk, a tape, or a memory.
[0026]
Various methods are conceivable for describing the movable route information. In the present embodiment, it is assumed that the information is described in a form as shown in FIG. 10, for example. As shown in FIG. 10, in the movable route information list 1000, for each movable route information, a movable route information ID 1001 for specifying the same, a route ID 1002 constituting the same, and the movable route information are added to the movable route information. The name 1003 is described. The name data may or may not be given.
[0027]
Reference numeral 5 denotes an operation unit, which includes, for example, a joystick. The operation of the operation unit 5 is manually performed by the user, and the user can reflect the intention. Through the operation of the operation unit 5, selection of the movable route information to be used, designation of a panorama actual photographed image as a basis for image generation, designation of a portion to be cut out from the panorama actual photographed image, and the like are performed. The operation information of the operation unit 5 is sent to the movable route selection unit 6 and the viewpoint calculation unit 7. The operation unit 5 is not limited to the joystick, and may be any device capable of selecting movable path information to be used, specifying a panoramic real image to be used for image generation, and specifying a portion to be cut out from the panoramic real image. Anything may be used, for example, a mouse, a trackball, a keyboard, or the like.
[0028]
Reference numeral 6 denotes a movable route selection unit that selects desired movable route information from a plurality of movable route information stored in the movable route storage unit 4. For example, when the operation information of the button press is transmitted from the operation unit 5, the movable path information corresponding to the button number is selected. Alternatively, a list of movable route information is displayed when the operation information of the button press is transmitted from the operation unit 5, and when another operation information of the button press is transmitted, the movable route information according to the number of button presses is displayed. Can be selected. It is to be noted that, by further having an operation unit different from the operation unit 5 and operating it, a desired movable path information is selected from a plurality of movable path information by, for example, the same method as described above. Is also possible.
[0029]
Further, for example, when the button of the operation unit 5 is not pressed, the default movable path information is selected, and the button is switched to the movable path information corresponding to the button number only while the button is pressed. You can also. For example, when both the outdoor space and the indoor space are photographed, while the button A of the operation unit 5 is pressed, the walk-through only in the outdoor space is enabled, and the button B of the operation unit 5 is pressed. During this operation, walk-through of only the indoor space can be performed, and when no button is pressed, walk-through of both the outdoor space and the indoor space can be performed.
[0030]
In addition, it is possible to limit the selectable movable route information based on the viewpoint information calculated by the viewpoint calculation means 7. For example, if it is possible to select only the movable path information such that the current viewpoint is present on the path from a plurality of movable path information, the viewpoint is separated from that before the switching when the movable path is switched. It is possible to prevent the position from being set.
[0031]
Reference numeral 7 denotes a viewpoint calculation unit that calculates viewpoint information including viewpoint position information and line-of-sight direction information based on the operation information received from the operation unit 5 and the movable path information selected by the movable path selection unit 6. It is sent to the image generation means 8. Here, the viewpoint position information can be made to match any one of the position information of each panoramic photographed image that can be calculated based on the movable route information, for example. In that case, the viewpoint position can be set on the movable route. In this case, the viewpoint position information can be represented by the frame number of the panoramic photographed image.
[0032]
Reference numeral 8 denotes an image generation unit which obtains a panorama real image from which the image is to be generated from the panorama real image storage unit 1 in accordance with the viewpoint information sent from the viewpoint calculation unit 7 and cuts out the image according to the viewpoint information. Generate an image. In generating the cut-out image, for example, a process as shown in FIG. 3 is performed. First, a panoramic real image as a basis for image generation is obtained from the panoramic real image storage unit 1. At this time, only the divided images necessary for image generation may be acquired using the line-of-sight direction information. Next, according to the line-of-sight direction information, a necessary partial image is cut out from the panoramic image or the divided image. Then, the cut-out partial image is texture-mapped on a cylindrical surface and subjected to perspective transformation processing, thereby generating a center projected image corresponding to the viewpoint information.
[0033]
Further, the image generating means 8 generates a path graphic representing the movable path based on the movable path information selected by the movable path selecting means 6 and combines it with the cut-out image to generate a display image. Here, a map image storage means (not shown) is further provided, and a route graphic representing a movable route on the map image is synthesized as shown in FIG. 4, and further synthesized with the above-described cut-out image and presented to the user. May be.
[0034]
Here, based on the movable route information sent from the movable route selecting means 6 and the viewpoint information sent from the viewpoint calculating means 7, a planned traveling route at the next branch point, which is a part of the route graphic, is determined. By changing the attributes such as the color of the partial figure to be represented, it is possible to easily recognize the planned route. Further, as shown in FIG. 5, by changing attributes such as a color of a partial graphic representing a currently traveling route, which is a part of the route graphic, the current traveling route can be easily recognized. It is possible. Further, a graphic such as an arrow indicating the current viewpoint position and the line of sight may be drawn on the route graphic. In addition, it is also possible to generate a route graphic representing a shooting route with an attribute different from the above-described attributes such as the color, and combine the generated image with the above-described display image.
[0035]
Further, by rotating the entire route graphic according to the user's line-of-sight direction information, the traveling direction can be easily recognized. The route graphic is not limited to a straight line or the like, and may be any graphic that represents a travelable route, a scheduled travel route, or a photographing route. In addition, the route graphic can be replaced with an image, a character, or the like.
[0036]
Reference numeral 9 denotes an image display unit, which includes, for example, a projector and a screen, and acquires and displays a display image generated by the image generation unit 8. The image display unit 9 is not limited to a projector and a screen, but may be any medium capable of displaying an image, for example, a CRT or a liquid crystal monitor. Further, a plurality of image display units 9 may be provided, in which case the user can observe a wide-field image. In this case, the number of the image generating means 8 and the number of the panoramic photographed image storing means 1 are the same as the number of the image display units 9, for example. Here, a route graphic representing a movable route may be displayed on some or all of the plurality of image display units.
[0037]
The control procedure of the first embodiment having the above configuration will be described below. FIG. 6 is a flowchart illustrating a procedure of a process in the image presentation device of the present invention.
[0038]
In step S601, data preparation is performed. In step S601, a panoramic photographed image stored in the panoramic photographed image storage unit 1 and photographing route information stored in the photographing route storage unit 2 are prepared. Further, the movable path setting means 3 sets movable path information based on the imaging path information stored in the imaging path storage means 2 and stores the information in the movable path storage means 6.
[0039]
After data preparation, the system is started in step S602. In step S603, the operation unit 5 is operated, and the operation information is sent to the movable route selection unit 6 and the viewpoint calculation unit 7.
[0040]
Next, in step S604, when the selection of the movable route information has been instructed by the operation of the operation unit 5, the movable route selection unit 6 stores the plurality of movable routes stored in the movable route storage unit 4. One movable route information is selected from the information.
[0041]
Next, in step S605, the viewpoint calculation unit 7 calculates viewpoint information based on the operation information received from the operation unit 5 and the movable path information selected by the movable path selection unit 6, and generates the calculated viewpoint information into an image. It is sent to the means 8.
[0042]
Next, in step S606, the image generation unit 8 acquires a panorama real image from which the image is to be generated from the panorama real image storage unit 5 in accordance with the viewpoint information sent from the viewpoint calculation unit 7, and responds to the viewpoint information. Generate a clipped image. Further, based on the movable route information sent from the movable route selection means 6, a route graphic representing a movable route is generated and combined with the cut-out image to generate a display image.
[0043]
Next, in step S607, the display image generated by the image generation unit 8 is displayed on the image display unit 9.
[0044]
Thereafter, it is checked in step S608 whether the system is to be terminated. If the system is to be terminated, the system is terminated in step S609. If not, the process returns to step S603, and the above processing is repeated.
[0045]
In this embodiment, the case where the path graphic representing the movable path is combined with the cutout image is described. However, the present invention is not limited to this. The path graphic representing the movable path is combined with the cutout image. Instead, the image may be displayed on a display unit different from the image display unit 9.
[0046]
Further, the display position and the size of the route graphic may be operated. Further, display / non-display of the route graphic may be switched.
[0047]
As described above, according to the first embodiment, it is possible to arbitrarily switch a route that can be moved by a user of the walk-through system using a panoramic photographed image. Therefore, for example, if both the outdoor space and the indoor space are included in the shooting route, if only the outdoor is selected as the movable route, a user who desires a walk-through only outdoors may accidentally enter the indoor. Such a case can be prevented.
[0048]
[Second embodiment]
In the present embodiment, an image presenting apparatus that allows a user to set a movable path while walking through a virtual space will be described.
[0049]
FIG. 7 is a block diagram illustrating a schematic configuration of the image presentation device according to the second embodiment. The same components as those of the image presentation device (FIG. 1) according to the first embodiment are denoted by the same reference numerals, and detailed description of the components having the same functions will be omitted. The second embodiment is different from the first embodiment in that the image presentation device according to the first embodiment does not include the movable route selection unit 6. Also, the functions of the movable route setting means 3, the operation unit 5, the viewpoint calculation means 7, and the image generation means 8 are different from those of the first embodiment.
[0050]
The movable path setting means 3 sets the path on which the walk-through has been performed by the operation of the operation unit 5, that is, the trajectory of the viewpoint calculated by the viewpoint calculating means 7 in the movable path information, and sets the movable path storage means 4 Is sent to the image generating means 8. At this time, it is possible to set the route where the walk-through has been performed as the movable route information only while the button of the operation unit 5 is being pressed, for example. Further, for example, only while the button of the operation unit 5 is being pressed, the route on which the walk-through has been performed can be removed from the movable route information.
[0051]
In the operation unit 5, the operation specifies a panoramic real image to be a basis for image generation, specifies a portion to be cut out from the panoramic real image, sets movable path information, and the like. The operation information of the operation unit 5 is sent to the viewpoint calculation unit 7 and the movable route setting unit 3.
[0052]
The viewpoint calculation unit 7 calculates viewpoint information based on the operation information received from the operation unit 5 and the photographing route information stored in the photographing route storage unit 2 and sends the viewpoint information to the movable route setting unit 3 and the image generation unit 8. I do. Here, the viewpoint position information can be made to coincide with any one of the position information of each panoramic photographed image that can be calculated from the photographing route information, for example. In that case, the viewpoint position can be set on the photographing route. Note that the viewpoint position information can also be represented by a frame number of a panoramic photographed image.
[0053]
The image generating means 8 acquires a panoramic real image from which the image is to be generated from the panoramic real image storing means 1 in accordance with the viewpoint information sent from the viewpoint calculating means 7, and generates a cut-out image corresponding to the viewpoint information. Based on the photographing path information stored in the photographing path storage unit 2, a path figure representing the photographing path is generated and combined with the cut-out image to generate a display image. Further, based on the movable route information set by the movable route setting means 3, a route graphic representing the movable route is generated with an attribute different from an attribute such as a color of the route graphic representing the photographing route. Is synthesized with the display image of. The route graphic representing the movable route changes as the walk-through is performed.
[0054]
The control procedure of the second embodiment having the above configuration will be described below. FIG. 8 is a flowchart illustrating a procedure of processing in the image presentation device of the present invention.
[0055]
In step S801, data preparation is performed. In step S801, a panoramic photographed image stored in the panoramic photographed image storage unit 1 and photographing route information stored in the photographing route storage unit 2 are prepared.
[0056]
After data preparation, the system is started in step S802. In step S803, the operation unit 5 is operated, and the operation information is sent to the viewpoint calculation unit 7 and the movable route setting unit 3.
[0057]
Next, in step S804, the viewpoint calculation unit 7 calculates viewpoint information based on the operation information received from the operation unit 5 and the photographing route information stored in the photographing route storage unit 2, and converts the calculated viewpoint information into a movable route. It is sent to the setting means 3 and the image generating means 8.
[0058]
Next, in step S805, the movable path setting unit 3 sets the path in which the viewpoint has been moved by the operation of the operation unit 5 as the movable path information, and sends the information to the movable path storage unit 4 and the image generation unit 8. I do.
[0059]
Here, a case where the movable route information is changed during the walk-through will be described with reference to FIG. For example, when the operation unit 5 is operated to instruct the change to the movable route information M2 while the movable route information M1 is selected, the route ID corresponding to the movable route information M2 is changed to the movable route information list 1000. Is determined based on Then, the walk-through is performed only along the route indicated by the obtained route ID. Here, the route graphic representing the movable route information is represented as a group of line segments connecting the start branch point and the end branch point (determined based on the route list 210 shown in FIG. 2B) of each route. Note that the coordinates of each branch point are obtained based on the branch point list 200 shown in FIG. 2A.
[0060]
Next, in step S806, the image generation unit 8 acquires a panorama real image from which the image is to be generated from the panorama real image storage unit 1 according to the viewpoint information sent from the viewpoint calculation unit 7, and according to the viewpoint information. Generate a clipped image. Further, based on the photographing route information stored in the photographing route storage unit 2, a route graphic representing the photographing route is generated. Further, based on the movable path information sent from the movable path setting means 3, a path graphic representing a movable path is generated, and a display image is generated by combining the extracted image with the photographing path.
[0061]
Next, in step S807, the display image generated by the image generation unit 8 is displayed on the image display unit 9.
[0062]
Thereafter, it is checked in step S808 whether the system is to be terminated. If the system is to be terminated, the system is terminated in step S809. If not, the process returns to step S803, and the above-described processing is repeated.
[0063]
In this embodiment, the case where the movable path information is set based on the photographing path information held in the photographing path storage means 2 has been described. However, the present invention is not limited to this. It is also possible to set new movable route information on the basis of the movable route information held in 4.
[0064]
Further, in the present embodiment, the case where the path graphic representing the movable path is combined with the cut-out image is described. However, the present invention is not limited to this, and the path graphic representing the movable path is combined with the cut-out image. Instead, the image may be displayed on a display unit different from the image display unit 9.
[0065]
Further, the display position and the size of the route graphic may be operated. Further, display / non-display of the route graphic may be switched.
[0066]
Further, a map image storage unit (not shown) may be further provided, a map image may be synthesized on a panoramic photographed image, and a route graphic representing a movable route may be synthesized on the map image and presented to the user. .
[0067]
As described above, according to the second embodiment, a user of the walk-through system using a panoramic real image can set a route that can be moved while walking. Therefore, for example, only a route with a beautiful scenery can be set as a movable route.
[0068]
[Third Embodiment]
In the present embodiment, an image presentation device that allows a user to easily determine a path that cannot be moved during a walkthrough in a virtual space will be described.
[0069]
The block diagram showing the schematic configuration of the image presentation device according to the third embodiment is the same as that of FIG. Note that the function of the operation unit 5 is different from that of the first embodiment.
[0070]
The map information storage means holds map information whose range has been determined based on the position and orientation information of the shooting location associated with the panoramic real image. The map information is stored in a medium such as a hard disk, tape, or memory.
[0071]
The non-progressive route calculating means calculates non-progressive route information indicating a non-progressible route. First, the next branch point is obtained based on the viewpoint information calculated by the viewpoint calculation means 7. Next, from the viewpoint information and the map information stored in the map information storage means, all traveling route information representing all traveling route candidates at the next branch point is obtained. On the other hand, based on the viewpoint information and the movable path information selected by the movable path selection means 6, the movable path information at the next branch point is obtained. The information obtained by removing the available route information from all the available route information is the inoperable route information. In addition, the branch point and the route are described in the form shown in FIGS. 2A and 2B, for example, in all the traveling route information, the travelable route information, and the non-traveling route information. In this case, the branch point ID 201 in FIG. 2A is only an ID representing the next branch point, and the path ID 212 in FIG. 2B is an ID in which either the corresponding start branch point 212 or end branch point 213 indicates the next branch point. It becomes only what becomes.
[0072]
The operation unit 5 performs the operation to select the movable route information, to specify a panoramic real image to be an image generation base, to specify a portion to be cut out from the panoramic real image, and the like. Further, when the user attempts to proceed on a path that cannot travel at the branch point, the operation unit 5 generates a force in a direction that hinders the travel and causes the user to determine the path that cannot travel.
[0073]
Here, a method of determining the magnitude of the reaction force generated by the operation unit 5 will be described. First, a gaze direction vector is obtained from gaze direction information. Further, a non-traveling direction vector is obtained from the non-traveling route information. Then, the angle θ between the two vectors is determined, and the smaller the angle, the larger the reaction force is generated. The magnitude F of the reaction force is, for example,
F = Fmax · cos (kθ) (0 ° ≦ kθ ≦ 90 °)
F = 0 (90 ° <kθ ≦ 180 °)
It should be determined. Here, Fmax is the maximum reaction force. K is a parameter that determines the range of the angle at which the reaction force is generated. For example, when k = 2, a reaction force is generated in the range of 0 ° ≦ θ ≦ 45 °. At this time, the reaction force may be applied in such a direction that the line-of-sight direction vector moves away from the unmovable direction vector. Here, when the line-of-sight direction information is specified by, for example, twisting the grip of the operation unit 5, the reaction force may be generated in the twist direction of the grip.
[0074]
If there are a plurality of non-traveling routes at the next branch point, the respective non-traveling direction vectors are obtained, and the magnitude of the reaction force is obtained for each of them by the above-described method. Just add them up.
[0075]
Further, when the angle formed between the line-of-sight direction vector and the non-progression direction vector at the next branch point is small, it is possible to generate a reaction force so as to make it difficult to approach the branch point. If the viewpoint position information is specified by, for example, tilting the grip of the operation unit 5 back and forth, the reaction force may be generated in a direction in which the grip is tilted back and forth. At this time, the reaction force may be applied in such a direction that the viewpoint position moves away from the branch point position.
[0076]
It should be noted that the magnitude of the reaction force as described above can be increased as approaching the branch point.
[0077]
The control procedure of the third embodiment having the above configuration will be described below. FIG. 9 is a flowchart illustrating a procedure of processing in the image presentation device of the present invention.
[0078]
Steps for performing the same processes as those in the flowchart (FIG. 6) shown in the first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted. In the following, different parts between the first embodiment and the third embodiment will be described.
[0079]
In the third embodiment, as in the first embodiment, data preparation is performed in step S601. In step S601, a panoramic real image stored in the panoramic real image storage unit 1, shooting route information stored in the shooting route storage unit 2, and map information stored in a map information storage unit (not shown) are prepared.
[0080]
In the third embodiment, the non-movable route information is calculated in step S901 following step S607. Unmovable route information can be obtained from viewpoint information, map information, and movable route information.
[0081]
Next, in step S902, a reaction force is generated in the operation unit 5. The magnitude of the reaction force can be obtained from the viewpoint information and the unmovable route information.
[0082]
After step S902, it is checked in step S608 whether the system is to be terminated. If the system is to be terminated, the system is terminated in step S609. If not, the process returns to step S603 and repeats the processing.
[0083]
In the present embodiment, a case where a reaction force is generated when the user attempts to proceed on a path that cannot proceed at the branch point has been described. However, the present invention is not limited to this. It is also possible to set a reaction force and generate a reaction force when trying to proceed on such a route. In this case, it is possible to prevent the vehicle from erroneously proceeding on such a route. On the other hand, it is also possible to proceed on such a route.
[0084]
Further, in the present embodiment, the case where the reaction force is generated to cause the user to determine the immovable route is described. However, the present invention is not limited to this. You may. In addition, it is also possible to cause the user to determine an immovable route by vibrating the operation unit 5 with a vibrator or the like or blinking an image displayed on the image display unit 9.
[0085]
As described above, according to the third embodiment, the user of the walk-through system using the panoramic real shot image can easily determine the immovable route during the walk-through. As a result, it is possible to guide the user in a direction in which the user can proceed, and it is also possible to make it difficult for the user to proceed to a route that he does not want to proceed.
[0086]
It is to be noted that an object of the present invention is to supply a storage medium (or a recording medium) storing program codes of software for realizing the functions of the above-described embodiments to a system or an apparatus, and to provide a computer (or a CPU or Needless to say, the present invention can also be achieved by an MPU) reading and executing a program code stored in a storage medium. In this case, the program code itself read from the storage medium realizes the function of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. When the computer executes the readout program code, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a part or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing.
[0087]
Further, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is executed based on the instruction of the program code. It goes without saying that the CPU included in the expansion card or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.
[0088]
【The invention's effect】
According to the present invention, in a walk-through in a virtual space using a panoramic photographed image, a route that can be moved by a user can be switched or set, and the traveling route can be restricted.
[0089]
Further, it is possible to easily determine a route that the user cannot move.
[Brief description of the drawings]
FIG. 1 shows a block diagram of a process according to a first embodiment.
FIG. 2A is a diagram illustrating a description example of a branch point in the first embodiment.
FIG. 2B is a diagram illustrating a description example of a route according to the first embodiment.
FIG. 3 is a diagram illustrating a method of generating a correct perspective image by cutting out a partial image corresponding to a viewpoint from a panoramic image, performing texture mapping on a cylindrical surface, and performing perspective transformation in the first embodiment. .
FIG. 4 is a diagram illustrating the synthesis of a clipped image, a map image, and a route graphic generated from a panoramic real image in the first embodiment.
FIG. 5 is a diagram illustrating a screen display example in the case where attributes such as colors of a partial graphic representing a planned travel route and a partial graphic representing a current travel route are changed in the first embodiment.
FIG. 6 is a flowchart illustrating a procedure of processing in the image presentation device according to the first embodiment.
FIG. 7 shows a block diagram of a process according to the second embodiment.
FIG. 8 is a flowchart illustrating a processing procedure in the image presentation device according to the second embodiment.
FIG. 9 is a flowchart illustrating a processing procedure in the image presentation device according to the third embodiment.
FIG. 10 is a diagram illustrating a description example of movable route information.
FIG. 11 is a diagram illustrating a process of changing the movable route information during a walk-through.

Claims (16)

Panoramic photographed image storage means for holding a plurality of panoramic photographed images;
Movable route setting means for setting movable route information;
Movable path storage means for storing a plurality of the movable path information,
An operation unit that allows the user to reflect the intention,
Movable path selecting means for selecting desired movable path information from the movable path storage means;
A viewpoint calculation unit configured to calculate viewpoint information including viewpoint position information and line-of-sight direction information based on the operation information and the movable path information transmitted from the operation unit;
Image generation means for generating a cutout image from the panoramic real image based on the viewpoint information to generate a display image,
An image presentation device, comprising: an image display unit that displays the display image. The apparatus further includes a photographing path storage unit that stores photographing path information at the time of photographing the panoramic real image, and the movable path setting unit extracts all or a part of the photographing path information and sets the movable path information. The image presentation device according to claim 1, wherein: The said movable route setting means extracts all or some of the already set movable route information, and sets new movable route information, The Claim 1 or Claim 2 characterized by the above-mentioned. Image presentation device. 4. The movable route setting unit according to claim 1, wherein all or a part of the route through which the walk-through is performed by the operation of the operation unit is set in the movable route information. The image presentation device according to the above. The moving path setting means generates a path figure representing a photographing path based on the photographing path information, and sets the movable path information by selecting all or a part of the path figure. 3. The image presentation device according to 2. The movable route setting means generates a route graphic representing a movable route based on the previously set movable route information, selects all or a part of the route graphic, and generates new movable route information. The image presentation apparatus according to claim 1, wherein the setting is set. The photographing route storage unit further holds attribute information for each route, and the movable route setting unit automatically moves the movable route information based on the photographing route information and the attribute information stored in the photographing route storage unit. The image presentation device according to claim 2, wherein The image presentation device according to claim 1, wherein the movable path selection unit selects desired movable path information from the movable path storage unit during a walk-through. 9. The image presentation device according to claim 1, wherein the movable route selection unit limits the selectable movable route information based on the viewpoint information. The image generating means generates a route graphic representing a movable route based on the movable route information, and generates a display image by combining the cut image and the route graphic. Item 10. The image presentation device according to any one of Items 9. 11. The image processing apparatus according to claim 10, further comprising a map image storage unit that stores a map image, wherein the image generation unit generates a display image by combining the cutout image, the map image, and the route graphic. Image presentation device. The vehicle further comprises a non-traveling route calculating means and a map information storage means for holding map information, wherein the non-traveling route calculating means calculates the non-traveling route information based on the viewpoint information, the map information and the movable route information. The image presentation device according to any one of claims 1 to 11, wherein 13. The image presentation device according to claim 12, wherein the operation unit generates a force in a direction that impedes the progress when the user attempts to proceed to a route that the user cannot proceed. The sound generating unit may further include a sound generating unit that generates a sound, wherein the sound generating unit generates a warning sound when a user attempts to proceed to a route that cannot be advanced by operating the operation unit. The image presentation device according to claim 12 or 13. Input the user's instruction using the operation unit,
Calculate viewpoint information including viewpoint position information and gaze direction information based on the input instruction and the movable path information,
An information processing method of generating a cut-out image from a plurality of panoramic real shot images held in advance based on viewpoint information from the panoramic real shot image,
The information processing method, wherein the movable route information can be edited. A program for realizing the information processing method according to claim 15.

Images