JP2004280250A - Apparatus and method for processing information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for displaying a view the user desires to watch in high resolution using a walk-through experience system with a live-action data. <P>SOLUTION: A virtual space experience system, which enables the user to experience a walk-through in a virtual space, comprises a means for selecting a content, a means for specifying a movable path within the content, a means for specifying the resolution for a data, a means for acquiring the data based on the content being selected, the path being specified, and the resolution being specified, an operational means for controlling a view point and a view direction, and a means for generating an image, which extracts the image corresponding to the view point specified from the acquisition data, generates a part of the image corresponding to the view direction specified, and displays the part of the image. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an apparatus for expressing a virtual space based on image data obtained by photographing a real space.
[0002]
[Prior art]
As a method of expressing the photographed real space as a virtual space based on the real image data captured by the imaging device mounted on the moving object, a geometric model of the real space is reproduced based on the real image data. However, there is a method of expressing by a conventional CG technique, but there is a limit in terms of accuracy, precision, and realism of the model. On the other hand, Image-Based Rendering (IBR) technology for expressing a virtual space using a real image without performing reproduction using a model has recently attracted attention. Since the IBR technology is based on a real image, it is possible to represent a realistic virtual space.
[0003]
In order to construct a walkable virtual space using such an IBR technology, it is necessary to generate and present an image according to the position of the user in the virtual space. Therefore, in this type of system, each frame of the real image data and the position in the virtual space are stored in association with each other, and the corresponding frame is acquired based on the position in the virtual space and the gaze direction of the user. Play this.
[0004]
In such a walkthrough in the virtual space, the user can see a desired direction at each viewpoint position. For this reason, the image at each viewpoint position is stored as a panoramic image covering a wider range than the angle of view at the time of reproduction, and the panoramic image is selected based on the viewpoint position and the viewing direction in the virtual space of the user. At the same time, it is conceivable to cut out and display a partial image to be reproduced. In this case, since the photographed image data is reproduced, the panoramic image is reproduced along the road on which the vehicle equipped with the photographing device runs, and the observer feels as if he / she is riding on the vehicle. You can taste. In addition, when the reproduced image from the walk-through enters the intersection and the observer selects a road in a direction different from the actual vehicle movement, the panoramic image group is switched to a different panoramic image group from the reproduced panoramic image group, and a new course is selected. Of the panoramic image is performed.
[0005]
As this kind of VR system, Japanese Patent Application Laid-Open No. 9-62861 discloses a system in which a series of panoramic images is held and a predetermined panoramic image is reproduced in response to a user operation. Japanese Patent Application Laid-Open No. 2001-008232 discloses a VR system that allows a user to specify a route to be walked through.
[0006]
[Problems to be solved by the invention]
However, in the above-described conventional example, since the data used for the walk-through holds only data of a single resolution for one path, there is a need for a user to watch a high-resolution video with a higher resolution. I cannot be satisfied. In addition, when acquiring data via a network, it is desired to adjust the amount of data according to the situation. However, when only data of a single resolution is available, the countermeasures such as dropping frames are limited.
[0007]
Japanese Patent Application Laid-Open No. 2001-008232 discloses that a route of a content is specified, but does not disclose any data resolution. Generally, when a route is specified, an object to be viewed must exist on the route, and it is desirable that the resolution of data at a position close to the object (or a large image) is higher. The easiest way to achieve this is to generate the data for each frame by changing the resolution in advance, but since the objects of interest vary from user to user, eventually all data will have multiple resolution data. Without it, not all user requirements can be satisfied.
[0008]
Further, when objects of interest to the user are sparsely present on the designated route, it is wasteful to acquire all data on the route.
The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a walk-through system capable of displaying an object of interest to a user with high image quality while suppressing a data amount. .
[0009]
[Means for Solving the Problems]
The present invention for achieving the above object has the following configuration.
[0010]
The invention according to claim 1 of the present application is an information processing apparatus applicable to a virtual space experience system capable of walking through a virtual space, comprising: a content selection unit for selecting content; Route designation means for designating a specific route, resolution designation means for designating data resolution, data acquisition means for acquiring data based on the selected content, the designated path, and the designated resolution, Point of view position, operating means for controlling the line of sight direction, image generation means for extracting an image corresponding to the specified viewpoint position from the acquired data, generating and displaying a partial image corresponding to the specified line of sight direction, It is characterized by having.
[0011]
The invention according to claim 4 of the present application is an information processing method applicable to a virtual space experience system capable of walking through a virtual space, and acquires an arbitrary object according to a user's instruction. A resolution designation step of designating the resolution of the data, a data acquisition step of acquiring data according to the resolution designated by the resolution designation means, a viewpoint position, an operation step of controlling a line-of-sight direction, and Generating a partial image corresponding to the viewpoint position and the line-of-sight viewpoint position.
[0012]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.
[0013]
<First embodiment>
First, a virtual space walk-through system according to the present embodiment will be described. In the present embodiment, for example, panoramic image data is generated from real image data obtained by photographing with a plurality of photographing devices mounted on a moving body such as an automobile, and the panoramic image data corresponds to a position in a real space. It is stored in a server or the like in association with the position on the map. At this time, each frame of the panoramic image data is held as data having a plurality of different resolutions. Data having different resolutions may be encoded using hierarchical encoding or the like. The playback system obtains information such as a shooting position, a shooting direction, and a map from the server, low-resolution panoramic image data of a specified route, and high-resolution panoramic image data of a frame specified by a user, and obtains a virtual image of the user. A display image is generated from the stored panoramic image data according to the viewpoint position (position on the map) and the line-of-sight direction in the space, and is displayed together with the map showing the route information, so that the walk-through in the virtual space is performed. To achieve.
[0014]
FIG. 1 is a block diagram illustrating a functional configuration of the walk-through system according to the present embodiment. The image reproducing apparatus 1 constituting the walk-through system includes a communication unit 4, a content selection unit 5, a data acquisition unit 10, a data storage unit 15, a correspondence data storage unit 20, a map data storage unit 25, and a route information synthesis unit 30. , An acquired path information holding unit 35, an operation unit 40, a viewpoint position / viewing direction determining unit 45, a corresponding image determining unit 50, a display image synthesizing unit 55, a display unit 60, and an image generating unit 65.
[0015]
FIG. 2 is a block diagram illustrating a hardware configuration of the image reproducing device 1 according to the present embodiment. The hardware configuration shown in FIG. 2 is equivalent to the configuration of a normal personal computer. In FIG. 2, a disk 105 constitutes the data storage unit 15. The disk 105 constitutes not only the image data storage unit 15 described above but also the association data holding unit 20, the map data holding unit 25, and the acquired route information holding unit 35 shown in FIG.
[0016]
The CPU 101 executes a program stored in the disk 105 or the ROM 106 or an external storage device (not shown) to execute the content selecting unit 5, the route information synthesizing unit 30, the viewpoint position / viewing direction determining unit 45, the corresponding image It functions as the determination unit 50 and the image generation unit 65.
[0017]
When the CPU 101 issues various display instructions to the display controller 102, desired display is performed on the display 104 by the display controller 102 and the frame buffer 103. In the figure, the CRTC is shown as the display controller 102 and the CRT is shown as the display 104. However, the display is not limited to a cathode ray tube, but may be a liquid crystal display or the like. Note that the CRTC 102, the frame buffer 103, and the CRT 104 constitute the display unit 60 described above. The mouse 108, the keyboard 109, and the joystick 110 are used to input a user's operation to the image reproducing device 1, and constitute the above-described operation unit 40.
[0018]
Next, an outline of functions of each unit of the image reproducing apparatus 1 in the walk-through system according to the present embodiment having the above configuration will be described.
[0019]
The communication unit 4 exchanges data with an external server in response to a request from each unit. The content selection unit 5 presents the content stored in the external server to the user, and causes the user to select the content. At the same time, object data included in the content is also presented, and a target to be displayed at a high resolution is designated.
[0020]
FIG. 24 shows an example of the content selection screen and an example of the object designation screen. On the screen shown in FIG. 24A, when the number of the content displayed in the list is input, the content is selected. As the content selection screen, a mark indicating a place may be arranged on a map, and a selection operation may be performed by clicking the mark. FIG. 24B shows an object designation screen when Kyoto is selected as an example. 721 to 727 represent objects. Here, when the user clicks on a circle in the figure, the object is designated as an object to be acquired as high-resolution data. The objects 723 and 727 in the figure are black circles, indicating that the objects have been set to high-resolution data. Otherwise, the resolution is set lower.
[0021]
The data acquisition unit 10 sequentially downloads necessary data such as map information, photographing position / direction information, and photographed image data from a server on a network or a local storage device based on the information specified by the content selection unit 5. . As for the captured image data, after obtaining all of the low-resolution data (including the low-resolution data of the object specified as the high resolution), the data of the object specified as the high resolution is obtained. The download order of the low-resolution data is determined in advance by the server or is determined by a request from the image reproducing apparatus 1 as a client. The same applies to high-resolution data. In this way, by capturing only the objects of interest to the user at a high resolution and the others at a low resolution, the desired data can be reproduced with high image quality and the total data amount can be suppressed. Real-time property can be secured. Further, by causing the user to specify an object of interest, it is possible to avoid obtaining useless data.
[0022]
The data storage unit 15 stores the image data (panoramic image data corresponding to each position on the map) acquired by the data acquisition unit 10. The association data holding unit 20 holds information on the correspondence between the image data stored in the data storage unit 10 and the map stored in the map data storage unit 25. The map data holding unit 25 stores map data that includes at least a position on a map obtained by the data acquisition unit 10 and that corresponds to the panoramic image data stored in the data storage unit 15. The acquired route information holding unit 35 stores the route information corresponding to the acquired image data stored in the data storage unit 15 (this is called partial route information, and the route indicated by the information is called the partial route). .) And the path information corresponding to the entire image data to be downloaded from the server (this is referred to as “all path information”, and the path indicated by the information is referred to as “all path”). Each piece of route information is expressed using an ID described later.
[0023]
FIG. 14 shows an example of the route information held by the acquired route information holding unit 35. Image data (intersections, roads, and the like) to be downloaded is held as a list, and “1” is recorded for acquired data and “0” is recorded for unacquired data. In the figure, the entire list of intersections and roads indicated by ID represents the entire route, and among them, the set of intersections and roads in which “1” is recorded in the downloaded column represents the partial route. The high resolution column in FIG. 14B indicates whether or not the entire road segment is to be acquired to a high resolution. When "1" is described in this column, it means that data of high resolution is obtained in addition to data of low resolution. “0” indicates that data of only low resolution is acquired. Here, it is determined whether or not the high resolution is set for each road. However, it is needless to say that a high resolution and a low resolution may be specified for each panorama image forming the road. However, when designating in units of panoramic images, the resolution may suddenly change between adjacent panoramic images. In that case, instead of two types of resolutions, more types of resolution data may be prepared, and the resolution of each panoramic image may be set so as to smoothly change between adjacent panoramic images.
[0024]
The route information synthesizing unit 30 synthesizes a map stored in the map data holding unit 25, the partial route information stored in the acquired route information holding unit 35, and the entire route information. The color of the entire route information combined with the map here is different from the color of the partial route information, so that the range of data to be downloaded and the range of already acquired data can be grasped, and walk-through can be done freely Information of a particular area can be provided to the user. Also, when displaying on a monitor or the like that does not allow color display, instead of changing the color, the line width and type of the entire route information and the partial route information to be combined on the map can be made different to achieve the same purpose. Can be achieved. It is also possible to simultaneously change the color and the line width. If the entire route information is displayed as a thin and thin line, and the partial route information is displayed as a dark and thick line, it is possible to display more clearly.
[0025]
FIG. 9A shows an example of a map stored in the map data holding unit 25, and FIGS. 9B and 9C show examples of an entire route and a partial route, respectively. FIG. 10 shows an image obtained by combining these three components. Here, all routes are displayed in light colors, and partial routes are displayed in dark colors. A circle represents a viewpoint position, and an arrow represents a line of sight.
[0026]
The operation unit 40 includes a mouse, a keyboard, a joystick, and the like. The operation unit 40 generates a movement parameter of the viewpoint position and a rotation parameter of the viewing direction. It also has functions such as content selection and object designation. In the present embodiment, the joystick is used in the operation unit 40, but the same operation can be performed with another input device such as a game controller. FIG. 3 is a diagram showing the relationship between the operation of the joystick on the operation unit 40 and the movement parameters and rotation parameters. The joystick can independently control the tilt and rotation of the stick. In the present embodiment, the operation of tilting the joystick back and forth corresponds to the movement of the viewpoint position in the virtual space, and the operation of twisting the joystick left and right corresponds to a change in the line of sight.
[0027]
FIG. 4 is a diagram illustrating a flow of data in the operation unit 40. From the operation unit 40, the tilt angle α (α> 0, front and rear, and a negative value when the joystick is tilted backward) of the joystick is used as a movement parameter to the viewpoint position / gaze direction determination unit 45 as a left and right twist angle β. (Β> 0, negative value when twisted to the right) is given as a rotation parameter.
[0028]
The viewpoint position / viewing direction determining unit 45 determines the viewpoint position (position on the map) of the observer on the map data stored in the map data holding unit 25 based on the movement parameter and the rotation parameter from the operation unit 40. , Determine the gaze direction.
[0029]
FIG. 5 is a diagram showing a flow of data in the viewpoint position / viewing direction determining unit 45. The viewpoint position / line-of-sight direction determining unit 45 acquires the map data from the map data holding unit 25 and the entire route information and the partial route information from the acquired route information holding unit 35, and observes the corresponding image determining unit 50. Give the viewpoint position and line-of-sight direction of the user. The processing in this part will be described in detail with reference to FIG.
[0030]
FIG. 12 shows an example of a screen displayed on the display means of the proposed system. A map on which the route information is superimposed is displayed in the upper left of the figure, where the current observation position of the user is indicated by a circle and the viewing direction is indicated by an arrow. The route indicated by a light color in the map is the entire route, and the route indicated by a darker color is a partial route. The viewpoint position / line-of-sight direction determination unit 45 determines the viewpoint position / line-of-sight direction on the map based on the information from the operation unit 40, but the viewpoint on the map calculated based on the information from the operation unit 40. If the position is not on the partial route, the movement is restricted.
[0031]
In this way, by allowing movement only on a partial route, the movement range is limited, but a smooth walk-through can be realized. In comparison, the waiting time of the user is reduced. In addition, data acquisition from the server is continued in the background, and the number of partial routes that can be walk-through increases with time without imposing a special load on the user. Therefore, the user can concentrate on the walk-through without being particularly aware of the state of the network. Further, by dividing the route data into low-resolution data and high-resolution data, and acquiring and displaying only the low-resolution data first, the time until display can be further reduced. After that, by obtaining the additional high-resolution data, the part of interest to the user can be expressed with higher image quality.
[0032]
The corresponding image determination unit 50 determines an image ID necessary for the image generation unit 65 to generate an image based on information on the observer's viewpoint position and line-of-sight direction.
[0033]
FIG. 6 is a diagram showing a data flow in the corresponding image determination unit 50. The display image synthesizing unit 55 is obtained from the viewpoint position, gaze direction, and map-image correspondence information data stored in the association data storage unit 20 in the virtual space of the user determined by the viewpoint position / gaze direction determination unit 45. The image storage unit 15 determines an image stored in the data storage unit 15 necessary for generating an image to be used in step S <b> 1, and passes the determined image ID to the image generation unit 65.
[0034]
In the present embodiment, the following data storage mode is adopted in the association data storage unit 20. That is, an ID is assigned to each of the intersections and roads, with intersections (branches) and corners being the division points, and a line segment sandwiched between the division points as the road, and the ID is added to the corresponding panoramic image. The panorama image group corresponding to one road is numbered sequentially from the top.
[0035]
FIG. 7 is a diagram illustrating this state. In FIG. 7, for example, an ID of R1 is given to a line segment (road) sandwiched between a section point having an ID of C1 and a section point having an ID of C2. The correspondence between the ID and the map is stored in the map data holding unit 25.
[0036]
When the image frames corresponding to the division points C1 and C2 are specified based on the GPS data and the like, the panoramic image group sandwiched between those frames corresponds to the road R1. In the figure, there are n images in this panoramic image group. The image frames corresponding to the segment points C1 and C2 are respectively assigned IDs of C1 and C2, and the images of the panoramic image group are assigned R1-1 to R1-n in order.
[0037]
FIG. 13 shows an example of ID and map correspondence data stored in the map data holding unit 25. FIG. 13A is a list of roads connected to each intersection. In this example, three roads R1, R2, and R3 are connected to the intersection C1. FIG. 13B shows coordinates indicating where each intersection exists on the map. In this example, a pixel position on a map image is used as a coordinate value. FIG. 13C shows which intersection each road is sandwiched between. For example, R2 indicates that the road is located between intersections C1 and C5.
[0038]
The image generation unit 65 generates an image to be used by the display image synthesis unit 55 for synthesis. FIG. 8 is a diagram showing a flow of data in the image generation unit 65. The image generation unit 65 reads an image from the data storage unit 15 based on the image ID determined by the corresponding image determination unit 50, cuts out a panoramic image according to the angle of view displayed on the display unit 60, and projects the image. The conversion is performed, and the display image synthesizing unit 55 generates an image used for synthesis. For example, an image as shown in FIG. 11 is generated by the image generation unit 65 and sent to the display image synthesis unit 55.
[0039]
The display image combining unit 55 combines the route information (for example, the image shown in FIG. 10) generated by the route information combining unit 30 and the image (for example, the image of FIG. 11) generated by the image generating unit 65. Combine. This composite image is shown in FIG. The composite image is sent to the display unit 60.
[0040]
The display unit 60 displays the image generated by the display image synthesizing unit 55. Here, the display information synthesizing unit 55 is provided to display the route information and the image on one display device. However, the route information generated by the route information synthesizing unit 30 without the display image synthesizing unit 55 is used. It is also possible to display the image generated by the image generating unit 65 on a separate display device.
[0041]
FIG. 15 is a flowchart showing the flow of the processing of the proposed system.
In step S1001, a list of contents is requested from the server, and contents are selected. Then, the download of the information of the selected content is started. Next, in step S1002, the experience system is activated.
[0042]
FIG. 16 is a flowchart showing a flow of processing of content selection and activation of a download process. In step S100, a content list is requested from the server, the content is selected from the content list, and the information is returned to the server. At the same time, an object to be displayed at a high resolution is designated and the information is sent to the server. In step S110, map information of the selected content is obtained, and in the next step S120, data corresponding to the image data and the map data in the content is obtained. Thereafter, in step S130, the image data on all the paths required for the walk-through is sequentially downloaded. This download process operates independently of the experience system process described below, and is continued until all the image data required for the walkthrough is downloaded from the server.
[0043]
FIG. 17 is a flowchart relating to the processing of the experience system process.
[0044]
First, in step S200, an initial viewpoint position and a line-of-sight direction at the time of observation are set, and an image corresponding to the viewpoint position and the line-of-sight direction is generated from the downloaded image group. After that, launch the viewer. In step S210, an image set according to the viewpoint position and the line-of-sight direction is combined with map information and displayed. In step S220, an operation device such as a joystick is used to input operation information based on the tilt angle and the twist of the joystick. In step S230, it is checked whether an end command has been input from the operation device, and if so, the experience system is stopped. Otherwise, the process proceeds to step S240. In step S240, the next viewpoint position and line-of-sight direction are determined from the joystick operation information. In step S250, an image necessary to generate a display image is specified from the determined viewpoint position and line-of-sight information, and a display image is generated from the image group specified in step S260. However, if the specified image is composed of both high-resolution and low-resolution data, an image is generated by integrating both, but if high-resolution data has not been acquired, only low-resolution data Use to generate a display image. As a result, high-speed image display can be performed although the quality of the display image is low, and the waiting time of the user is reduced. If both data are configured as hierarchically encoded data, final high-resolution data is generated by decoding low-resolution data and merging the decoded high-resolution data. Using this image, an image for display is further generated. When the low-resolution data and the high-resolution data are independent data, the display image may be generated using only the high-resolution data. Thereafter, the process returns to step S210.
[0045]
FIG. 18 is a flowchart showing the flow of the image display process.
[0046]
In step S300, partial route information corresponding to the already downloaded image is obtained from the server, and in step S310, the map information stored in the map data holding unit 25, all route information, and partial route information are combined. At this time, the colors of the partial route information and the entire route information are made different. At the same time, the viewpoint position and the line-of-sight direction are also synthesized. Next, in step S320, the map in which the viewpoint position, the line-of-sight direction, and the route information are combined and the image data corresponding to the viewpoint position and the line-of-sight direction are combined, and the image combined in step S330 is displayed on the display device. .
[0047]
As described above, by controlling to walk through only the downloaded area while downloading the walk-through data from the server, a smooth walk-through experience can be achieved regardless of the state of the network. Also, at that time, by changing the color of the partial route corresponding to the downloaded data on the entire route or displaying it with different line width, you can check the range that you can walk through, It is possible to reduce the stress on the experienced person caused by the inability to grasp the information.
[0048]
<Second embodiment>
In the first embodiment, the movable range of the user is displayed by superimposing the route information on the map. However, the map itself is divided into tiles, and only the movable area is displayed differently from the others. A similar effect can be exhibited. FIG. 19 shows such an example.
[0049]
FIG. 19A shows a map divided into tiles. Each tile holds information on a road and an intersection included therein, and when data corresponding to a road or an intersection is downloaded, a portion of the tile including the data is displayed brightly. For example, as shown in FIG. 19 (b), a tile whose color is entirely black indicates a tile for which data has not yet been acquired, and a bright (white) tile indicates that data of a road or an intersection therein has been acquired. Is shown. The user is free to walk through the area of the brightly colored tiles. As in the case of the first embodiment, data download is continuously performed in the background, and tile areas from which data can be obtained are displayed brighter in order. Here, if the data of roads and intersections is grouped in tiles, the download order of each tile is determined in advance, and the data of roads and intersections is downloaded for each tile, it is possible to obtain data efficiently. Can be.
[0050]
Further, in FIG. 19, the viewpoint position and the line-of-sight direction of the user are not shown, but it is also possible to display the viewpoint position and the line-of-sight direction by circles and arrows as in the first embodiment.
[0051]
By adopting such a display method, there is an advantage that the movable range can be intuitively shown to the user.
[0052]
<Other embodiments>
When a user walks through a sightseeing spot or the like, a place to be viewed is often determined in advance. In such a case, assuming that only one data acquisition order is determined in advance, it takes time until data at a target location is acquired, and this gives a stress to the user. Therefore, if a plurality of patterns are prepared for the order of acquiring road and intersection data, and the user selects the patterns before the walkthrough, this problem can be reduced. For example, in the case of an area walk-through as shown in FIG. 20, a pattern for acquiring road and intersection data in the order of Nijo Castle → Nishi Honganji → Higashi Honganji →. → Chion-in → Yasaka Shrine →…, that is, prepare a pattern to acquire 521, 522, 523, ... in order, and those who want to see Nijo Castle want to see the former acquisition pattern, Heian Jingu and Chion-in It is sufficient that a person can select the latter acquisition pattern. Further, when the user selects a route, by simultaneously specifying an object to be viewed at a high resolution, a large amount of data can be assigned to a part of interest, so that the appearance impression can be improved, and Since the amount of data does not increase, real-time processing when generating an image is also ensured. This is applicable to both the first embodiment and the second embodiment. Also, the present invention is not limited to this example, and any number of acquisition patterns may be set as long as the number is one or more.
[0053]
Here, a method has been described in which a plurality of acquisition patterns are prepared for one route, but it is also possible to prepare a plurality of route patterns and select the route itself. Taking FIG. 21 as an example, if you want to walk through Gosho, Nijo Castle, and Higashi Honganji Temple but do not want to see Kiyomizu Temple, it is only necessary to set a route pattern as shown in FIG. 21 (a). If you want to walk through Chion-in Temple and Kiyomizu-dera Temple, and if the route pattern of FIG. 21B is set, the user does not need to obtain unnecessary data by selecting them, so It is possible to achieve the purpose with. Of course, it goes without saying that a plurality of acquisition patterns can be set for each of the plurality of routes.
[0054]
FIG. 22 shows the flow of processing related to content selection. In step S400, information on the type of content existing in the server is acquired, and in step S410, the user is made to select which content to use. This can be realized by displaying a content list on the screen of the experience system that the user is watching and selecting the content list. The present invention is not limited to this. For example, a method of arranging a representative image or symbol of usable content on a wide area map and selecting it can be used. When the content is selected, a route pattern for performing a walk-through on the content is selected in step S420. FIG. 23 shows an example of a GUI (Graphical User Interface) that allows the user to select content. In this figure, two route pattern candidates are presented and the user selects one of them. However, three or more route patterns may be displayed and selected, or only one pattern may be used. . When only one pattern is set, the process of step S420 can be skipped, and the process can be appropriately performed according to the content information. In addition, a pattern (such as a recommended course) to be preferentially selected from a plurality of route patterns is specified in advance, and if the user does not perform the selection processing within a predetermined time, the previously specified course is selected. It may be possible to do so. When a route pattern is selected, the order of data acquisition in the route pattern is selected in step S430. As in the case of the route selection pattern, a place name, a building name, or the like, which is a point, such as a famous place, may be presented as a list in the order of acquisition and may be selected by the user, or the route may be indicated on a map and selected. You may do so. Regarding the data acquisition order, one or more types may be set. In addition, an acquisition order (such as a recommended tour order) to be preferentially selected is specified in advance, and if the user does not perform a selection process within a certain period of time, the previously specified acquisition order is selected. You may keep it. Next, in step S440, information on the objects existing on the route is acquired, and in step S450, the object is specified. As for these processes, a method of presenting object information on a map as shown in FIG. 24B and selecting by clicking an object on the map is simple. Alternatively, a pattern of a set of objects having a high resolution may be created in advance, and a pattern may be selected from the list. In step S460, the resolution of the specified object is determined. When assuming two types of low resolution and high resolution as in the first embodiment, this step is not necessary, but a plurality of resolutions are set to smooth the change in resolution between adjacent frames. It is necessary. In such a case, the resolution is specified in step S460.
[0055]
Regarding the resolution, a method of changing the number of pixels of the image and a method of controlling the spatial frequency (control by how much high-frequency components are cut or how coarsely the coefficients are quantized in the spatial frequency domain) However, it goes without saying that the above example is applicable to both cases.
[0056]
In order to make the change in resolution smooth between adjacent frames, the change in resolution of each frame may be made gentle. For example, assume that the object to be represented at high resolution is closest to the frame number N on the map. If the data of this frame is acquired at the highest resolution and the resolution of several to several tens of frames before and after frame N is gradually reduced (see FIG. 25), a rapid change in the resolution between frames can be avoided. In FIG. 25, each frame is represented as data having M levels of resolution. The circles indicate the resolution of each frame to be acquired. Here, frames N, N-1, and N-2 are set at a resolution M, frames N-2, N-3, N + 2, and N + 3 are set at a resolution M-1, and frames N-4, N-5, N + 4, In this example, the resolution gradually decreases with increasing distance from the frame N, such as setting N + 5 at the resolution M-2. Note that the resolution control method shown here is an example, and the present invention is not limited to this. As described above, by acquiring frames near the designated object at a high resolution and setting the others at a low resolution, it is possible to reduce the entire data amount to be acquired.
[0057]
As described above, by allowing the user to select a path pattern, select a data acquisition order, and specify an object to be displayed at a high resolution, an operation can be performed so that a desired portion can be quickly viewed with high image quality. It can provide a comfortable walk-through environment.
[0058]
【The invention's effect】
As described above, the data of all the frames is stored at a plurality of resolutions, and the frame in which the object specified by the user is captured or the frame set according to the route is obtained at a high resolution. By obtaining as low-resolution data, the amount of data can be suppressed and a high-quality video experience can be provided to the user.
[0059]
In addition, by setting a plurality of route patterns and acquisition patterns, the user can specify an object to be displayed at a high resolution on the route and select the route and the acquisition pattern so that the target location can be set earlier. Walk-through can be made possible, and higher quality and comfortable walk-through become possible.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a functional configuration of a walk-through system according to an embodiment.
FIG. 2 is a block diagram illustrating a hardware configuration of the image reproducing device 1 according to the present embodiment.
FIG. 3 is a diagram illustrating a joystick operation in the walk-through system of the embodiment.
FIG. 4 is a diagram showing a data flow in the operation unit 40 in the walk-through system of the present embodiment.
FIG. 5 is a diagram showing a data flow in a viewpoint position / viewing direction determining unit 45 in the walk-through system of the present embodiment.
FIG. 6 is a diagram showing a data flow in a corresponding image determination unit 50 in the walk-through system of the embodiment.
FIG. 7 is a diagram for explaining association of a panoramic image with map data using a road and a dividing point;
FIG. 8 is a diagram showing a data flow in an image generation unit 65 in the walk-through system of the present embodiment.
FIG. 9 is a diagram for explaining the operation of the route information synthesizing unit 30 according to the present embodiment.
FIG. 10 is a diagram for explaining the operation of the route information synthesizing unit 30 according to the present embodiment.
FIG. 11 is an example of an image generated by the image generation unit 45 according to the present embodiment.
FIG. 12 is an example of an image generated by a display image synthesis unit 65 according to the present embodiment.
FIG. 13 is an example of ID and map correspondence data stored in the map data holding unit 25 in the embodiment.
FIG. 14 is an example of route information held by an obtained route information holding unit 35 in the present embodiment.
FIG. 15 is a flowchart illustrating a flow of processing according to the present embodiment.
FIG. 16 is a flowchart illustrating a flow of processing of content selection and activation of a download process according to the present embodiment.
FIG. 17 is a flowchart relating to processing of an experience system process according to the present embodiment.
FIG. 18 is a flowchart illustrating a flow of an image display process according to the present embodiment.
FIG. 19 is an example of a map divided into tiles according to the second embodiment.
FIG. 20 is a diagram for explaining the order of data acquisition.
FIG. 21 is a diagram for explaining selection of a route pattern.
FIG. 22 is a flowchart of a process related to content selection.
FIG. 23 is a diagram illustrating an example of a GUI for content selection.
FIG. 24 is a diagram showing an example of a content selection screen and an object designation screen.
FIG. 25 is an example of a data resolution control method.

Claims (8)

An information processing apparatus applicable to a virtual space experience system capable of walking through a virtual space,
Content selection means for selecting content,
Route designating means for designating a movable route in the content;
Resolution specifying means for specifying the resolution of the data;
Data acquisition means for acquiring data based on the selected content, the designated path, and the designated resolution,
Operation means for controlling the viewpoint position and the line-of-sight direction,
An information processing apparatus comprising: an image generation unit that extracts an image corresponding to the designated viewpoint position from the acquired data, generates a partial image corresponding to a designated line-of-sight direction, and displays the partial image. 2. The information processing apparatus according to claim 1, wherein the resolution designating unit designates a high resolution for the displayed object on the map specified by the user. 3. The information processing apparatus according to claim 2, wherein the data acquisition unit acquires low-resolution and high-resolution data for an object designated as having a high resolution by the resolution designation unit. An information processing method applicable to a virtual space experience system capable of walking through a virtual space,
A resolution specifying step of specifying a resolution of data to be obtained on an arbitrary object according to a user's instruction;
A data acquisition step of acquiring data according to the resolution designated by the resolution designation means;
An operation process for controlling the viewpoint position and the line-of-sight direction,
Generating a partial image corresponding to the viewpoint position and the line-of-sight viewpoint position from the acquired data. 5. The information processing method according to claim 4, wherein in the resolution specifying step, the user specifies an arbitrary object on the displayed walkable map. 5. The information processing method according to claim 4, further comprising smoothing a change in resolution between adjacent frames. 5. The information processing method according to claim 4, wherein the data acquiring step acquires low-resolution and high-resolution data for an object designated as having a high resolution. A program for implementing the information processing method according to any one of claims 4 to 7 using a computer.

Images