JP5861499B2 - Movie presentation device - Google Patents

Description

  The present invention relates to a moving image presentation device, and more particularly to a device that presents a field of view as a moving image of a real place viewed from a viewpoint that moves along a predetermined route.

  When a camera equipped with a fisheye lens or an omnidirectional mirror is used, an omnidirectional image having a 360 ° field of view can be taken. If such an omnidirectional image is prepared in advance, it is possible to provide a service for presenting an image obtained by observing an arbitrary direction from an arbitrary viewpoint position desired by the viewer.

  For example, in Patent Document 1 below, an omnidirectional image is prepared in the information providing center, and an image obtained by observing an arbitrary direction from an arbitrary viewpoint is displayed on the screen of the terminal device in response to a viewer's request. A technique for transmitting the image data is disclosed. Google Inc. (Headquarters: Google Inc. in California, USA) is called “Street View” and provides a service that displays the scenery from any point of view on the road using the Internet. ing. On the other hand, Patent Document 2 discloses an apparatus that removes obstacles such as pedestrians, telephone poles, and roadside trees from a video image of an urban area taken by an in-vehicle camera, and presents an urban area image that does not include foreground objects such as moving objects. Has been.

JP 2001-008232 A JP 2011-118495 A

  When presenting a video about an actual location by the system disclosed in each of the above-mentioned patent documents or the method used in “Street View” provided by Google, as described above, It is necessary to take a panoramic view using an omnidirectional camera. Normally, such shooting is performed in units of individual paths connected through nodes, so that the video presentation device connects videos shot in units of paths every time the viewpoint passes through the nodes. They are also played back and presented to the viewer.

  However, when the path-unit moving images shot as separate scenes are connected and played back, the images at the joints become discontinuous, which may give the viewer a sense of discomfort. For example, when the first route and the second route are connected via a node (for example, a turning corner or a branch point), the first route is taken by moving the omnidirectional camera along the first route. A moving image and a second moving image captured by moving the omnidirectional camera along the second route are obtained. Then, when the moving image presentation in which the viewpoint is moved from the first route to the second route is performed, the second moving image is presented subsequent to the first moving image. Continuity is likely to occur.

  In this way, the reason why the discontinuity occurs in the image is that the first moving image and the second moving image are originally obtained by separate shooting operations, and thus are scenes shot at the same node position. However, this is because a geometrical displacement occurs or the lighting environment fluctuates. That is, when the second moving image is connected to the first moving image and played back, an image error such as a positional deviation or a luminance fluctuation is generated between the last scene of the first moving image and the first scene of the second moving image. Continuation will occur, giving the viewer a sense of discomfort.

  Therefore, an object of the present invention is to provide a moving image presentation apparatus that can present a moving image in a manner that does not make the viewer feel uncomfortable due to discontinuity of images at an actual place including a plurality of routes.

(1) In the first aspect of the present invention, a visual field in a predetermined line-of-sight direction viewed from a viewpoint of moving along a predetermined path for an actual place including a plurality of paths connected via nodes is used as a moving image. In the video presentation device to present,
Moving image data obtained by shooting using an omnidirectional camera that moves a route along a predetermined shooting direction, and configured by arranging omnidirectional images having a 360 ° field of view in units of frames. A video data storage for storing data for each route;
A basic information storage unit for storing connection information of each route through the node and route information indicating a reference direction of each route, and moving image data correspondence information indicating moving image data corresponding to each route;
A position / direction update unit that updates the viewpoint position and line-of-sight direction, a moving video playback instruction unit that instructs playback of a moving video viewed from the moving viewpoint when the viewpoint is moving on the route by the update, and a viewpoint A switching video playback instructing unit for instructing the node to play a switching video viewed from a stationary viewpoint when a transition is made from the old route to the new route via the node;
A moving video reproduction unit that reproduces a moving video based on an instruction given from the moving video reproduction instruction unit; and a switching video reproduction unit that reproduces a switching video based on an instruction given from the switching video reproduction instruction unit. A video playback unit;
Provided,
The position / direction update unit updates the viewpoint position so that the viewpoint moves along a predetermined route based on the route information, updates the line-of-sight direction as necessary, and the viewpoint is updated via the node. When the transition from the route to the new route is made, the viewpoint is temporarily stopped on the node, and after the switching video playback unit completes the playback of the switching video, the movement of the viewpoint is resumed on the new route,
The moving video playback instruction unit refers to the video data correspondence information, recognizes the video data corresponding to the moving path as the playback target video data, and sets the playback target video data in the frame order according to the moving direction of the viewpoint. A moving video playback instruction indicating that the video is cut out and played back according to the predetermined line-of-sight direction indicated by the position / direction update unit is given to the moving video playback unit,
Based on the moving video playback instruction, the moving video playback unit reads out the omnidirectional images constituting each frame of the playback target video data stored in the video data storage unit in the order of the specified frames, and reads out the omnidirectional images read out From, cut out the visual field image that constitutes the field of view of the specified predetermined line-of-sight direction, and output the resulting series of visual field images as a movie,
When the viewpoint changes from the old route to the new route via the node, the position / direction update unit determines the line-of-sight direction after switching by referring to the reference direction of the new route, and after switching from the line-of-sight direction before switching Perform a process of gradually updating the gaze direction to the gaze direction,
The switching video playback instructing unit refers to the video data correspondence information, so that switching is performed from the video of the old route to the video of the new route with a change from the line-of-sight direction before switching to the line-of-sight direction after switching. Give a video playback instruction to the switching video playback unit,
Based on the switching video playback instruction, the switching video playback unit reads the last frame in the frame order at the time of moving video playback of the video data of the old path stored in the video data storage unit as an omnidirectional image before switching, The moving image data of the new path stored in the data storage unit is read out as the first frame in the frame order at the time of moving image playback, and is read out as an omnidirectional image. Each of the visual field images to be cut out, and a process of generating a blend image obtained by synthesizing the two cut out visual field images at a predetermined blend ratio, while gradually changing the common line-of-sight direction from the line-of-sight direction before switching to the line-of-sight direction after switching And, while changing so that the ratio of the visual field image cut out from the omnidirectional image after switching is gradually increased, A series of blends images made is obtained so as to reproduce as switching video.

(2) According to a second aspect of the present invention, in the video presentation device according to the first aspect described above,
The elevation angle is below a predetermined reference value from a distorted circular image obtained by photographing a hemispherical field located above a predetermined horizontal plane using an omnidirectional camera equipped with a fisheye lens or an omnidirectional mirror. The moving image data constituted by arranging rectangular panoramic images obtained by cutting out the above-mentioned areas and arranging rectangular panorama images obtained by performing distortion correction on these areas is stored.

(3) According to a third aspect of the present invention, in the video presentation device according to the first or second aspect described above,
The route information stored in the basic information storage unit includes position information indicating coordinate values on the two-dimensional coordinate system of nodes located at both ends of each route, and the reference direction of each route is the both ends of the route. It is defined by the direction of a straight line connecting between two nodes located at.

(4) According to a fourth aspect of the present invention, in the video presentation device according to the first to third aspects described above,
The video data correspondence information stored in the basic information storage unit includes information indicating the shooting direction for each route,
The moving video playback instruction unit gives a moving video playback instruction indicating that video data is played back in the frame order at the time of shooting when the moving direction of the viewpoint is forward with respect to the shooting direction. When the direction is opposite to the direction, a moving moving image reproduction instruction indicating that the moving image data is reproduced in a frame order opposite to that at the time of shooting is given.

(5) According to a fifth aspect of the present invention, in the video presentation device according to the first to fourth aspects described above,
The basic information storage unit determines whether there is a path connecting the position information indicating the coordinate value of each node on the two-dimensional coordinate system and any two nodes in all the nodes. And route information constituted by the connection information shown.

(6) According to a sixth aspect of the present invention, in the video presentation device according to the first to fifth aspects described above,
The basic information storage unit stores route information indicating the route defined on the two-dimensional plane,
The position / direction updating unit uses an azimuth angle φ (0 ° ≦ φ <360 °) with respect to a predetermined reference axis on the two-dimensional plane as a parameter indicating the line-of-sight direction.

(7) According to a seventh aspect of the present invention, in the video presentation device according to the sixth aspect described above,
The position / direction update unit updates the azimuth angle D corresponding to the reference direction of the moving route as the azimuth angle φ indicating the line-of-sight direction,
The switching video reproduction instruction unit sets the azimuth angle Dold corresponding to the reference direction of the old route as the azimuth angle φold indicating the visual line direction before switching, and the azimuth angle Dnew corresponding to the reference direction of the new route indicates the visual line direction after switching. A switching video playback instruction with an azimuth angle φnew is given.

(8) According to an eighth aspect of the present invention, in the video presentation device according to the first to fifth aspects described above,
An instruction input unit for inputting a user instruction;
The position / direction updating unit has a function of updating the viewpoint position and the line-of-sight direction in accordance with a user instruction to the instruction input unit.

(9) According to a ninth aspect of the present invention, in the video presentation device according to the eighth aspect described above,
The basic information storage unit stores route information indicating the route defined on the two-dimensional plane,
The position direction update unit uses the azimuth angle φ (0 ° ≦ φ <360 °) with respect to a predetermined reference axis on the two-dimensional plane as a parameter indicating the line-of-sight direction,
The instruction input unit has a function of inputting an instruction to set an offset angle θ in the line-of-sight direction with respect to the moving direction on the two-dimensional plane,
The position / direction updating unit determines the azimuth angle φ indicating the line-of-sight direction based on the formula φ = D + θ based on the azimuth angle D corresponding to the reference direction of the moving route.

(10) According to a tenth aspect of the present invention, in the video presentation device according to the ninth aspect described above,
Based on the azimuth angle Dold corresponding to the reference direction of the old route, the position / direction updating unit determines the azimuth angle φold indicating the line-of-sight direction before switching based on the formula φold = Dold + θ, and the reference direction of the new route The azimuth angle φnew indicating the line-of-sight direction after switching is determined based on the formula φnew = Dnew + θ based on the azimuth angle Dnew corresponding to.

(11) According to an eleventh aspect of the present invention, in the video presentation device according to the ninth aspect described above,
Based on the azimuth angle Dold corresponding to the reference direction of the old route, the position / direction updating unit determines the azimuth angle φold indicating the line-of-sight direction before switching based on the formula φold = Dold + θ, and the reference direction of the new route Based on the azimuth angle Dnew corresponding to, the azimuth angle φnew indicating the line-of-sight direction after switching is determined based on the formula φnew = Dnew.

(12) According to a twelfth aspect of the present invention, in the video presentation device according to the eighth to eleventh aspects described above,
The position / direction update unit has a function of giving a marker display instruction to the moving video reproduction unit to display a marker indicating each path connected to the node when the viewpoint reaches the node.
Based on the marker display instruction, the moving video playback unit performs a process of superimposing individual markers on the visual field image when reaching the node,
The instruction input unit has a function of inputting a user instruction to select a specific marker,
The position / direction updating unit recognizes a route corresponding to the selected marker as a new route.

(13) According to a thirteenth aspect of the present invention, in the video presentation device according to the sixth, seventh, ninth to eleventh aspects described above,
Q (Fold (φ)) is the pixel value of the visual field image in the direction indicated by the azimuth angle φ cut out from the omnidirectional image before switching, and the pixel of the visual field image in the direction indicated by the azimuth angle φ cut out from the omnidirectional image after switching. When the value is Q (Fnew (φ)),
G = (1−α) · Q (Fold (φ)) + α · Q (Fnew (φ))
Based on the following formula, the process of generating the blend image by defining the pixel value G of the blend image is performed by gradually changing the azimuth angle φ from φold to φnew and gradually increasing the blend ratio α from 0 to 1. It is repeatedly executed while changing, and a series of generated blend images is output as a moving image.

(14) According to a fourteenth aspect of the present invention, in the video presentation device according to the sixth, seventh, ninth to eleventh, and thirteenth aspects described above,
The moving video playback unit and the switching video playback unit display the field of view within the range of azimuth angles “φ−Δ / 2” to “φ + Δ / 2” (where Δ is a predetermined cut-off angle) from the read omnidirectional images. A field-of-view image to be constructed is cut out.

(15) According to a fifteenth aspect of the present invention, in the video presentation device according to the first to seventh aspects described above,
The position / direction updating unit automatically updates the viewpoint position and the line-of-sight direction based on a predetermined rule set in advance.

  (16) In a sixteenth aspect of the present invention, the moving picture presentation apparatus according to the first to fifteenth aspects described above is configured by incorporating a dedicated program into a computer.

(17) According to a seventeenth aspect of the present invention, a place having a first route and a second route connected via a node moves from the first route to the second route via the node. In a video presentation method for presenting a field of view in a predetermined line-of-sight direction viewed from a viewpoint as a video,
The first movement indicating the field of view in the first line-of-sight direction viewed from the viewpoint toward the node along the first path, based on the first omnidirectional video imaged along the first path. A first moving video playback stage to play the video;
A switching video playback stage for playing back a switching video showing a field of view obtained when the computer gradually changes the line-of-sight direction with the viewpoint stationary at the node;
The second movement indicating the field of view in the second line-of-sight direction as viewed from the viewpoint of moving away from the node along the second path based on the second omnidirectional video imaged along the second path A second moving video playback stage for playing back the video;
And
In the switching moving image reproduction stage, the omnidirectional image before switching that forms the last frame of the first omnidirectional moving image is obtained by gradually changing the viewing direction from the first viewing direction to the second viewing direction. Obtained by gradually changing the line-of-sight direction from the first line-of-sight direction to the second line-of-sight direction for the first panning video and the switched omnidirectional image constituting the first frame of the second omnidirectional video. A moving image obtained by blending the second panned moving image with a predetermined ratio that gradually increases the ratio of the second panned moving image is reproduced as a switching moving image.

  According to the present invention, when the viewpoint changes from the old route to the new route through the node, the blend ratio gradually changes from the last scene of the old route movie to the first scene of the new route movie. In addition, the switching moving image is presented, and in the switching moving image, a pan video gradually changing from the sight line direction of the old route to the sight line direction of the new route is presented. Therefore, it is possible to present a moving image in a manner that does not make the viewer feel uncomfortable due to the discontinuity of the image at an actual place including a plurality of routes.

It is a top view of the art museum used as the presentation subject by the animation presentation device concerning the present invention (hatching shows a wall surface). It is a top view which shows each node and each path | route which comprise the visit path of the museum shown in FIG. 4 is a moving image frame chart showing each frame constituting a moving image obtained by photographing along one route R. It is a side view which shows an example of the omnidirectional camera utilized in order to image | photograph the moving image shown in FIG. It is a top view of the distortion circular image image | photographed using the omnidirectional camera shown in FIG. FIG. 6 is a plan view showing an example of a rectangular panoramic image (omnidirectional image) created based on the distorted circular image shown in FIG. 5. 6 is a plan view showing a definition example of a viewing direction (azimuth angle) φ (i) at a point P (i) on a route R. FIG. It is a top view which shows the operation | work which cuts out the visual field image Q (P (i), (phi) (i)) corresponding to the visual line direction of azimuth | direction angle (phi) = 90 degrees from the panoramic image (omnidirectional image) shown in FIG. It is a top view of the controller which inputs a user's instruction | indication. It is a block diagram which shows the structure of the moving image presentation apparatus 100 which concerns on fundamental embodiment of this invention. It is a top view which shows the example which represented the node and path | route shown in FIG. 2 on XY two-dimensional coordinate. It is a figure which shows an example of the route information RT stored in the basic information storage part 120 in the moving image presentation apparatus 100 shown in FIG. It is a figure which shows an example of the moving image data corresponding | compatible information MC stored in the basic information storage part 120 in the moving image presentation apparatus 100 shown in FIG. FIG. 2 shows an image (FIG. (A)) displayed at the time of arrival from the node N1 to the node N2 and an image (FIG. (B)) displayed at the time of departure toward the node N5 in the visit route shown in FIG. It is a top view. FIG. 15 is a plan view showing a pre-switching omnidirectional image Fold and a post-switching omnidirectional image Fnew from which the two visual field images QA and QB shown in FIG. 14 are cut out. It is a time chart which shows the process of the switching process at the time of changing from the old path | route in this invention to a new path | route. It is a top view which shows the 1st basic method for producing | generating a switching image. It is a top view which shows the 2nd basic method for producing | generating a switching image. From the omnidirectional images Fold and Fnew, the visual field images QA and QB with respect to the azimuth angle φ = 90 ° which is a common line-of-sight direction are cut out and synthesized with a blend ratio of 1: 0 to generate a blend image G (90 °). It is a top view which shows a process. From the omnidirectional images Fold and Fnew, the visual field images QC and QD with respect to the azimuth angle φ = 120 ° which is a common line-of-sight direction are cut out and synthesized at a blend ratio of 2/3: 1/3. It is a top view which shows the process which produces | generates. From the omnidirectional images Fold and Fnew, the field-of-view images QE and QF for the azimuth angle φ = 150 °, which is a common line-of-sight direction, are cut out and synthesized at a blend ratio of 1/3: 2/3. It is a top view which shows the process which produces | generates. From the omnidirectional images Fold and Fnew, the visual field images QY and QZ for the azimuth angle φ = 180 °, which is a common viewing direction, are cut out and synthesized with a blend ratio of 0: 1 to generate a blend image G (180 °). It is a top view which shows a process. It is a time chart which shows the specific process of the switching process at the time of changing from the old path | route in this invention to a new path | route. It is a top view which shows transition of the gaze direction by the path | route transition at the time of performing the setting which makes a gaze direction correspond with a moving direction. It is a top view which shows transition of the gaze direction by the path | route transition at the time of setting a gaze direction arbitrarily. It is a figure which shows the type | formula which determines the gaze direction before and behind switching through a node. It is a top view which shows the modification using a curved path | route.

  Hereinafter, the present invention will be described based on the illustrated embodiments.

<<< §1. Basic method of video presentation >>>
Here, first, a basic method of video presentation employed by the video presentation device according to the present invention will be described. The video presentation device according to the present invention is a device that presents a field of view as viewed from a viewpoint that moves along a route for an actual location, and the location to be presented is indoors, even outdoors. It does not matter. Therefore, this device, for example, presents various historical sites and scenery of sightseeing spots on the display screen, and allows the user (viewer) to feel as if they are walking freely around the historical sites and sightseeing spots. It is possible to make it. In the following description, for the sake of convenience, a simple example will be described in which a moving image is presented to the user in a state where the user is walking in a museum with a floor plan as shown in FIG.

  The museum shown in Fig. 1 is an actual facility for exhibiting paintings and sculptures. As shown in the figure, visitors who enter the museum can freely walk around the corridor and appreciate the exhibits, and then exit from the exit. You can do it. The hatched portion in FIG. 1 shows the wall surface of the corridor, and the picture is presented on this wall surface. In addition, sculptures are appropriately placed on the floor of the corridor. The video presentation device according to this embodiment is basically a device for presenting to the user an image of the actual museum, but the user can freely It has a function of simulating a state observed in an arbitrary direction and presenting it as a moving image.

  In this case, such a museum visit path is expressed by a plurality of nodes and paths connecting these nodes, and the entire visit path is expressed by route information having information on each node and each path. To do. FIG. 2 is a plan view showing the nodes and paths constituting the visit path of the museum shown in FIG. 1. Each black dot indicates a node, each straight line indicates a path, and each broken line indicates an outline of the actual visit path. Yes. As illustrated, in this example, six nodes N1 to N6 and six routes R1 to R6 are defined. Each path is defined as a straight line connecting these end points with a pair of nodes as end points.

  It is left up to the device designer to define the nodes on the visit path and the paths between the nodes, but in general, the branch point, turn corner, start point, and end point of the visit path Nodes can be defined, and routes can be defined between necessary nodes according to the actual visiting route. In the case of the illustrated example, each of the paths R1 to R6 is a straight line. However, when the actual path includes a curve, a path composed of a curve may be defined using a Bezier curve or the like. However, if each path is a straight line, the path is also specified only by specifying the nodes at both end points. Therefore, in practice, it is preferable that all the paths are constituted by straight lines.

  The route information indicating the entire visiting route includes information indicating the position of each node (for example, coordinate values on the XY coordinate system) and information indicating a route between a pair of nodes (for example, “nodes N1, N2: route R1”). And the like). A specific configuration example of the route information will be described later.

  In order to present a moving image for such a visit route, it is only necessary to prepare moving images taken while moving along the individual routes R1 to R6. FIG. 3 is a moving image frame chart showing a frame group constituting moving image data M obtained by photographing along one route R. The illustrated example shows a frame structure of a moving image obtained by shooting while moving from the node A to the node B along the route R connecting the pair of nodes A and B.

  As shown in the figure, this moving image data M is obtained by discretely defining points P (1), P (2),..., P (i),. ), P (n) by the aggregate of the frame images F (1), F (2),..., F (i),. Composed. Here, the point P (1) coincides with the node A, the point P (n) coincides with the node B, and unique frame images are respectively taken for a total of n points. Each frame image is a still image, but a moving image can be presented by reproducing the frame image in order. The interval between the points is determined by the number of frames per second of the photographing apparatus and the moving speed of the photographing apparatus. For example, if shooting is performed while moving a 30-frame / second imaging device at a speed of 150 mm / second, the interval between points is 5 mm.

  In the case of the example shown here, a photographing device as shown in FIG. 4 is used as a device for photographing a moving image. As shown in the figure, this photographing apparatus is constituted by a fisheye lens 10, a video camera 20, a data processing unit 30, and a carriage 40, and functions as an omnidirectional camera capable of photographing an image having a 360 ° field of view. As shown in FIG. 4, the fisheye lens 10 is disposed on the uppermost part of the apparatus, and forms an image of a hemispherical field located above a predetermined horizontal plane on the imaging surface of the video camera 20. If such a configuration is adopted, a distorted circular image as shown in FIG. 5 is formed on the imaging surface of the video camera 20 (both the hatched donut-shaped portion and the white circular portion inside thereof are formed). And a distorted circular image taken with the fisheye lens 10). The celestial vertex O at the center of the distorted circular image is a point (corresponding to the ceiling of the corridor) at a position vertically above the imaging device shown in FIG. 4, and the outer periphery of the distorted circular image is shown in FIG. This corresponds to the lower end position of the field of view taken by the apparatus (field of view around 360 °).

  In the case of this example, the museum's hall displaying paintings and sculptures is the display target, so the image of the white circular portion in the vicinity of the top vertex O (corresponding to the image of the ceiling portion of the corridor) is not used. Only the donut-shaped area hatched in the figure will be used. FIG. 6 shows a rectangular panoramic image obtained by cutting out a region having an elevation angle equal to or smaller than a predetermined reference value (a hatched region in the drawing) from the distorted circular image shown in FIG. 5 and applying distortion correction thereto. FIG. If this panoramic image is an image obtained based on a distorted circular image photographed at the point P (i), the image constitutes the i-th frame F (i) of the moving image data M. become. The azimuth angle of 0 ° to 360 ° shown on the lower side of the panoramic image corresponds to the azimuth angle of 0 ° to 360 ° shown on the circumference of the distorted circular image in FIG. That is, the panoramic image shown in FIG. 6 is an image in which the viewpoint is viewed at the position of the fisheye lens 10 shown in FIG.

  The data of the distorted circular image taken by the video camera 20 is sent to the data processing unit 30. The data processing unit 30 is configured by a computer, and distorted circular image data is stored in the hard disk device. In the example shown here, the data processing unit 30 has a function of performing a process of creating a panoramic image from the distorted circular image. Therefore, the distorted circular image as shown in FIG. 5 is converted into a panoramic image as shown in FIG. 6 in the data processing unit 30 and stored in the hard disk device. In the case of the example shown here, the panoramic image obtained in this way is used as an omnidirectional image constituting each frame of the moving image data M.

  Since the distorted circular image shown in FIG. 5 is a distorted donut-shaped image, the panoramic image shown in FIG. 6 is a rectangular image. Therefore, the data processing unit 30 performs image conversion processing with distortion correction. There is a need to do. Since such image conversion processing is a known technique, detailed description thereof is omitted here. Normally, even if an image conversion process with such distortion correction is performed, it is difficult to completely remove the distortion, so that a slight distortion remains in the obtained panorama image. Distortion correction to a level that does not hinder it is possible.

  The procedure for obtaining the panoramic image around the shooting device shown in FIG. 4 at one point on the route R has been described above. The moving image is moved by the video camera 20 while moving the shooting device along the route R. If shooting is performed, a panoramic image (omnidirectional image) as shown in FIG. 6 is used as one frame, and moving image data M composed of a plurality of consecutive frames is obtained. The moving image data M composed of n frames F (1) to F (n) shown in FIG. 3 is obtained by such shooting.

  However, when presenting a moving image to the user, it is not preferable to display the panoramic image shown in FIG. 6 as it is as one frame image. This panoramic image (omnidirectional image) is a peculiar image including all information of the surrounding 360 °, and is far from a general image entering the human visual field. Therefore, in practice, it is necessary to determine a specific line-of-sight direction, cut out only a part of the visual field in the line-of-sight direction, and present it to the user.

  FIG. 7 is a plan view illustrating a definition example of the line-of-sight direction at the specific point P (i) on the route R. The line-of-sight direction is defined as the direction of the line-of-sight vector E (i) extending from the point P (i). In the example shown here, the direction of the line-of-sight vector E (i) is defined using the azimuth angle φ on the two-dimensional plane including the route R. That is, in the illustrated example, the direction from the point P (i) toward the upper side of the figure is 0 °, and the azimuth angle φ (i) from 0 ° to 360 ° is defined clockwise. In short, the angle φ (i) (where 0 ° ≦ φ (i) <360 °) formed by the line-of-sight vector E (i) is defined as the azimuth angle with the axis oriented in the 0 ° direction as the reference axis. Become.

  The azimuth angle φ (i) indicating the line-of-sight direction is a parameter indicating in which direction the virtual user located at the i-th point P (i) on the route R points the line of sight. In the case of the illustrated example, the path R is a straight line extending in the left-right direction (horizontal direction) in the figure, and therefore the azimuth angle φ (i) = 90 ° indicates the direction toward the node B, and the azimuth angle φ (i) = 270 ° indicates a direction toward the node A.

  Thus, if the point P (i) indicating the current position of the virtual user and the azimuth angle φ (i) indicating the line-of-sight direction are determined, the visual field image Q (P (i), φ (i) viewed from the virtual user is determined. )) Can be extracted. FIG. 8 is a plan view showing an operation of cutting out such a field image Q (P (i), φ (i)). A visual field image Q (P (i), φ (i)) surrounded by a thick line in the figure indicates that the virtual user located at the point P (i) on the route R shown in FIG. ) = 90 ° is an image that should appear in the field of view of the virtual user when it is oriented in the direction indicated by 90 ° (the direction toward Node B).

  In order to obtain such an image, a field of view within the range of azimuth angles “φ (i) −Δ / 2” to “φ (i) + Δ / 2” is configured from the i-th frame image F (i). What is necessary is just to perform the process which cuts out the part to perform. Here, Δ is a predetermined cut-out angle, which is an angle corresponding to the lateral width of the cut out view field image Q (P (i), φ (i)). In the case of the illustrated example, since the setting is such that the azimuth angle φ = 90 °, the visual field image corresponding to the horizontal width Δ is cut out with the position of φ = 90 ° as the center.

  When a virtual user who is moving from the point P (i) shown in FIG. 7 toward the node B is looking in the moving direction (the direction toward the node B), to present a moving image that enters the field of view. , Cut out the field images shown at the azimuth angle φ = 90 ° from the frame images F (i), F (i + 1), F (i + 2),..., And display these field images in order (according to the shooting order). What is necessary is just to perform the process. Then, in the scenery shown in the thick frame in FIG. 8, a moving image in which the sculpture image placed near the azimuth angle of 90 ° gradually approaches is presented.

  Of course, the virtual user can change the line-of-sight direction (azimuth angle φ) during movement as needed. When a change occurs in the line-of-sight direction (azimuth angle φ), the cut-out position indicated by a thick frame in FIG. 8 moves to the left and right. The virtual user can also make the line-of-sight direction reverse to the movement direction. For example, when a virtual user moving from the point P (i) shown in FIG. 7 toward the node B performs the setting of the azimuth angle φ = 270 °, the moving direction is the direction toward the node B. The direction is a direction toward the node A, and a field-of-view image corresponding to the horizontal width Δ is cut out from the frame image as shown in FIG. 8 with the position at φ = 270 ° as the center. As a result, a moving image in which the door located in the vicinity of the azimuth angle 270 ° gradually moves away is presented.

  On the other hand, the user can also set the moving direction to an arbitrary direction. For example, for the virtual user located at the point P (i) shown in FIG. 7, when the moving direction is set to the direction toward the node A and the line-of-sight direction is set to the azimuth angle φ = 270 °, FIG. A moving image in which the door located near the azimuth angle of 270 ° gradually approaches is presented. In order to present such a moving image, a visual field image indicated by an azimuth angle φ = 270 ° is cut out from each frame image F (i), F (i-1), F (i-2),. A process for displaying these visual field images in order (in the order opposite to the shooting order) may be performed.

  As described above, the basic method for presenting a moving image when a virtual user moves on one route R (route between nodes A and B) shown in FIG. 7 has been described. A similar video presentation process may be performed for the route. For example, the visit path of the museum shown in FIG. 2 is composed of six routes R1 to R6. If each of the routes R1 to R6 is filmed in the same manner as in the above example, Even if the virtual user advances in which direction on the viewing path in which direction, and in which direction the line of sight is directed, appropriate video presentation can be performed.

  As described above, the controller 50 as shown in FIG. 9 is prepared in order to efficiently perform the instruction input for moving the virtual user in the desired direction on the viewing path and directing the line of sight in the desired direction. Is preferred. The controller 50 is a general device used as an input device for a computer game, and is provided with five buttons B1 to B5 on the right side, three buttons B6 to B8 on the center, and one button B9 on the left side. ing. Here, the illustrated button B1 functions as a forward button, the button B2 functions as a backward button, the button B3 functions as a left button, the button B4 functions as a right button, and the button B5 functions as a stop button.

  Here, the forward button B1 functions as a button for giving an instruction to start an operation of moving the virtual user in the first direction on the route, and the backward button B2 moves the virtual user in the second direction on the route. The stop button B5 functions as a button for giving an instruction to stop the virtual user.

  For example, when the virtual user is in a stopped state at the point P (i) on the route R shown in FIG. 7, when the forward button B1 is pressed, the virtual user is directed toward the node B at a predetermined speed. The moving video starts to play. That is, the field-of-view image indicated by a predetermined azimuth angle φ is cut out from each frame image F (i), F (i + 1), F (i + 2),... ) Is displayed. When the stop button B5 is pressed, the reproduction of the moving image is stopped, and the frame image displayed at that time is displayed as a still image.

  On the other hand, when the virtual user is stopped at the position of the point P (i) on the route R shown in FIG. 7, when the backward button B2 is pressed, the virtual user is directed toward the node A at a predetermined speed. The moving video starts to play. That is, the field-of-view image indicated by a predetermined azimuth angle φ is cut out from each frame image F (i), F (i-1), F (i-2),. They are displayed in the reverse order of the shooting order. When the stop button B5 is pressed, the reproduction of the moving image is stopped, and the frame image displayed at that time is displayed as a still image.

  In addition, when the backward button B2 is pressed in a state in which a moving image that is moving toward the node B at a predetermined speed is being played by pressing the forward button B1, the virtual user's The moving direction is reversed in the direction toward the node A. Similarly, when the forward button B1 is pressed in a state in which a moving image that is moving toward the node A at a predetermined speed is being played by pressing the backward button B2, the virtual user at that time is pressed. Will be reversed in the direction toward Node B.

  Here, for convenience of explanation, they are referred to as the forward button B1 and the backward button B2. However, since the virtual user's line-of-sight direction can be directed to any direction of 360 °, “forward” or “reverse” referred to here. Does not necessarily indicate the front and rear along the visual line direction of “going in the visual line direction” or “going in the opposite direction to the visual line direction”, but “a direction close to the visual line direction among the two directions along the path R”. (Proceed toward the direction close to the direction in which the virtual user is facing) ”. For example, in the case of the example shown in FIG. 7, the line-of-sight vector E (i) faces the direction closer to the node B than the node A, so the forward button B1 moves to the right in the figure (the direction toward the node B). Will give instructions to proceed.

  In order to change the viewing direction of the virtual user, an operation of pressing the left button B3 or the right button B4 may be performed. Whether the virtual user is moving or stopped, the azimuth angle φ is updated so that the line-of-sight direction changes to the left at a predetermined speed while the left button B3 is pressed, and the right button B4 is pressed. Meanwhile, the azimuth angle φ is updated so that the line-of-sight direction changes to the right at a predetermined speed. For example, if the azimuth angle φ indicating the line-of-sight direction is set to 90 °, the azimuth angle φ is decreased to 89 °, 88 °, 87 °,... While the left button B3 is pressed ( When the angle reaches 0 °, a process of decreasing from 360 ° is performed, and while the right button B4 is pressed, the azimuth angle φ is increased to 91 °, 92 °, 93 °,. When the angle reaches 360 °, a process of increasing from 0 ° is performed.

  Thus, by operating the buttons B1 to B5, the user can freely walk in the virtual space constituting the corridor of the museum and turn his gaze in an arbitrary direction. On the display screen, the visual field image viewed from the virtual user is displayed as a moving image based on the user's instruction.

  As shown in FIG. 2, the museum's visiting path is composed of a plurality of routes R1 to R6 having nodes as both end points. Thus, in the video presentation device according to the present invention, when the virtual user reaches any node position, the marker for automatically stopping the virtual user at the node and selecting a new route on the display screen. Is displayed. That is, when the user performs a process of selecting a new route starting from the reaching node, and the user inputs a new route selection instruction, the video is reproduced to show the state in which the virtual user moves to the new route. I have to. By adopting such a method, the user can give an instruction to proceed in a desired direction even when there is a path including a branch as in the vicinity of the nodes N2 and N5 shown in FIG.

<<< §2. Basic configuration of moving picture presentation apparatus according to the present invention >>
Now, in §1, the basic method of video presentation employed by the video presentation device according to the present invention has been described. If a moving image is presented using this basic method, an image such as a visual field image Q (P (i), φ (i)) shown in a thick frame in FIG. 8 is displayed on the display screen. It will be. In addition, a moving image is presented in accordance with the movement of the virtual user and the change in the viewing direction. Here, a basic configuration of a moving picture presentation apparatus having a function of performing such moving picture presentation will be described.

  FIG. 10 is a block diagram showing the configuration of the video presentation device 100 according to the basic embodiment of the present invention. As shown in the figure, this video presentation device 100 includes a video data storage unit 110, a basic information storage unit 120, a video playback unit 130, a playback instruction unit 140, and an instruction input unit 150, which are connected via nodes. For an actual place including a plurality of routes, the function of presenting a field of view in a predetermined line-of-sight direction viewed from a viewpoint moving along a predetermined route as a moving image is achieved. The moving image is displayed on the screen of the display device 200 based on the display signal given from the moving image presentation device 100.

  The moving image data storage unit 110 is a component that stores moving image data shot using an omnidirectional camera while moving along a predetermined shooting direction for each path. In the illustrated example, moving image data M1 to M6 for the six routes R1 to R6 shown in FIG. 2 are stored. Each moving image data is configured by arranging a large number of frames along the time axis with an omnidirectional image having a field of view of 360 ° as one frame. For example, the moving image data M1 is an aggregate F of n frames obtained by shooting an omnidirectional camera as shown in FIG. 4 from the nodes N1 to N2 shown in FIG. 1) to F (n).

  Specifically, as described in §1, using a omnidirectional camera equipped with a fisheye lens or an omnidirectional mirror as shown in FIG. 4, a hemispheric field of view located above a predetermined horizontal plane is photographed. 5 is obtained by extracting a distorted circular image as shown in FIG. 5 and cutting out an area whose elevation angle is equal to or smaller than a predetermined reference value (a donut-shaped area hatched in FIG. 5) from the distorted circular image and correcting the distortion. A process for generating a rectangular panoramic image (an omnidirectional image as shown in FIG. 6) may be performed. When a large number of panoramic images are obtained by shooting while moving the omnidirectional camera along the route, moving image data can be configured by arranging each panoramic image as one frame and arranging the panoramic images in units of frames. The moving image data storage unit 110 may store moving image data for each route obtained in this way. Of course, the moving image data storage unit 110 uses a system that shoots using an omnidirectional camera equipped with a plurality of cameras for normal shooting and creates a panoramic image by synthesizing the obtained images. It is also possible to prepare moving image data to be stored in the.

  The basic information storage unit 120 is a component that stores the route information RT and the moving image data correspondence information MC for the visit route as shown in FIG. Here, the route information RT is information indicating the connection state of each route through the node and the reference direction of each route, which is information defining the geometric structure of a plurality of routes as shown in FIG. is there. Referring to the route information RT, a route R1 exists between the nodes N1 and N2, and three routes R1, R2, and R6 are connected to the node N2, and the routes R1 and R2 are in the horizontal direction in the figure. It can be understood that the route R6 is orthogonal to this.

  In the case of the embodiment shown here, the visit path shown in FIG. 2 is defined on an XY two-dimensional coordinate system as shown in FIG. That is, the nodes N1 to N6 are represented by points having coordinate values (x, y) on this two-dimensional coordinate system, and the paths R1 to R6 are each represented as a path connecting specific nodes. In the case of the illustrated example, for example, the node N1 is defined as a point having the coordinate value (10, 60), the node N2 is defined as a point having the coordinate value (30, 60), and the route R1 is defined as the node N1. , N2 is defined as a straight line.

  In addition, on the XY two-dimensional coordinate system, as shown in the upper part of FIG. 11, the Y-axis positive direction is 0 °, the X-axis positive direction is 90 °, the Y-axis negative direction is 180 °, and the X-axis negative direction is An azimuth angle of 270 ° is defined, and an arbitrary direction can be indicated by an azimuth angle of 0 ° to 360 °. Therefore, the direction of each path can also be indicated by this azimuth. Here, the direction of the route in consideration of the moving direction is referred to as the reference direction of the route. For example, in the case of the illustrated route R1, the reference direction when moving from the node N1 to the node N2 can be represented by an azimuth angle of 90 °, and the reference direction when moving from the node N2 to the node N1 can be expressed by an azimuth angle of 270 °. .

  FIG. 12 is a diagram showing route information RT for the specific route shown in FIG. The route information RT includes position information indicating the positions of the individual nodes N1 to N6 and connection information indicating a path connecting the nodes. As described above, the node position information is configured by information indicating coordinate values (x, y) on the XY two-dimensional coordinate system. That is, the position information shown in the upper part of FIG. 12 indicates the XY coordinate values of the nodes N1 to N6 in FIG.

  On the other hand, the route connection information is configured by information indicating whether or not there is a route connecting each other for any combination of two nodes in all nodes. For example, the connection information shown in the lower part of FIG. 12 has six nodes N1 to N6 in the horizontal direction and six nodes N1 to N6 in the vertical direction, and has a total of 36 elements. A number “0” or “1” is written in the matrix. Here, “0” indicates that there is no path connecting the two node combinations, and “1” indicates that such a path exists.

  For example, in the first row, the number “010000” is described. This is because there is a path connecting the nodes “N1-N2”, but the node “N1-N1”. ”,“ N1-N3 ”,“ N1-N4 ”,“ N1-N5 ”, and“ N1-N6 ”indicate that there is no path connecting them. Here, the diagonal elements of the matrix always indicate “0” because they indicate combinations of the same nodes. Referring to the connection relationship through the path of each node in the path shown in FIG. 11, it can be easily understood that the connection information shown in the lower part of FIG. 12 indicates the path in FIG.

  Thus, “1” in each column of the connection information shown in the lower part of FIG. 12 indicates a route having two specific nodes as both end points, and the position information shown in the upper part of FIG. As a result, the route information RT eventually includes information indicating the reference direction of each route. For example, the reference direction of the route R1 when moving from the node N1 toward the node N2 is the direction from the point of the coordinate value (10, 60) to the point of the coordinate value (30, 60). It can be recognized that the azimuth angle is 90 °.

  The basic information storage unit 120 shown in FIG. 10 stores route information RT as shown in FIG. After all, in this example, the route information RT includes position information indicating the coordinate values on the two-dimensional coordinate system of the nodes located at both ends of each route, and the reference direction of each route is the route information of the route. It is defined by the direction of a straight line connecting two nodes located at both ends. In addition, the connection state of each path is defined by connection information indicating whether or not there is a path connecting each other for any combination of two nodes in all nodes.

  On the other hand, the moving image data correspondence information MC stored in the basic information storage unit 120 is information indicating moving image data corresponding to each route. FIG. 13 is a diagram illustrating an example of the moving image data correspondence information MC. In this example, the moving image data correspondence information MC indicates that the moving image data M1 to M6 correspond to the six paths R1 to R6, respectively, and also plays a role of indicating the shooting direction for each moving image data. That is, with reference to the shooting direction column in the illustrated table, for example, it can be recognized that the moving image data M1 for the route R1 is a moving image shot while moving the omnidirectional camera from the node N1 toward the node N2. This information indicating the shooting direction is used when determining the frame order when moving moving images are reproduced.

  With reference to the moving image data correspondence information MC, for example, it can be recognized that the moving image data M1 in the moving image data storage unit 110 may be reproduced in order to present a moving image on the route R1. Moreover, since it can be recognized that the shooting direction of the moving image data M1 is “N1 → N2”, when the viewpoint is moving from the node N1 toward N2, the moving image data M1 is reproduced in the frame order at the time of shooting. If the viewpoint is moving from the node N2 toward the node N1, it can be recognized that the moving image data M1 may be reproduced in the reverse frame order from that at the time of shooting.

  The table shown in FIG. 13 includes a “route” column for indicating the route symbols “R1 to R6” for convenience of explanation, but the nodes at both ends are specified by the information in the “shooting direction” column. As a result, the route is also specified, so the “route” field may be omitted in practice. Of course, if the route symbol “R1 to R6” of the corresponding route is described instead of “1” in the connection information column shown in the lower part of FIG. 12, it is described in the “route” column shown in FIG. Correspondence with each path on the connection information shown in the lower part of FIG. 12 can be performed by the path symbols “R1 to R6”.

  Alternatively, instead of “1” in the connection information column shown in the lower part of FIG. 12, if the moving image data and the information indicating the shooting direction for the corresponding route are directly described, the connection information constituting the route information RT. Since it is possible to embed the moving picture data correspondence information MC, it is not necessary to separately prepare the moving picture data correspondence information MC as shown in FIG. For example, in the connection information shown in the lower part of FIG. 12, if N1 to N6 arranged in the vertical direction on the left side of the table are defined as departure nodes and N1 to N6 arranged in the horizontal direction on the upper side of the table are defined as arrival nodes, "1" in the column of the second row and the second column indicates the route R1 in the direction from the node N1 to N2, and "1" in the column of the second row and the first column indicates the direction from the node N2 to the N1. Route R1.

  Therefore, in the connection information shown in the lower part of FIG. 12, the information “M1 (+)” (information indicating that the moving image data M1 is reproduced in the frame order at the time of shooting) in the column of the first row and the second column. And the information “M1 (−)” (information indicating that the moving image data M1 is to be reproduced in the reverse frame order from that at the time of shooting) is written in the second row and first column. The table of the connection information is a table including the moving image data correspondence information MC. Therefore, it is possible to recognize the moving image data corresponding to each route and its shooting direction without separately preparing the moving image data correspondence information MC as shown in FIG.

  In the above, several specific data structure examples of the route information RT and the moving image data correspondence information MC stored in the basic information storage unit 120 have been described. Of course, in implementing the present invention, the route information RT and the moving image data correspondence information The data structure of the MC is not limited to the example described above.

  Next, the video playback unit 130 in the video presentation device 100 shown in FIG. 10 will be described. The video playback unit 130 is a component that performs video playback based on the video data M1 to M6 stored in the video data storage unit 110. Basically, the video playback unit 130 receives specific video data from the video data storage unit 110. A specific frame F (i) of Mj is read out, and as shown in FIG. 8, a specific visual field image Q (P (i), φ (i)) is cut out from the specific frame F (i) and cut out. The process of providing the image Q to the display device 200 is repeatedly executed at a predetermined cycle. For example, if a process of creating and providing a new visual field image Q at a cycle of 30 times per second is performed, a smooth moving image is presented on the screen of the display device 200.

  An important feature of the apparatus according to the present invention is that a moving video playback unit 131 and a switching video playback unit 132 are provided in the video playback unit 130. Here, the moving video playback unit 131 is a component for presenting a video viewed from the viewpoint of moving along a specific route (here, referred to as a moving video). This is a component for presenting a moving image (herein referred to as a switching moving image) viewed from a stationary viewpoint to the node when a transition is made from the old route to the new route. Specific functions of these components will be described in detail later.

  The reproduction instruction unit 140 refers to the route information RT and the moving image data correspondence information MC in the basic information storage unit 120 while managing the viewpoint position and the line-of-sight direction, and is a component that issues a reproduction instruction to the moving image reproduction unit 130 As shown in the figure, it has a position / direction update unit 141, a moving video playback instruction unit 142, and a switching video playback instruction unit 143.

  The position / direction updating unit 141 is a component that holds data indicating the viewpoint position and the line-of-sight direction on the route of the virtual user (viewer) and sequentially updates the data. Based on the viewpoint position and line-of-sight direction (current viewpoint position and line-of-sight direction) updated by the position / direction updating unit 141, a specific visual field image is displayed on the screen of the display device 200.

  The moving video playback instruction unit 142 gives the moving video playback unit 131 a playback instruction of a moving video viewed from the moving viewpoint when the viewpoint is moving on the route by the update by the position / direction update unit 141. The switching video playback instruction unit 143 has a function, and when the viewpoint changes from the old route to the new route through the node by the update by the position / direction updating unit 141, the switching video playback instruction unit 143 is a switching video viewed from the viewpoint stationary at the node. Is provided to the switching video playback unit 132. The moving video reproduction unit 131 reproduces the moving video based on the instruction given from the moving video reproduction instruction unit 142, and the switching video reproduction unit 132 uses the switching video reproduction instruction unit 143 based on the instruction given from the switching video reproduction instruction unit 143. Perform playback.

  On the other hand, the instruction input unit 150 is a component that inputs a user instruction. Specifically, for example, a controller 50 as illustrated in FIG. 9 and an input interface that receives a user instruction via the controller 50 are provided. Can be configured. The position / direction updating unit 141 updates the viewpoint position and the line-of-sight direction according to the user's instruction input by the instruction input unit 150.

  In the embodiment shown here, the viewpoint position is defined by the coordinate value P (x, y) of the point on the XY two-dimensional coordinate system, and the line-of-sight direction is defined by the azimuth angle φ on the XY two-dimensional coordinate system. The Therefore, the position / direction updating unit 141 stores the coordinate value P (x, y) indicating the viewpoint position and the azimuth angle φ indicating the line-of-sight direction, and sequentially updates them. The update of the viewpoint position P (x, y) and the line-of-sight direction φ is performed based on the user instruction given from the instruction input unit 150 and the route information RT stored in the basic information storage unit 120.

  For example, let us consider a case where the entrance position is set as the starting point in the visit route shown in FIG. In this case, the position / direction updating unit 141 may be set with the coordinate value P (10, 60) as the initial viewpoint position and the azimuth angle φ = 90 ° as the initial line-of-sight direction. In such a setting, in FIG. 11, the position of the node N1 (10, 60) is the initial viewpoint position, and the direction toward the node N2 is the initial line-of-sight direction.

  In this state, when the user presses the forward button B1 of the controller 50 shown in FIG. 9, the position / direction updating unit 141 recognizes that the forward instruction is given from the user, and performs an update process according to this. become. That is, by referring to the route information RT, it can be seen that the user is currently on the route R1, and that it is possible to move forward by gradually increasing the X coordinate value of the viewpoint position. Therefore, the coordinate value P (10 indicating the current viewpoint position) , 60) is sequentially updated as P (11, 60), P (12, 60), P (13, 60),. When the viewpoint arrives at the node, the position / direction updating unit 141 temporarily stops the movement of the viewpoint. For example, in the case of the above example, when the viewpoint position reaches the coordinate value P (30, 60), it means that the node N2 has been reached, and thus updating of the viewpoint position is temporarily stopped.

  Note that the user can also give an instruction to change the line-of-sight direction while moving or standing still. For example, when the user presses the right button B4 of the controller 50 shown in FIG. 9, the position / direction update unit 141 recognizes that the user has given a right direction instruction, and performs an update process corresponding thereto. Become. That is, the azimuth angle φ = 90 ° indicating the current line-of-sight direction is sequentially updated as 91 °, 92 °, 93 °,.

  Now, let us consider a case where the viewpoint is moved from the node N1 to the node N2 in the state of view shown in FIG. 11 with the line-of-sight direction fixed at φ = 90 °. In this case, the position / direction updating unit 141 performs a process of updating the viewpoint position from P (10, 60) to P (30, 60). The moving video playback instruction unit 142 recognizes that the viewpoint is moving on the route R1 by the update process, and gives the moving video playback unit 131 an instruction to play the moving video viewed from the viewpoint moving on the route R1. . Specifically, the image data M1 is read out in the frame order at the time of shooting, and an instruction to cut out and reproduce the visual field region corresponding to the viewing direction of φ = 90 ° is given. As a result, a moving image showing the field of view when moving from the node N1 to the node N2 in the direction of φ = 90 ° is presented on the screen of the display device 200.

  The moving video playback speed by the moving video playback unit 131 is adjusted to synchronize with the viewpoint moving speed by the position / direction update unit 141. Therefore, the visual field image Q corresponding to the field of view viewed from the viewpoint position P at each time point is always displayed on the screen of the display device 200. When the viewpoint arrives at the node N2, the node N2 A visual field image corresponding to the viewed field of view is displayed. Of course, the user can also give an instruction to pause while moving the route. In this case, the moving video playback unit 131 pauses the video playback based on the instruction from the moving video playback instruction unit 142, so that the field of view viewed from the stopped viewpoint position P is displayed on the screen of the display device 200. The still image shown will be displayed.

  FIG. 14A is a plan view showing a field-of-view image QA displayed on the screen of the display device 200 when it arrives at the node N2, and corresponds to an image showing a field of view viewed from the node N2 in the direction of the node N3. . At this time, the position / direction updating unit 141 stores information about the viewpoint position P (30, 60) and the line-of-sight direction φ = 90 °, and the moving moving image reproduction unit 131 stores the last frame F of the moving image data M1. As shown in FIG. 8, a visual field image QA centered at φ = 90 ° is cut out from the panoramic image (omnidirectional image) constituting (n), and processing for presenting this is performed. In the example shown here, since the distortion correction is also performed when the visual field image QA is cut out from the panoramic image (omnidirectional image), the visual field image QA shown in FIG. It has become.

  In the case of the embodiment shown here, when the viewpoint reaches the node, the position / direction updating unit 141 temporarily stops the viewpoint and displays a marker indicating each route connected to the node on the screen. The marker display instruction is given to the moving video reproduction unit 131. The moving moving image reproduction unit 131 has a function of performing processing for superimposing individual markers on the visual field image when the node arrives based on the marker display instruction. On the visual field image QA of FIG. 14A, markers mk1 to mk3 displayed in a superimposed manner are shown.

  As shown in FIG. 11, three paths are connected to the node N2, and the markers mk1 to mk3 correspond to these paths. That is, the marker mk1 corresponds to the route R2, the marker mk2 corresponds to the route R1, and the marker mk3 corresponds to the route R6. By referring to the route information RT, it is possible to recognize the geometric positional relationship between each node and each route. Therefore, the position / direction updating unit 141 determines in which position on the visual field image QA each marker should be displayed. I can decide. Accordingly, in FIG. 14A, the marker mk1 is displayed at a position suitable for the traveling direction to the route R2 (the straight traveling direction as it is), and the marker mk2 returns to the traveling direction to the route R1 (the route that has traveled so far). Direction: In this case, the old route R1 becomes the new route as it is) and the marker mk2 is displayed at a position suitable for the direction of travel to the route R6 (the direction of turning right at the node N2). ing.

  In addition, the instruction input unit 150 has a function of inputting a user instruction to select a specific marker, and the position / direction update unit 141 has a function of recognizing a route corresponding to the selected marker as a new route. Have. For example, when the controller 50 shown in FIG. 9 is used as the instruction input unit 150, the up, down, left, and right buttons B1 to B4 may correspond to the buttons for selecting the up, down, left, and right markers displayed on the visual field image QA. When any button is pressed, it can be determined that the marker corresponding to the button has been selected.

  Here, let us consider a case where the user performs an operation of selecting the marker mk3 by pressing the button B4. This user operation is an instruction to select the route R6 as a new route. In response to such an instruction, the position / direction updating unit 141 starts the process of shifting the viewpoint to the new route R6 and moving along the new route R6. Specifically, first, the viewpoint position is moved to the node N5 along the new route R6 after updating the line-of-sight direction to the reference direction of the new route R6 while the viewpoint position is stationary on the node N2. I do. In this example, it can be recognized that the reference direction of the new route R6 (the reference direction when heading from the node N2 to N5) is the direction of the azimuth φ = 180 ° by referring to the route information RT. The position / direction updating unit 141 updates the line-of-sight direction φ from 90 ° to 180 °.

  On the other hand, in response to the fact that the route R6 has been selected as the new route, the moving movie playback instruction unit 142 sets the azimuth angle φ = 180 ° in the frame order at the time of shooting the movie data M6 corresponding to the new route R6. An instruction to be cut out and played around the corresponding line-of-sight direction is given to the moving video playback unit 131. Thus, a moving image showing the field of view when moving from the node N2 to the node N5 in the direction of φ = 180 ° is presented on the screen of the display device 200.

  FIG. 14 (b) is a plan view showing a visual field image QZ displayed on the screen of the display device 200 at the time of departure from the node N2, and corresponds to an image showing a field of view viewed from the node N2 in the direction of the node N5. . At this time, the position / direction updating unit 141 stores information about the viewpoint position P (30, 60) and the line-of-sight direction φ = 180 °, and the moving moving image reproduction unit 131 stores the first frame F of the moving image data M6. A view image QZ centered at φ = 180 ° is cut out from the panoramic image (omnidirectional image) constituting (1), and a process of presenting this is performed. As described above, since distortion correction is performed when the view image QZ is cut out from the panoramic image (omnidirectional image), the view image QZ shown in FIG. 14B is also a regular image with almost no distortion. Yes.

  As described above, in the video presentation device according to the present invention, the video data to be reproduced is switched when the viewpoint arrives at the node. For example, in the case of the example shown in FIG. 14, the visual field image QA shown in FIG. 14 (a) is an image cut out from the last frame F (n) of the moving image data M1 corresponding to the old path R1, and the last of the moving image M1. 14 (b) is an image cut out from the first frame F (1) of the moving image data M6 corresponding to the new path R6, and the first image of the moving image M6. It is a scene. Therefore, in order to present a scene in which the virtual user moves from the old route R1 to the new route R6 via the node N2 as a moving image, the scene shown in FIG. 14 (a) is changed to the scene shown in FIG. 14 (b). Switching is necessary.

  The switching video playback unit 132 in the video presentation device 100 shown in FIG. 10 is a component that plays back a switching video for smoothly switching such scenes, and the switching video playback instruction unit 143 is a switching video playback unit. This is a component that gives an instruction to play a switching video to 132. Since all are important components for the present invention, specific functions thereof will be described in detail in §4.

  The configuration of the video presentation device 100 according to the basic embodiment of the present invention has been described above with reference to the block diagram of FIG. 10, but actually, each of these components uses a computer and its peripheral devices. Can be configured. For example, the moving image data storage unit 110 and the basic information storage unit 120 can be configured by a storage device such as a computer memory or a disk device, and the moving image reproduction unit 130 and the reproduction instruction unit 140 are based on a predetermined program. The instruction input unit 150 can be configured by a computer input device having a user interface. Therefore, practically, the moving picture presentation apparatus 100 is configured by incorporating a predetermined program into a computer device.

<<< §3. Role of switching video in the present invention >>>
Here, the role of the switching video presented by the switching video playback instruction unit 143 and the switching video playback unit 132 in the video presentation device 100 shown in FIG. 10 will be described based on an actual example.

  As already described in §2, the position / direction updating unit 141 follows the predetermined route based on the user instruction input by the instruction input unit 150 and the route information RT stored in the basic information storage unit 120. The viewpoint position is updated so that the viewpoint moves, and the line-of-sight direction is updated as necessary. In addition, the moving moving image reproduction instruction unit 142 refers to the moving image data correspondence information MC stored in the basic information storage unit 120 when the viewpoint is moving on the route, and the moving image corresponding to the moving route. The data is recognized as playback target video data, and a moving video playback instruction for the playback target video data is given to the moving video playback unit. This moving moving image reproduction instruction includes information indicating the reproduction target moving image data, information indicating the frame order according to the moving direction of the viewpoint, and information indicating the line-of-sight direction.

  Here, the information indicating the frame order according to the moving direction of the viewpoint is information indicating either the frame order at the time of shooting or the frame order opposite to that at the time of shooting. As illustrated in FIG. 13, the moving image data correspondence information MC stored in the basic information storage unit 120 includes information indicating the shooting direction for each route. Accordingly, when the moving direction of the viewpoint is forward with respect to the shooting direction, the moving video playback instruction unit 142 gives a moving video playback instruction indicating that the playback target video data is played back in the frame order at the time of shooting. When the moving direction of is reverse to the shooting direction, it is possible to give a moving moving image playback instruction indicating that the playback target moving image data is played back in a frame order opposite to that at the time of shooting.

  On the other hand, the information indicating the line-of-sight direction indicates the latest line-of-sight direction updated by the position / direction updating unit 141 and is given as the azimuth angle φ. Of course, when the line-of-sight direction is updated by the user's instruction during the path movement, the value of the azimuth angle φ in the moving moving image reproduction instruction given to the moving moving image reproduction unit 131 is also updated sequentially. In the end, the moving video playback instruction given from the moving video playback instruction unit 142 to the moving video playback unit 131 cuts out specific playback target video data according to a predetermined line-of-sight direction in frame order according to the moving direction of the viewpoint. This is an instruction to show that it is played back.

  In FIG. 10, the symbols “Mj (±)” and “φ” appended to the arrows from the moving video playback instruction unit 142 to the moving video playback unit 131 indicate information included in the moving video playback instruction. It is. Here, the symbol “Mj” is information indicating that the j-th moving image data stored in the moving image data storage unit 110 is reproduction target moving image data, and the symbol “(±)” is a value at the time of shooting. This is information indicating either the frame order “+” or the frame order “−” opposite to that at the time of shooting, and the symbol “φ” is information indicating the line-of-sight direction (azimuth angle). Specifically, in FIG. 11, when moving the route R1 from the node N1 to the node N2 with the line of sight facing forward, a moving video reproduction instruction “M1 (+), φ = 90 °” is issued. Given this, when moving the route R6 from the node N2 toward N5 with the line of sight facing forward, a moving video reproduction instruction of “M6 (+), φ = 180 °” is given. On the other hand, when the route R4 is moving from the node N4 toward the node N5 with the line of sight facing forward, a moving video reproduction instruction “M4 (+), φ = 270 °” is given, and the node N5 to the node N2 When the user is moving the path R6 with the line of sight facing forward, a moving video reproduction instruction of “M6 (−), φ = 0 °” is given.

  Based on such a moving moving image reproduction instruction, the moving moving image reproduction unit 131 performs omnidirectional composing each frame F (1) to F (n) of the reproduction target moving image data Mj stored in the moving image data storage unit 110. The images are read in the order of the designated frames, and the visual field images Q that constitute the field of view in the designated predetermined line-of-sight direction (azimuth angle φ) are cut out from the read omnidirectional images, and the series of visual field images Q thus obtained are animated. To output as.

  It is also possible to give a pause or a pause release instruction as the moving video playback instruction. When a pause instruction is given, the moving video reproduction unit 131 continuously outputs a visual field image Q obtained from the same frame that is currently being read until the pause cancellation instruction is given. Can be done. In addition, when adopting a form in which the moving speed is made variable based on the user's instruction, the moving video playback instruction includes the information indicating the moving speed, and the moving video playback unit 131 responds to the moving speed. It suffices to read out frames at a speed.

  As described above, the “moving moving image” presented by the functions of the moving moving image reproduction instructing unit 142 and the moving moving image reproducing unit 131 is a moving image viewed from a moving viewpoint on a certain route, and thus has different viewpoints. When transitioning to a route, it is necessary to switch from moving movie playback of the old route to moving movie playback of the new route. The “switching video” presented by the functions of the switching video playback instruction unit 143 and the switching video playback unit 132 is a video presented at the time of this switching. When the viewpoint changes from the old route to the new route via the node, the position / direction updating unit 141 temporarily stops the viewpoint on the node, and after the switching video playback unit 132 completes the playback of the switching video, the new path The movement of the viewpoint will be resumed.

  Hereinafter, in the example described in §2, that is, in the visit path shown in FIG. 11, the route R1 is advanced from the node N1 to the node N2 in a state where the line of sight is directed to the front (φ = 90 °). A specific content of the switching video to be presented at the node N2 will be described for an example of traveling along the route R6 in a state where the line of sight is directed from the node N2 to N5 (φ = 180 °).

  In the case of the above example, when the node N2 is reached, a view image QA as shown in FIG. 14 (a) is presented, and when leaving the node N2, the view image QZ as shown in FIG. 14 (b) is displayed. Presented. Here, the visual field image QA is an image cut out from the last frame F (n) of the moving image data M1 corresponding to the old route R1, whereas the visual field image QZ is the moving image data M6 corresponding to the new route R6. Is an image cut out from the first frame F (1).

  FIG. 15 is a plan view showing a panoramic image (omnidirectional image) from which the two visual field images QA and QZ shown in FIG. 14 are cut out. First, FIG. 15A is a panoramic image (omnidirectional image) from which the view image QA is cut out, and is an image constituting the last frame F (n) of the moving image data M1 of the old path R1. Hereinafter, the omnidirectional image constituting the frame is referred to as a pre-switching omnidirectional image Fold. On the other hand, FIG. 15B is a panoramic image (omnidirectional image) from which the view image QZ is cut out, and is an image constituting the first frame F (1) of the moving image data M6 of the new path R6. Hereinafter, the omnidirectional image constituting the frame is referred to as a post-switching omnidirectional image Fnew.

  Note that “first” and “last” mentioned here mean “first” and “last” with respect to “frame order at the time of reproduction” of the moving video. Therefore, when moving movies are played back in the reverse frame order from the time of shooting, the “first” frame at the time of shooting becomes the “last” frame at the time of playback, and the “last” frame at the time of shooting is played back. It becomes the “first” frame of the hour.

  After all, the visual field image QA shown in FIG. 14 (a) corresponds to an image cut out from the omnidirectional image Fold before switching shown in FIG. 15 (a) around the azimuth angle φ = 90 °, which is the sight line direction. The field-of-view image QZ shown in (b) corresponds to an image cut out from the post-switching omnidirectional image Fnew shown in FIG. 15 (b) around the azimuth angle φ = 180 °, which is the line-of-sight direction. Specifically, as illustrated in FIG. 8, the field of view within the range of azimuth angles “φ−Δ / 2” to “φ + Δ / 2” (where Δ is a predetermined cut-off angle) from the read out omnidirectional image. ) Is cut out.

  In the example shown here, distortion correction is performed when a rectangular visual field image is cut out from the panoramic image (omnidirectional image), so that the visual field images QA and QZ (FIGS. 14A and 14B) are displayed. The regular rectangular image having almost no distortion is slightly different from the images in the extraction target area shown in FIGS. 15A and 15B (strictly speaking, it is not a rectangular image). For convenience, the images in the clipping target area on the omnidirectional image Fold before switching and the omnidirectional image Fnew after switching are also referred to as view images QA and QZ. Of course, if the distortion of the image cut out from the panoramic image (omnidirectional image) is such that there is no practical problem, the distortion correction may be omitted and the cut-out image may be presented as a view image as it is. Alternatively, the magnification of the field-of-view image can be made variable by arbitrarily setting the size of the extraction target area as necessary.

  As shown in FIG. 15 (a), the field-of-view image QA is an image obtained by cutting out the field of view with the line of sight directed to the azimuth angle φ = 90 ° from the omnidirectional image Fold before switching, whereas FIG. As shown in FIG. 4, the field-of-view image QZ is an image obtained by cutting out the field of view with the line of sight at an azimuth angle φ = 180 ° from the omnidirectional image Fnew after switching. Accordingly, when the user instructs the instruction input unit 150 to turn right at the node N2 (instruction to select the route R6 as a new route), the process illustrated in FIG. 15A (FIG. 14A) is performed. If the visual field image QA shown is suddenly switched to the visual field image QZ shown in FIG. 15B (FIG. 14B), the viewer feels a sense of discomfort due to the discontinuity of the image. .

  In the present invention, a switching process for alleviating such a sense of incongruity is performed. FIG. 16 is a time chart showing the process of such switching processing. The t-axis in the figure is a time axis indicating the time flowing from left to right. A point t1 on the time axis t indicates a time when the viewpoint arrives at the node N2, and a point t2 indicates a time when the viewpoint leaves the node N2. Indicates. Here, a period between time points t1 and t2 is referred to as a switching period T. On the other hand, the P-axis in the figure is a position axis indicating the position on the visit road, the section marked R1 shows the position on the route R1 (left is the node N1 side), and the section marked R6 is on the route R6. (The right side is the node N5 side), and the section labeled N2 remains at the node N2.

  Eventually, this chart shows that the viewpoint moved from the node N1 arrives at the node N2 at the time t1, stays at the node N2 during the switching period T, leaves the node N2 at the time t2, and goes to the node N5. It shows that.

  The rectangular cells shown in the upper part of the chart indicate individual frames of moving image data read out by the moving moving image reproducing unit 131 at each time point. For example, the frames F (n−2) and F (n−1) immediately before the end of the moving image data M1 are read until the viewpoint reaches the node N2, and when the node N2 is reached at time t1, the last frame is read. F (n) is read out. The moving moving image reproduction unit 131 cuts out and presents a visual field image corresponding to the viewing direction of the azimuth angle φ = 90 ° from each frame (omnidirectional image) thus read out. On the other hand, when the viewpoint departs from the node N2, the frames F (1), F (2), and F (3) of the moving image data M6 are read, and the moving moving image playback unit 131 reads each of the frames thus read out. From the frame (omnidirectional image), the visual field image corresponding to the viewing direction with the azimuth angle φ = 180 ° is cut out and presented.

  As described above, the last frame F (n) of the moving image data M1 in this time chart is a frame constituting the pre-switching omnidirectional image Fold. As shown in FIG. Will be cut out. Similarly, the first frame F (1) of the moving image data M6 is a frame constituting the omnidirectional image Fnew after switching, and the view image QZ is cut out from this frame as shown in FIG. 15 (b). Become. The switching video presented by the switching video playback instruction unit 143 and the switching video playback unit 132 is a video that is presented in order to perform smooth switching from the visual field image QA to the visual field image QZ in the switching period T.

  The main role of the switching video in the present invention is to make the viewer recognize smooth switching of the line-of-sight direction when changing from the old route to the new route. The field-of-view image QA shown in FIG. 15 (a) is an image obtained by cutting out the line-of-sight direction with an azimuth angle φ = 90 °, whereas the field-of-view image QZ shown in FIG. 15 (b) has a line-of-sight with an azimuth angle φ = 180 °. Since this is an image obtained by cutting out the direction, if a sudden switch is made from the visual field image QA to the visual field image QZ, the user's sense of direction will be confused. Therefore, in the present invention, a panning video that gradually changes from the line-of-sight direction of the old route to the line-of-sight direction of the new route is presented as the switching video. In the case of the above example, intermediate visual field images QB, QC, QD, which are cut out by gradually changing the line-of-sight direction from azimuth angle φ = 90 ° to 180 ° with the viewpoint position stationary at node N2. .., And inserting these intermediate visual field images between the visual field images QA and QZ to generate a switching moving image.

<<< §4. Features of switching video in the present invention >>>
As described in §3, in order to generate a switching video that plays a role of allowing the viewer to recognize smooth switching of the line-of-sight direction, several specific methods can be taken. FIG. 17 is a plan view showing the first basic method. Here, FIG. 17A shows the omnidirectional image Fold before switching, and FIG. 17B shows the omnidirectional image Fnew after switching. First, a visual field image QA as shown in FIG. 17A (same as the visual field image QA shown in FIG. 15A) is presented as the first image constituting the switching video. Next, a visual field image QB as shown in FIG. 17B is presented as the second image of the switching video. Here, although the field image QA and the field image QB are images obtained by performing clipping based on the azimuth angle φ = 90 °, the field image QA is an image obtained by clipping the omnidirectional image Fold before switching. However, the view image QB is different from the omnidirectional image Fnew after switching.

  Subsequently, as the third image of the switching video, the visual field image QC obtained by performing clipping based on the azimuth angle φ = 91 ° from the omnidirectional image Fnew after switching is presented, and the switching image is displayed as the fourth image of the switching video. The visual field image QD obtained by cutting out the rear omnidirectional image Fnew based on the azimuth angle φ = 92 ° is presented, and the azimuth angle φ is gradually changed from 90 ° to 180 °. However, the field-of-view images cut out from the omnidirectional image Fnew after switching are sequentially presented. Finally, a visual field image QZ (an image cut out at φ = 180 °) shown in FIG. 15B is presented. In short, the first basic method first switches the visual field image QA shown in FIG. 17 (a) to the visual field image QB shown in FIG. 17 (b), and then the visual field image QB shown in FIG. 17 (b). It can be said that the cutting frame is gradually moved to the right and finally the position of the visual field image QZ shown in FIG.

  On the other hand, FIG. 18 is a plan view showing the second basic method. 18A shows the omnidirectional image Fold before switching, and FIG. 18B shows the omnidirectional image Fnew after switching. In this method, first, a field-of-view image QA shown in FIG. 15A is presented as the first image constituting the switching video. Next, as the second image of the switching video, a view image QB obtained by performing clipping based on the azimuth angle φ = 91 ° from the omnidirectional image Fold before switching shown in FIG. As the third image, a view image QC obtained by performing clipping based on the azimuth angle φ = 92 ° from the pre-switching omnidirectional image Fold is presented, and the azimuth angle φ is changed from 90 ° to 180 °. The visual field images cut out from the pre-switching omnidirectional image Fold are sequentially presented while being gradually changed toward, and the azimuth angle φ = 180 from the pre-switching omnidirectional image Fold as shown in FIG. A visual field image QY obtained by cutting out based on ° is presented.

  Finally, the visual field image QZ shown in FIG. 18B (same as the visual field image QZ shown in FIG. 15B) is presented. Here, the field-of-view image QY and the field-of-view image QZ are common in that they are images obtained by clipping based on the azimuth angle φ = 180 °, but the field-of-view image QY is an image obtained by clipping from the omnidirectional image Fold before switching. However, the difference is that the field-of-view image QZ is an image cut out from the omnidirectional image Fnew after switching. In short, this second basic method gradually moves the cutout frame of the visual field image QA shown in FIG. 15 (a) to the right, and finally the visual field image QY shown in FIG. 18 (a). This can be said to be a method of taking the position and finally switching to the visual field image QZ shown in FIG.

  Both of the two methods described above can generate a switching video with the effect of “smooth switching of the line-of-sight direction”, so it is possible to remove the user's uncomfortable feeling due to discontinuity in the line-of-sight direction. is there. However, in the present invention, neither of these two methods is adopted. This is because these methods cannot remove the user's uncomfortable feeling due to discontinuities such as positional deviation and luminance fluctuation. First, let me explain why.

  In the case of the first method described above, the visual field image QB shown in FIG. 17B is presented following the visual field image QA shown in FIG. Here, the omnidirectional image Fold before switching shown in FIG. 17 (a) and the omnidirectional image Fnew after switching shown in FIG. 17 (b) are omnidirectional images photographed at the position of the same node N2. The images should be exactly the same, and in fact, the same image is also shown in the figure. However, in practice, both images are not the same, and a slight positional shift and luminance shift occur. This is because the moving image data of each route is obtained by a separate shooting operation at the actual location.

  For example, when shooting moving image data for the visit path shown in FIG. 2, shooting for the routes R1 to R6 is normally performed as a separate shooting operation. Of course, the shooting operation is devised so that the route from the node N1 through the node N2 to the node N3 is imaged in one operation, and the moving image data M1 and M2 are created based on the obtained series of imaging data. If this method is adopted, it is possible to completely match the omnidirectional image constituting the last frame of the moving image data M1 and the omnidirectional image constituting the first frame of the moving image data M2.

  However, in practice, it is also necessary to perform preparation work such as removing general passersby from the route to be photographed, so it is difficult to photograph all the routes in one operation. For this reason, in practice, individual video data must be created by dividing it into multiple shooting tasks (theoretically, unless all traces can be traced with a single stroke, it is not possible to shoot all paths in a single operation. Can not).

  Moreover, unless a method such as laying a rail for shooting on the route to be shot is taken (in such a general facility, it is difficult to lay such a rail), the shot was taken at the same node position. Even if it is the intended omnidirectional image, it is inevitable that a subtle displacement occurs. In addition, since the photographing time is different, a difference occurs in the lighting environment (particularly, the difference becomes remarkable when photographing outdoors), and it is inevitable that a luminance shift occurs. In this way, as long as the method of shooting with an omnidirectional camera is used for actual facilities and places, it is inevitable that there will be a difference in the shooting results at the same node, resulting in a geometric displacement. Or the lighting environment will change.

  In the case of the above example, the pre-switching omnidirectional image Fold shown in FIG. 17A is an image obtained at the time of shooting the moving image data M1, and the post-switching omnidirectional image Fnew shown in FIG. Since this is an image obtained at the time of M6 shooting, even if it is an omnidirectional image shot at the same node N2, a positional shift and a luminance shift are caused. Therefore, when the visual field image QB is presented after the visual field image QA as the switching video, the discontinuity of the image such as displacement and luminance fluctuation occurs, which gives the viewer a sense of discomfort.

  Such a situation is the same in the case of the second method described above. That is, the omnidirectional image Fold before switching shown in FIG. 18 (a) and the omnidirectional image Fnew after switching shown in FIG. 18 (b) are misaligned with each other. When the visual field image QZ is presented subsequent to the image QY, discontinuity of the image such as positional deviation and luminance fluctuation occurs, which gives the viewer a sense of discomfort.

  Such a problem is a problem that never occurs when virtual moving image data is created using the CG technique, and should be a problem inherent to shooting an actual facility or place. .

  According to the switching moving image presented in the present invention, it is possible to smoothly switch from moving image data shot on the old route to moving image data shot on the new route. Therefore, in the present invention, a process is performed to generate a blend image obtained by combining a visual field image cut out from the pre-switching omnidirectional image Fold and a visual field image cut out from the post-switching omnidirectional image Fnew at a predetermined blend ratio. A method of outputting a series of blended images as a switching video is adopted. At this time, a common line-of-sight direction is set for the pre-switching omnidirectional image Fold and the post-switching omnidirectional image Fnew, and the visual field images cut out in the same line-of-sight direction are synthesized.

  For example, FIGS. 19 (a) and 19 (b) cut out visual field images QA and QB for an azimuth angle φ = 90 °, which is a common viewing direction, from the omnidirectional image Fold before switching and the omnidirectional image Fnew after switching, It is a top view which shows the process which synthesize | combines both by the ratio of 1: 0 and produces | generates the blend image G (90 degrees). Since the ratio of the image QB is 0, the blend image G (90 °) is substantially the same image as the visual field image QA.

  Similarly, FIGS. 20 (a) and 20 (b) cut out view images QC and QD with respect to a common azimuth angle φ = 120 ° from images Fold and Fnew, and the two images are in a ratio of 2/3: 1/3. FIGS. 21A and 21B are plan views showing a process of generating a blend image G (120 °) by synthesizing, and FIGS. 21A and 21B are views of a field image for a common azimuth angle φ = 150 ° from images Fold and Fnew. It is a top view which shows the process which cuts out QE and QF, synthesize | combines both by the ratio of 1/3: 2/3, and produces | generates the blend image G (150 degrees). 22 (a) and 22 (b) show, from the images Fold and Fnew, the view images QY and QZ with respect to the common azimuth angle φ = 180 ° are cut out and synthesized at a ratio of 0: 1 to blend images. It is a top view which shows the process which produces | generates G (180 degrees). Since the ratio of the image QY is 0, the blend image G (180 °) is substantially the same image as the visual field image QZ.

  Here, for convenience of explanation, the step size of the azimuth angle φ is set to 30 °, and four blend images G (90 °), G (120 °), G (150 °), and G (180 °) are generated. However, in practice, if the step size of the azimuth angle φ is made finer, it is possible to present a switching video that expresses smooth switching of the line-of-sight direction. For example, if the step size of the azimuth angle φ is set to 1 °, the common line-of-sight direction gradually changes in units of 1 ° from the line-of-sight direction φ = 90 ° before switching to the line-of-sight direction φ = 180 ° after switching. Therefore, a switching image composed of blend images G (90 °), G (91 °), G (92 °),..., G (179 °), G (180 °) is presented. become.

  Moreover, in the process of generating the blend image, if the ratio of the field-of-view image cut out from the omnidirectional image Fnew after switching is defined as the blend ratio α, the blend ratio α is gradually increased from 0 to 1. The smooth switching from the moving image data for the old route to the moving image data for the new route is also performed at the same time. This is the basic principle of the switching process in the present invention.

  FIG. 23 is a time chart showing a specific process of such switching processing. Similarly to the chart shown in FIG. 16, the t-axis and the P-axis are the time axis t and the position axis P, and the rectangular grid shown in the upper part of the chart indicates the moving video playback unit 131 or the switching video playback unit 132 at each time point. The individual frames of the moving image data read out by are shown. This chart also shows an example in which the viewpoint moved from the node N1 arrives at the node N2 at the time t1, stays at the node N2 during the switching period T, leaves the node N2 at the time t2, and goes to the node N5. However, the generation process of the switching video presented during the switching period T is shown in more detail.

  In the upper part of the chart, the grid of the frames constituting the moving image data is drawn over two lines. Here, the cells in the first row indicate the frames constituting the moving image data M1 for the old route R1, and the cells in the second row indicate the frames constituting the moving image data M6 for the new route R6. The frames constituting the moving image data M1 and M6 are overlapped in the switching period T, and as described above, the process of generating the blend image by combining the two visual field images is performed.

  That is, when focusing on the switching period T, the cells in the first row are all the last frame F (n) constituting the moving image data M1 for the old route R1, that is, the omnidirectional image Fold before switching. The eye meshes are the first frame F (1) constituting the moving image data M6 for the new route R6, that is, the omnidirectional image Fnew after switching. In addition, the value of the blend ratio α is described at the upper part of these meshes, and the value of the common line-of-sight direction (azimuth angle φ) is described at the lower part of these meshes. Symbols G (90 °), G (91 °),...

  In the example shown here, the step size of the azimuth angle φ is set to 1 °, and during the switching period T, the azimuth angle φ gradually increases in units of 1 ° from 90 ° to 180 °. , A total of 91 blend images G (90 °), G (91 °), G (92 °),..., G (179 °), G (180 °) constitute a switching image. . Moreover, since the blend ratio α is gradually increased from 0 to 1, the ratio of the component of the image Fnew to the component of the image Fold in the blend image G is gradually increased.

Expression shown in the lower part of FIG. 23 G = (1−α) · Q (Fold (φ)) + α · Q (Fnew (φ))
Q (Fold (φ)) is the pixel value of the field image in the direction indicated by the azimuth angle φ cut out from the omnidirectional image Fold before switching, and the direction indicated by the azimuth angle φ cut out from the omnidirectional image Fnew after switching. This is an equation for determining the pixel value G of the blend image when the pixel value of the field image is Q (Fnew (φ)) and the blend ratio is α (0 ≦ α ≦ 1). For the pixel value of the pixel at the corresponding position of each field-of-view image, if the calculation for obtaining the pixel value is performed by applying the above formula (in the case of a color image, the calculation is performed for each primary color), the blend image G Can be obtained.

  In this way, the process of generating the blend image G is performed while gradually changing the azimuth angle φ from φold to φnew (in the above example, changing from 90 ° to 180 ° in 1 ° increments), and When the blend ratio α is gradually changed from 0 to 1, it is repeatedly executed, and a series of the generated blend images G (90 °) to G (180 °) is output as a video, so that a switching video can be presented. become.

  After all, the position / direction updating unit 141 in the video presentation device 100 illustrated in FIG. 10 refers to the reference direction of the new route when the viewpoint changes from the old route to the new route via the node, so that the line-of-sight direction after switching is changed. φnew is determined, and the azimuth angle φ indicating the line-of-sight direction is gradually updated from the line-of-sight direction φold before switching to the line-of-sight direction φnew after switching. On the other hand, the switching video playback instruction unit 143 refers to the video data correspondence information MC, and switches from the old path video to the new path video with a change from the gaze direction φold before switching to the gaze direction φnew after switching. A process of giving a switching video playback instruction to the switching video playback unit 132 so as to be performed is performed.

  In FIG. 10, the symbols “Fold”, “Fnew”, and “φold → φnew” added to the arrows from the switching video playback instruction unit 143 to the switching video playback unit 132 are information included in the switching video playback instruction. Is shown. That is, the symbols “Fold” and “Fnew” indicate information for specifying the pre-switching omnidirectional image Fold and the post-switching omnidirectional image Fnew. Specifically, in the above example, the information specifying the pre-switching omnidirectional image Fold is information indicating “the last frame F (n) of the moving image data R1”, and specifies the post-switching omnidirectional image Fnew. The information is information indicating “first frame F (1) of moving image data R6”. The symbol “φold → φnew” is given to the switching video reproduction unit 132 by the value of the azimuth angle φ that gradually changes from the line-of-sight direction φold before switching to the line-of-sight direction φnew after switching. Is shown.

  Based on such a switching video playback instruction, the switching video playback unit 132 reads the pre-switching omnidirectional image Fold and the post-switching omnidirectional image Fnew from the video data stored in the video data storage unit 110, A series of blended images are generated by the procedure as described above, and a process of outputting them as a switching video is performed. Here, the frame that is read out as the pre-switching omnidirectional image Fold is the last frame in the frame order when moving moving image reproduction of the moving image data of the old path (in the above example, the last frame of the moving image data M1 of the old path R1). Frame F (n)), and the frame read out as the omnidirectional image Fnew after switching is the first frame (in the above example, the new path in the frame order during moving moving image playback of the moving image data of the new path). This is the first frame F (1) of the moving image data M6 of R6.

  As described above, the blended images are generated by cutting out the field images constituting the field of view of the common line-of-sight direction (common azimuth angle φ) from the read out omnidirectional images Fold and Fnew, respectively. Is synthesized at a predetermined blend ratio α. In addition, such blend image generation processing is performed by gradually changing the common line-of-sight direction (azimuth angle φ) from the line-of-sight direction φold before switching to the line-of-sight direction φnew after switching, and from the omnidirectional image Fnew after switching. A series of blended images that are repeatedly executed and output as a moving image are output while changing the ratio of the cut-out visual field image to gradually increase.

  In the end, the present invention is based on the viewpoint of moving from the first route to the second route via the node for a place having the first route and the second route connected via a certain node. If it sees as a moving image presentation method which presents the field of view of the predetermined gaze direction seen as a moving image, the moving image presentation method which concerns on this invention will be comprised by the following steps.

  (1) Based on the first omnidirectional video imaged along the first path, the computer shows the field of view in the first line-of-sight direction φold viewed from the viewpoint toward the node along the first path. A first moving moving image reproduction stage for reproducing the first moving moving image;

  (2) A switching video playback stage in which the computer plays back a switching video showing the field of view obtained when the line-of-sight direction is gradually changed while the viewpoint is stationary at the node.

  (3) Based on the second omnidirectional video imaged along the second path, the computer shows the field of view in the second line-of-sight direction φnew viewed from the viewpoint away from the node along the second path. A second moving moving image reproduction stage for reproducing the second moving moving image.

  Here, in the switching moving image reproduction stage, the line-of-sight direction is gradually changed from the first line-of-sight direction φold to the second line-of-sight direction φnew for the pre-switching omnidirectional image Fold constituting the last frame of the first omnidirectional moving image. For the first panning video obtained by changing and the switched omnidirectional image Fnew constituting the first frame of the second omnidirectional video, the gaze direction is changed from the first gaze direction φold to the second gaze direction φnew. A process of reproducing a moving image obtained by blending the second panning video obtained by gradually changing the second panning video at a predetermined ratio such that the ratio of the second panning video gradually increases as a switching video. Done.

  As described above, according to the present invention, when the viewpoint transitions from the old route to the new route via the node, the pan video gradually changing from the sight line direction of the old route to the sight direction of the new route is switched video. In addition, an image obtained by blending the moving image of the old route and the moving image of the new route is presented while gradually changing the blend ratio. Therefore, even if there is a positional shift or luminance fluctuation between the old route video and the new route video, the video can be presented in a manner that does not make the viewer feel uncomfortable due to image discontinuity. become able to.

<<< §5. Modification of the present invention >>
So far, the basic embodiment of the video presentation device according to the present invention has been described with reference to the block diagram shown in FIG. However, the present invention is not limited to this basic embodiment, and can be implemented in various other modes. Therefore, here, some modifications of the moving picture presentation apparatus according to the present invention will be described.

<5-1: Modification Example Regarding Setting of Gaze Direction>
In the embodiments described so far, the description has been made on the assumption that the virtual user always moves on the route with the line of sight always facing the front (movement direction). For example, in the visit path shown in FIG. 11, when the viewpoint is moving along the route R1 from the node N1 toward the node N2, the viewing direction is the direction of the azimuth φ = 90 °, and the viewpoint is from the node N2 to the node N5. An example in which the line-of-sight direction when moving along the route R6 is the direction of the azimuth angle φ = 180 ° is described, and the switching image presented at the node N2 is changed from the line-of-sight direction φold = 90 ° before switching after switching. The example in which the panning image gradually changing to the viewing direction φnew = 180 ° has been described.

  FIG. 24 is a plan view showing the transition of the line-of-sight direction due to the path transition when the line-of-sight direction is set to match the movement direction as described above. Here, an example of transition from the route R1 to the route R2 via the node N is shown. In this case, the moving video M1 is presented while moving along the route R1, the switching video Mt is presented when paused at the node N, and the moving video M2 is presented while moving along the route R2. The broken-line arrows in the figure indicate the reference direction of each route (that is, the viewpoint movement direction). For example, Dold indicates a reference direction when the route R1 moves toward the node N, and Dnew indicates a reference direction when the route R2 moves away from the node N.

  In the setting in which the line-of-sight direction always coincides with the movement direction, the line-of-sight direction moving along the route R1 coincides with the reference direction Dold, and the line-of-sight direction moving along the route R2 coincides with the reference direction Dnew. Therefore, the moving video M1 may be constituted by a series of visual field images cut out with the reference direction Dold as the line-of-sight direction, and the moving video M2 may be constituted by a series of visual field images cut out with the reference direction Dnew as the line-of-sight direction. That's fine. In short, the position / direction updating unit 141 illustrated in FIG. 10 may always update the azimuth angle D corresponding to the reference direction of the moving route to the azimuth angle φ indicating the line-of-sight direction.

  In the node N, the switching video Mt is presented. The position / direction updating unit 141 illustrated in FIG. 10 sets the azimuth angle Dold corresponding to the reference direction of the old route R1 to the azimuth angle indicating the line-of-sight direction before switching. Update processing for gradually changing the azimuth angle φ from φold to φnew may be performed by setting φold as the azimuth angle Dnew corresponding to the reference direction of the new route R2 as the azimuth angle φnew indicating the line-of-sight direction after switching. Then, at the node N, a pan video that gradually changes from the field of view at the azimuth angle Dold to the field of view at the azimuth angle Dnew is presented as a switching video.

  In this way, if the setting is made so that the line-of-sight direction always matches the movement direction, the state in which the virtual user always moves toward the front is presented as a video, so it can be used sufficiently for general purposes It is. However, in order to provide the user with a more flexible viewing experience, it is preferable that the line-of-sight direction can be freely set according to the user's intention. In particular, in the case of a device that provides a tour experience of a museum visit or a historic site visit as described above, the user will want to pay attention to a specific direction of interest.

  The moving picture presentation apparatus shown in FIG. 10 has a function of arbitrarily setting the line-of-sight direction in accordance with a user instruction, as described in §2. That is, the position / direction updating unit 141 holds information indicating the current line-of-sight direction as data of the azimuth angle φ, and the data of the azimuth angle φ is updated when the switching video is presented at the node. In response to a user instruction, the line of sight is also updated when the sight line direction is set to an arbitrary direction. In §2, an example has been described in which the user freely changes the line-of-sight direction by pressing the left button B3 or the right button B4 of the controller 50 shown in FIG.

  Here, the angle formed by the moving direction and the line-of-sight direction on the two-dimensional plane in which the route is defined is referred to as an offset angle θ (−180 ° <θ ≦ + 180 °). If the instruction input unit 150 has a function of inputting an instruction to set the offset angle θ, the user can arbitrarily set the line-of-sight direction.

  For example, in the case of the controller 50 shown in FIG. 9, the left button B3 may function as a button that gives an instruction to decrease the offset angle θ, and the right button B4 may function as a button that gives an instruction to increase the offset angle θ (θ is from −180 °). If it becomes smaller, 360 ° should be added, and if θ becomes larger than + 180 °, 360 ° should be reduced). The position / direction updating unit 141 holds the value of the offset angle θ and performs a process of sequentially updating the offset angle θ in accordance with an instruction from the instruction input unit 150 (controller 50).

  In this case, the azimuth angle φ indicating the line-of-sight direction is given by the equation φ = D + θ, where D is the azimuth angle indicating the reference direction of the route being moved. Based on the azimuth angle D indicating the reference direction of the route and the offset angle θ, the azimuth angle φ indicating the line-of-sight direction can be obtained. Eventually, the moving video playback instruction unit 142 gives the moving video playback unit 131 a moving video playback command including information on the azimuth angle φ set based on the formula φ = D + θ. The example shown in FIG. 24 can be said to be an example in which the offset angle θ is always fixed at 0 °.

  FIG. 25 is a plan view showing changes in the line-of-sight direction due to path transition when the line-of-sight direction is arbitrarily set. The example shown in FIG. 25 is an example of transition from the route R1 to the route R2 via the node N as in the example shown in FIG. 24, but the line-of-sight direction does not coincide with the movement direction. Here, let us consider a case where the user sets a desired offset angle θ (θ = + 30 ° in the illustrated example) while moving along the route R1. In this case, the line-of-sight direction is an oblique right direction with respect to the movement direction, which corresponds to a state in which the virtual user is moving forward on the route R1 with the face slightly toward the right.

  Now, as shown in the drawing, when considering the line-of-sight direction when the viewpoint is at the predetermined position P1 on the old route R1, the azimuth angle φold indicating the line-of-sight direction is defined as Dold corresponding to the reference direction of the old route R1. φold = Dold + θ. Therefore, the moving video reproduction unit 131 cuts out the visual field image in the line-of-sight direction indicated by the azimuth angle φold while the viewpoint is moving along the old route R1, and is directed slightly to the node N toward the node N. A moving video M1 showing the field of view is presented.

  Here, it is assumed that the virtual user (viewpoint) reaches the node N while keeping the offset angle θ constant. In the node N, the switching video Mt is presented. At this time, the position / direction updating unit 141 needs to determine the azimuth angle φold indicating the line-of-sight direction before switching and the azimuth angle φnew indicating the line-of-sight direction after switching. is there. Since the azimuth angle φold indicating the line-of-sight direction before switching indicates the line-of-sight direction at the time of arrival at the node N, in the illustrated example, φold = Dold + θ may be set. On the other hand, several determination methods can be employed for the azimuth angle φnew indicating the line-of-sight direction after switching. Here, two types of determination methods will be exemplified.

  The first determination method is a method based on the basic policy of maintaining the offset angle θ set in the old route as it is even after the transition to the new route. FIG. 25 shows the azimuth angle φnew determined by the first determination method. That is, an offset angle θ (the same as the offset angle θ set in the old route R1) is added to the position P2 on the new route R2 shown in the figure with respect to the azimuth angle Dnew indicating the reference direction of the new route R2. An example is shown in which the azimuth angle φnew determined thereby is a new line-of-sight direction.

  When this first determination method is adopted, the orientation of the face of the virtual user moving on the old route is maintained even on the new route. That is, in the case of the example shown in FIG. 25, the virtual user travels toward the node N while facing his face in the right direction on the old route R1, and the moving video M1 has a line of sight indicated by φold = Dold + θ. A field of view is presented. On the other hand, also in the new route R2, the virtual user leaves the node N with his / her face facing diagonally to the right, and the moving image M2 is presented with a field of view in the line of sight indicated by φnew = Dnew + θ. Therefore, the switching video Mt presented at the node N becomes a pan video in which the line-of-sight direction gradually changes from the azimuth angle φold thus determined to φnew.

  FIG. 26 (a) is a diagram showing an equation for determining the line-of-sight direction before and after switching via the node N when the first determination method is adopted. According to this equation, the azimuth angle φold indicating the line-of-sight direction before switching is determined based on the equation φold = Dold + θ based on the azimuth angle Dold corresponding to the reference direction of the old route R1, and the line-of-sight after switching The azimuth angle φnew indicating the direction is determined based on the formula φnew = Dnew + θ based on the azimuth angle Dnew corresponding to the reference direction of the new route R2. That is, the position / direction updating unit 141 determines φold and φnew based on these equations, respectively, and the switching video reproduction instruction unit 143 performs switching in which the azimuth angle φ gradually changes from φold to φnew. A moving image reproduction instruction is given to the switching moving image reproduction unit 132.

  On the other hand, the second determination method is a method based on the basic policy of resetting the offset angle θ set in the old route to 0 when transitioning to the new route. FIG. 26B is a diagram showing an equation for determining the line-of-sight direction before and after switching via the node N when the second determination method is adopted. According to this equation, the azimuth angle φold indicating the line-of-sight direction before switching is φold = Dold + θ based on the azimuth angle Dold corresponding to the reference direction of the old route R1, as in the first determination method described above. Determined based on the formula. However, the azimuth angle φnew indicating the line-of-sight direction after switching is determined based on the formula φnew = Dnew based only on the azimuth angle Dnew corresponding to the reference direction of the new route R2. That is, the position / direction updating unit 141 determines φold and φnew based on these equations, respectively, and the switching video reproduction instruction unit 143 performs switching in which the azimuth angle φ gradually changes from φold to φnew. A moving image reproduction instruction is given to the switching moving image reproduction unit 132.

  When this second determination method is adopted, the direction of the face of the virtual user moving along the old route is in any direction, but the direction of the face in the new route is always facing the front (that is, the moving direction). become. In FIG. 25, if the virtual user proceeds along the old route R1 with his / her face facing diagonally to the right, the moving image M1 is presented with a field of view in the line of sight indicated by φold = Dold + θ. Then, the virtual user will proceed facing the front, and as the moving moving image M2, a field of view in the line-of-sight direction indicated by φnew = Dnew will be presented. The switching video Mt presented at the node N becomes a pan video in which the line-of-sight direction gradually changes from the azimuth angle φold thus determined to φnew.

  If the first determination method is adopted, even when the path changes, the offset angle θ is always maintained, so that the face orientation with respect to the traveling direction of the virtual user is always constant. Therefore, it is convenient when the virtual user takes a visiting method in which the user visually appreciates the painting along the route while always looking at the exhibits on the right wall in the viewing path of the museum shown in FIG. On the other hand, if the second determination method is adopted, in the initial state of entering a new route, a front-facing image is always presented, so that the user can easily understand the spatial positional relationship of the route. This is convenient when presenting a scene of a virtual city or historic site.

  As described above, the first determination method and the second determination method relating to the line-of-sight direction have their own characteristics, taking into consideration the location to be presented, the nature of the facility, the purpose of presenting the video to the user, and the like. An optimal determination method may be adopted as appropriate. Of course, in carrying out the present invention, the method of determining the line-of-sight direction at the time of path transition is not limited to the above-described two determination methods, and various other determination methods can be adopted. .

  Note that if a user instruction is given to increase / decrease the offset angle θ during playback of the switching video, φnew may be recalculated by changing θ according to the instruction. May not accept an instruction from the user to change the offset angle θ.

<5-2: Modified example of panning direction of switching video>
As described above, the switching video is presented as a pan video with a change from the sight line direction before switching (azimuth angle φold) to the sighting direction after switching (azimuth angle φnew). Here, the azimuth angle φold The direction of rotation from to the azimuth angle φnew may be any direction. For example, in the examples shown in FIG. 24 and FIG. 25, a switching video Mt consisting of a pan video with the line-of-sight direction rotated from the azimuth angle φold to the azimuth angle φnew centering on the node N is presented. The rotation direction of the line of sight may be clockwise or counterclockwise. However, in general, it is preferable to select a rotation direction in which the rotation angle is smaller, and in the example shown in FIGS. 24 and 25, it is preferable to rotate the rotation direction clockwise.

  In the example shown in FIG. 23, in order to change from the line-of-sight direction φ before switching to 90 ° to the line-of-sight direction after switching φ = 180 °, φ = 91 °, 92 °, 93 °,. The rotation direction is changed because the rotation direction with a smaller rotation angle span is selected. Of course, when a special effect is produced, the rotation direction with a larger rotation angle span is selected. It doesn't matter if you do. In this case, the line-of-sight direction in the switching video changes counterclockwise as φ = 90 °, 89 °, 88 °,..., 0 °, 359 °, 358 °,. It will be.

  Further, when it is necessary as a production effect, first, it is possible to make one rotation over 360 ° and then further rotate it so as to face the line-of-sight direction after switching. In short, a panning image in which a field of view in the direction of the sight line before switching (azimuth angle φold) is presented as the first image, and a field of view in the direction of the line of sight after switching (azimuth angle φnew) is presented as the last image. Is presented as a switching video, it does not matter how the line-of-sight direction is moved during that time.

  Furthermore, in the embodiments described so far, individual paths are defined on the same two-dimensional plane, but it is also possible to define each path on a three-dimensional space. Then, it is possible to define a route with a difference in elevation, such as a slope or a staircase. When the route is defined in the three-dimensional space, the reference direction of the route is also defined as a vector in the three-dimensional space, so the gaze direction before switching and the gaze direction after switching are the same as the direction of the three-dimensional vector. The pan video presented as the switching moving image is a video showing a change in the line of sight in a three-dimensional space combining not only so-called horizontal pan but also vertical pan (tilt). FIG. 8 shows a cut-out example in which the vertical width of the visual field image Q (P (i), φ (i)) is matched with the vertical width of the panoramic image F (i). If the vertical width is reduced, the position of the visual field image Q is defined by the azimuth angle φ and the elevation angle Ψ. Therefore, if the elevation angle Ψ is changed, moving videos and switching videos with the effect of tilt are reproduced. It is also possible to do.

<5-3: Modification using curved path>
In the embodiments described so far, an example in which each route is a straight line has been described. However, a route existing at an actual place is not necessarily a straight route. For example, a visit path of a museum having a circular corridor is constituted by an arc-shaped route. Thus, when the actual path is a curved path, the shooting path by the omnidirectional camera must be the curved path, and the obtained moving image data is also the shooting data along the curved path. become.

  In this case, as the route information RT, for example, information that defines a route composed of a curve using a Bezier curve or the like may be prepared. Alternatively, it is also possible to define a plurality of nodes on a real curved path and a straight path connecting these nodes and approximate the curved path by a broken line.

  In any case, if a route is defined by two nodes that are both end points, the reference direction of the route is the direction of a straight line connecting the two nodes located at both ends of the route. Can be defined. For example, as shown in FIG. 27, when a curved path R is defined between two nodes N1 and N2 by a Bezier curve or the like, the moving direction of the viewpoint moving along the curved path R is sequentially determined. Although it changes, if the vector direction D from the node N1 to N2 is adopted as the reference direction of the route R as shown by the broken line in the figure, there is a practical problem in implementing the present invention. Does not occur.

  In this way, even when the actual route is curved, the reference direction of the route can be defined as a straight line connecting both end points. There is no problem in determining the viewing direction (azimuth angle φold) and the viewing direction after switching (azimuth angle φnew).

<5-4: Modification of controller>
The controller 50 shown in FIG. 9 is an example of a device that constitutes a part of the instruction input unit 150. In practice, various other types of devices can be used as the controller. is there. In particular, the form and arrangement of the operation buttons and the like can be freely set by design. In §2, the role of the buttons B6 to B9 is not described, but various functions such as a function for setting a moving speed and a function for setting a display magnification can be assigned to these buttons. it can.

  A joystick can be used instead of the button. For example, if one joystick is tilted forward, a forward instruction is given; if it is tilted backward, a backward instruction is given; if it is tilted leftward, a leftward instruction is given; Can be used. Further, it is possible to change the moving speed based on the tilt angle of the stick.

<5-5: Modification in which instruction input unit is omitted>
The moving image presentation apparatus 100 shown in FIG. 10 includes an instruction input unit 150 for inputting an instruction from the user. However, the instruction input unit 150 is not necessarily a necessary component for carrying out the present invention. Depending on the usage mode, the instruction input unit 150 may be omitted.

  In other words, if the position / direction updating unit 141 has a function of automatically updating the viewpoint position and the line-of-sight direction based on a predetermined rule set in advance, a moving image can be presented without a user's instruction input. For example, a square tour route constituted by routes R2, R3, R4, and R6 is set for the visit route shown in FIG. 2, and the position / direction updating unit 141 automatically sets the viewpoint position along the tour route. If the line-of-sight direction is updated, a moving image showing the field of view along the patrol route can be presented as a so-called demo video.

  In this case, a moving moving image based on the moving image data M2 is presented while moving on the route R2, a switching moving image for moving to the new route R3 is presented when reaching the node N3, and moving image data is moved on the new route R3. A moving video based on M3 is presented, a switching video for transitioning to the new route R4 is presented when node N4 is reached, and so on. become.

10: Fisheye lens 20: Video camera 30: Data processing unit 40: Cart 50: Controller 100: Movie presentation device 110: Movie data storage unit 120: Basic information storage unit 130: Movie playback unit 131: Moving movie playback unit 132: Switching movie Playback unit 140: Playback instruction unit 141: Position / direction update unit 142: Moving video playback instruction unit 143: Switched video playback instruction unit 150: Instruction input unit 200: Display device A: Node B serving as a path endpoint Nodes B1 to B9: Buttons D, D1, D2: Path reference direction Dnew: Azimuth angle Dold corresponding to the reference direction of the new path Dold: Azimuth angle E (i) corresponding to the reference direction of the old path: eye vector F ( 1), F (2), F (i-1), F (i), F (i + 1), F (n-1), F (n): individual frames constituting moving image data ( Directional image)
Fnew: Omnidirectional image after switching (first frame of new path video data)
Fold: Omnidirectional image before switching (last frame of moving image data of old route)
G: Pixel value G (φ) of blend image: Blend image for azimuth angle φ: Frame number j: Movie data number M, M1 to M6: Movie data / moving movie MC: Movie data correspondence information Mt: Switching movie mk1 ˜mk3: Marker N, N1 to N6: Node n: Number of frames included in moving image data O: Top point of apex / two-dimensional coordinate system P1, P2: Position P (1), P (2) on the path , P (i-1), P (i), P (i + 1), P (n-1), P (n): Points on the route Q, Q (P (i), φ (i)): Field of view Images QA to QZ: Field image Q (Fnew (φ)): Pixel value Q (Fold (φ)) of the field image in the direction indicated by the azimuth angle φ cut out from the omnidirectional image Fnew after switching: Omnidirectional image before switching Pixel values R, R1 to R6 of the visual field image in the direction indicated by the azimuth angle φ cut out from Fold: path RT Route information T: Switching period t: Time t1, t2: Specific time point X: Coordinate axis Y of the two-dimensional coordinate system Y: Coordinate axis α of the two-dimensional coordinate system α: Blend ratio Δ: Cutting angle φ, φ1, φ2, φ (i ): Gaze direction (azimuth)
φnew: Azimuth angle indicating line-of-sight direction after switching φold: Azimuth angle indicating line-of-sight direction before switching θ: Offset angle

Claims (17)

A video presentation device that presents a field of view in a predetermined line-of-sight direction viewed from a viewpoint moving along a predetermined path for a real place including a plurality of paths connected via nodes,
Moving image data obtained by shooting using an omnidirectional camera that moves a route along a predetermined shooting direction, and configured by arranging omnidirectional images having a 360 ° field of view in units of frames. A video data storage for storing data for each route;
A basic information storage unit for storing connection information of each route through the node and route information indicating a reference direction of each route, and moving image data correspondence information indicating moving image data corresponding to each route;
A position / direction update unit that updates the viewpoint position and line-of-sight direction, a moving video playback instruction unit that instructs playback of a moving video viewed from the moving viewpoint when the viewpoint is moving on the route by the update, and a viewpoint A switching video playback instructing unit for instructing the node to play a switching video viewed from a stationary viewpoint when a transition is made from the old route to the new route via the node;
A moving video reproduction unit that reproduces a moving video based on an instruction given from the moving video reproduction instruction unit; a switching video reproduction unit that reproduces a switching video based on an instruction given from the switching video reproduction instruction unit; A video playback unit having
With
The position / direction update unit updates the viewpoint position so that the viewpoint moves along a predetermined route based on the route information, updates the line-of-sight direction as necessary, and the viewpoint passes through the node. When the transition from the old route to the new route is made, the viewpoint is temporarily stopped on the node, and after the switching video playback unit completes the playback of the switching video, the movement of the viewpoint is resumed on the new route,
The moving moving image reproduction instructing unit refers to the moving image data correspondence information, recognizes the moving image data corresponding to the moving path as reproduction target moving image data, and sets the reproduction target moving image data as a frame corresponding to the moving direction of the viewpoint. In order, the moving video reproduction unit is given a moving video reproduction instruction indicating that the position direction update unit performs cutout according to a predetermined line-of-sight direction and reproduces,
The moving moving image reproduction unit reads out and reads out the omnidirectional images constituting each frame of the reproduction target moving image data stored in the moving image data storage unit in the designated frame order based on the moving moving image reproduction instruction. From the omnidirectional image, cut out the visual field image that constitutes the field of view in the specified predetermined line-of-sight direction, and output the resulting series of visual field images as a movie,
The position / direction update unit determines the line-of-sight direction after switching by referring to the reference direction of the new path when the viewpoint changes from the old path to the new path via the node, and after switching from the line-of-sight direction before switching The process of gradually updating the gaze direction to the gaze direction of
The switching video playback instruction unit refers to the video data correspondence information, and performs switching with a change from the sight line direction before the switching to the sight line direction after the switching from the old path video to the new path video. Is given to the switching video playback unit,
Based on the switching video playback instruction, the switching video playback unit sets the last frame in the frame order at the time of moving video playback of the video data of the old path stored in the video data storage unit as an omnidirectional image before switching. Read, read the first frame in the frame order at the time of moving moving image playback of moving image data of the new path stored in the moving image data storage unit as a omnidirectional image after switching, and from the two omnidirectional images read out, the common gaze direction Processing for generating a blended image obtained by synthesizing the field-of-view images constituting each field of view and synthesizing the two field-of-view images extracted at a predetermined blend ratio, and changing the common line-of-sight direction from the line-of-sight direction before switching to the line-of-sight after switching The ratio of the field of view image cut out from the omnidirectional image after switching is gradually increased while gradually changing to the direction. While reduction, and repeatedly executed, video presentation device characterized by playing a series of blends images generated as a switching video. The video presentation device according to claim 1,
The elevation angle is below a predetermined reference value from a distorted circular image obtained by photographing a hemispherical field located above a predetermined horizontal plane using an omnidirectional camera equipped with a fisheye lens or an omnidirectional mirror. A moving picture presenting apparatus comprising: moving picture data configured by arranging rectangular panoramic images obtained by cutting out a region and arranging a rectangular panoramic image obtained by performing distortion correction on the area. In the moving image presentation apparatus according to claim 1 or 2,
The route information stored in the basic information storage unit includes position information indicating coordinate values on the two-dimensional coordinate system of nodes located at both ends of each route, and the reference direction of each route is the both ends of the route. A moving picture presenting apparatus defined by a direction of a straight line connecting two nodes located in the area. The video presentation device according to any one of claims 1 to 3,
The video data correspondence information stored in the basic information storage unit includes information indicating the shooting direction for each route,
The moving video playback instruction unit gives a moving video playback instruction indicating that video data is played back in the frame order at the time of shooting when the moving direction of the viewpoint is forward with respect to the shooting direction. A moving image presentation apparatus that gives a moving moving image reproduction instruction indicating that moving image data is reproduced in a frame order opposite to that at the time of shooting when the direction is reverse to the direction. In the moving image presentation device according to any one of claims 1 to 4,
The basic information storage unit determines whether there is a path connecting the position information indicating the coordinate value of each node on the two-dimensional coordinate system and any two nodes in all the nodes. The moving picture presentation apparatus characterized by storing the route information comprised by the connection information shown. In the moving image presentation device according to any one of claims 1 to 5,
The basic information storage unit stores route information indicating the route defined on the two-dimensional plane,
The moving image presentation apparatus, wherein the position / direction updating unit uses an azimuth angle φ (0 ° ≦ φ <360 °) with respect to a predetermined reference axis on the two-dimensional plane as a parameter indicating a line-of-sight direction. The video presentation device according to claim 6,
The position / direction update unit updates the azimuth angle D corresponding to the reference direction of the moving route as the azimuth angle φ indicating the line-of-sight direction,
The switching video reproduction instruction unit sets the azimuth angle Dold corresponding to the reference direction of the old route as the azimuth angle φold indicating the visual line direction before switching, and the azimuth angle Dnew corresponding to the reference direction of the new route indicates the visual line direction after switching. A moving picture presentation apparatus characterized by giving a switching moving picture reproduction instruction with an azimuth angle φnew. In the moving image presentation device according to any one of claims 1 to 5,
An instruction input unit for inputting a user instruction;
A moving image presentation apparatus, wherein the position / direction updating unit has a function of updating a viewpoint position and a line-of-sight direction in accordance with a user instruction to the instruction input unit. The video presentation device according to claim 8,
The basic information storage unit stores route information indicating the route defined on the two-dimensional plane,
The position direction update unit uses an azimuth angle φ (0 ° ≦ φ <360 °) with respect to a predetermined reference axis on the two-dimensional plane as a parameter indicating the line-of-sight direction,
The instruction input unit has a function of inputting an instruction to set an offset angle θ in the line-of-sight direction with respect to the moving direction on the two-dimensional plane,
The moving image presentation device, wherein the position / direction updating unit determines an azimuth angle φ indicating a line-of-sight direction based on an equation φ = D + θ based on an azimuth angle D corresponding to a reference direction of a moving route. The video presentation device according to claim 9,
Based on the azimuth angle Dold corresponding to the reference direction of the old route, the position / direction updating unit determines the azimuth angle φold indicating the line-of-sight direction before switching based on the formula φold = Dold + θ, and the reference direction of the new route An azimuth angle φnew indicating a line-of-sight direction after switching is determined based on an expression of φnew = Dnew + θ based on an azimuth angle Dnew corresponding to. The video presentation device according to claim 9,
Based on the azimuth angle Dold corresponding to the reference direction of the old route, the position / direction updating unit determines the azimuth angle φold indicating the line-of-sight direction before switching based on the formula φold = Dold + θ, and the reference direction of the new route An azimuth angle φnew indicating a line-of-sight direction after switching is determined based on an equation φnew = Dnew based on the azimuth angle Dnew corresponding to. In the moving image presentation apparatus in any one of Claims 8-11,
The position / direction update unit has a function of giving a marker display instruction to the moving video reproduction unit to display a marker indicating each path connected to the node when the viewpoint reaches the node.
Based on the marker display instruction, the moving video playback unit performs a process of superimposing individual markers on the visual field image when reaching the node,
The instruction input unit has a function of inputting a user instruction to select a specific marker,
A moving image presentation device in which a position / direction updating unit recognizes a route corresponding to a selected marker as a new route. In the moving image presentation apparatus in any one of Claim 6, 7, 9-11,
Q (Fold (φ)) is the pixel value of the visual field image in the direction indicated by the azimuth angle φ cut out from the omnidirectional image before switching, and the pixel of the visual field image in the direction indicated by the azimuth angle φ cut out from the omnidirectional image after switching. When the value is Q (Fnew (φ)),
G = (1−α) · Q (Fold (φ)) + α · Q (Fnew (φ))
Based on the following formula, the process of generating the blend image by defining the pixel value G of the blend image is performed by gradually changing the azimuth angle φ from φold to φnew and gradually increasing the blend ratio α from 0 to 1. A moving image presenting apparatus that repeatedly executes while changing, and outputs a series of generated blend images as a moving image. In the moving image presentation device according to any one of claims 6, 7, 9 to 11, and 13,
The moving video playback unit and the switching video playback unit display the field of view within the range of azimuth angles “φ−Δ / 2” to “φ + Δ / 2” (where Δ is a predetermined cut-off angle) from the read omnidirectional images. A moving image presenting apparatus characterized by cutting out a field-of-view image. In the moving image presentation device according to any one of claims 1 to 7,
A moving image presentation apparatus, wherein the position / direction updating unit automatically updates the viewpoint position and the line-of-sight direction based on a predetermined rule set in advance.   A program that causes a computer to function as the moving picture presentation device according to claim 1. For a place having a first route and a second route connected via a node, a predetermined line-of-sight direction viewed from a viewpoint of moving from the first route to the second route via the node A video presentation method for presenting a field of view as a video,
Based on the first omnidirectional video imaged along the first path, the computer shows a first line-of-sight view viewed from a viewpoint toward the node along the first path. A first moving video playback stage for playing back one moving video;
A switching video playback stage for playing back a switching video showing a field of view obtained when the computer gradually changes the line-of-sight direction in a state where the viewpoint is stationary at the node;
Based on the second omnidirectional video imaged along the second route, the computer shows a second line-of-sight field of view viewed from a viewpoint away from the node along the second route. A second moving video playback stage for playing back two moving videos;
Have
In the switching moving image reproduction step, the line-of-sight direction is gradually changed from the first line-of-sight direction to the second line-of-sight direction for the pre-switching omnidirectional image constituting the last frame of the first omnidirectional moving image. With respect to the first pan moving image obtained by this and the switched omnidirectional image constituting the first frame of the second omnidirectional moving image, the line-of-sight direction is changed from the first line-of-sight direction to the second line-of-sight direction. A video obtained by blending a second panning video obtained by gradually changing the second panning video at a predetermined ratio such that the ratio of the second panning video gradually increases is reproduced as a switching video. A video presentation method.

Images