JP2015069445A - Video output device, video output method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video output device, etc., capable of automatically outputting a video viewed in a discretionary visual line direction from a view point moving on a previously designed route in a pre-shot space and further capable of non-continuously moving the position of the view point to a designated position when a user designates a prescribed position in the space.SOLUTION: A video output device 1 calculates coordinate values of a point P upon acceptance of designation of the point P on a floor plan 20. The video output device 1 calculates coordinate values of a point R constituting a search candidate on a route, and calculates Euclidean distances d of all points R and the point P. The video output device 1 specifies one point R in which the Euclidean distance d is at minimum, and stores a branch to which the specified point R belongs as a branch of concern. The video output device 1 also converts the coordinate values of the specified point R into a progress ratio within the branch of concern. The video output device 1 cuts out a corresponding visual-field image from an omnidirectional image 90 and displays a user screen 61.

Description

  The present invention relates to a video output device or the like for outputting a video corresponding to a line of sight viewed from the viewpoint of a user moving in space.

  An omnidirectional image having a 360 ° field of view can be taken using a camera equipped with a fisheye lens or an omnidirectional mirror. In recent years, a panoramic image providing service has been developed that can provide a panoramic image according to the field of view of a user that moves freely in an actual shooting space created by omnidirectional images taken in advance while facing an arbitrary line-of-sight direction.

  As an example of using the panoramic image providing service, there is an online map search service, and Non-Patent Document 1 is an example. Non-Patent Document 1 is a site that provides a service that allows a user to view a panoramic image taken at a point when the user clicks on a point on the map. In addition, by clicking an arrow displayed on the screen, the user can move the viewpoint to the front or near side in the panoramic photo.

  Further, in Non-Patent Document 2, it is possible to view an image in which the user is moving while freely changing the line-of-sight direction by joining the captured panoramic images at the time of reproduction.

“GoogleStreet View”, [online], Google, Inc, [September 11, 2013 search], Internet <http://maps.google.co.jp/intl/en/help/maps/streetview/> “Motion VR of the Month 2006 calendar” using Quick Time VR, [online], AppleComputer, Inc, [searched on September 11, 2013], Internet <http://www.worldinmotionvr.com/ motionvr_month / calendar.html>

  However, in Non-Patent Document 1 described above, the displayed images are a plurality of still images viewed from a discrete viewpoint, and a moving image that matches the movement is output to a user moving in the space. Can not.

  In Non-Patent Document 2 described above, the displayed image is a moving image, but bidirectional movement is not possible. Further, the range in which the user can move is only on a preset route, and there is no branch on the route.

  Furthermore, when an arbitrary position on the floor map, which is a plan view (overhead view) of the live-action space, is specified, there is a request to change (jump) the viewpoint position of the output video to a location specified by the user. .

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide an image that can be seen in an arbitrary line-of-sight direction from a viewpoint position that moves on a route designed in advance in a space that has been previously photographed. And a video output device that can move the viewpoint position to the designated position when the user designates an arbitrary position on the floor map that is a plan view of the space. is there.

In order to solve the above-described problem, a first invention is a video output device that outputs a video in a line-of-sight direction at a viewpoint position that moves in a path arranged in a coordinate space, and includes a frame of an omnidirectional image. A route storage unit that stores a sequence in association with the route, a floor map display unit that displays the route as a floor map, accepts the movement of the viewpoint position and the line-of-sight direction, and the position on the floor map. An input means for receiving an input; and a viewpoint position moving means for moving the viewpoint position to a position in the route in the coordinate space corresponding to the position when an input of the position on the floor map is received; Video generation means for cutting out and generating a video of a visual field range in the viewpoint position and the line-of-sight direction from the omnidirectional image, and the generated video Is a video output device, characterized by comprising: a video output means for outputting an image, the.
According to the video output device of the first aspect of the present invention, it is possible to automatically output a video that can be seen in an arbitrary line of sight from a viewpoint position (position information in the space) that freely moves on the route in a space previously captured. it can. When an arbitrary position on the floor map representing the route is designated by the user, the viewpoint output can be moved to the designated position and video output can be resumed.

In addition, it is preferable that the path includes a node and a directed branch, and a sequence including the omnidirectional image frame is associated with each directed branch.
The method of associating a directed branch with a sequence of frames of omnidirectional images reduces the data construction workload and the amount of data compared to the method of associating all positional information on a route with omnidirectional images. There is a merit. Furthermore, since the user moves on the directed branch, bidirectional movement is easily possible.
Here, a node is a branch connection, and a branch is an edge of a directed graph that represents a path connecting a starting node and an end node.

The floor map is set with a two-dimensional coordinate system with a predetermined position as an origin, and the viewpoint position moving means converts the position on the floor map into a position in the coordinate space, and It is desirable to move the viewpoint position to a position in the route that is closest to the position.
Thereby, the viewpoint position can be moved to a position in the route close to the position on the floor map designated by the user.

Further, the viewpoint position moving means calculates a distance between the converted position and each candidate point set in the branch that constitutes the route, and the viewpoint position is calculated to the candidate point with the smallest calculated distance. It is desirable to move.
Thereby, since the distance between the converted position in the coordinate space and the candidate point set in the branch can be accurately calculated, the viewpoint position can be moved to an appropriate position.

The candidate points are preferably break points that are generated when the branch is equally divided into a predetermined number.
Thereby, a candidate point can be set simply.

According to a second aspect of the present invention, there is provided route storage means for storing a sequence composed of frames of omnidirectional images for outputting a video in a sight line direction at a viewpoint position moving in a route arranged in a coordinate space in association with the route. A floor map display step for displaying the route as a floor map, and an input for receiving an input of a position on the floor map and receiving a movement of the viewpoint position and the line-of-sight direction A viewpoint position moving step of moving the viewpoint position to a position in the path in the coordinate space corresponding to the position when the position input on the floor map is accepted, and the viewpoint position and A video generation step of generating a video of a visual field range in the line-of-sight direction by cutting out from the omnidirectional image; and A video output step of outputting the images that form a video output method, which comprises a.
According to the video output method of the second aspect of the invention, it is possible to automatically output a video that can be seen in an arbitrary line of sight from a viewpoint position (position information in the space) that freely moves on the route in a space previously captured. it can. When an arbitrary position on the floor map representing the route is designated by the user, the viewpoint output can be moved to the designated position and video output can be resumed.

A third invention is a program for causing a computer to function as an image output device that outputs an image in a line-of-sight direction at a viewpoint position that moves in a route arranged in a coordinate space, and the computer is configured to be omnidirectional. A route storage unit that stores a sequence of image frames in association with the route, a floor map display unit that displays the route as a floor map, accepts movement of the viewpoint position and the line-of-sight direction, and on the floor map An input means for receiving an input of the position of the eye, and a viewpoint position moving means for moving the viewpoint position to a position in the path in the coordinate space corresponding to the position when the input of the position on the floor map is received. , Generating a video of the visual field range in the viewpoint position and the line-of-sight direction by cutting out from the omnidirectional image That the image generation means is a program for generating the image output means for outputting the image function as a.
By installing the program according to the third invention in a general-purpose computer, the video output device according to the first invention can be obtained and the video output method according to the second invention can be executed.

  According to the present invention, in a pre-captured space, an image that can be seen in an arbitrary line-of-sight direction from a viewpoint position that moves on a route designed in advance is output. It is possible to provide a video output device or the like that can move a viewpoint position to a designated position when an arbitrary position is designated.

FIG. 2 is a block diagram showing a hardware configuration example of a video output apparatus according to the present embodiment. Plan view showing an example of a facility building where the viewpoint position moves The figure which shows an example of the data structure which the memory | storage part of a video output device hold | maintains The figure which shows an example of the viewpoint position on the path | route represented on XY two-dimensional coordinate The figure which shows the node information and branch information which memorize | store the path | route shown in FIG. Flowchart showing the flow of video output processing according to the present embodiment The figure which shows an example of the data which a video output apparatus hold | maintains temporarily by video output processing The figure which shows an example of a user input screen The figure explaining the visual field range corresponding to a gaze direction The figure explaining the operation | work which cuts out the image corresponding to a gaze direction from the omnidirectional image corresponding to a viewpoint position Flow chart showing the flow of jump processing Flow chart showing the flow of blend image display processing The figure explaining the movement of a gaze direction

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a hardware configuration diagram of a computer that implements a video output apparatus 1 according to an embodiment of the present invention. As shown in FIG. 1, the computer includes a control unit 11, a storage unit 12, a media input / output unit 13, a communication control unit 14, an input unit 15, a display unit 16, a peripheral device I / F unit 17, and the like. 18 is connected.

The control unit 11 is a CPU (Central
A processing unit (ROM), a read only memory (ROM), a random access memory (RAM), and the like.
The CPU calls and executes a program stored in the storage unit 12, ROM, storage medium, or the like to the work memory area on the RAM, and drives and controls each device connected via the bus 18. The process to be described later is realized. The ROM is a non-volatile memory and permanently holds a computer boot program, a program such as BIOS, data, and the like. The RAM is a volatile memory, and temporarily holds a loaded program, data, and the like, and includes a work area used by the control unit 11 to perform each process.

The storage unit 12 is an HDD (Hard
Disk drive), the program executed by the control unit 11, data necessary for program execution, OS (Operating)
System) etc. are stored. These program codes are read by the control unit 11 as necessary, transferred to the RAM, and read and executed by the CPU.

The media input / output unit 13 is a media input / output device such as a CD drive, a DVD drive, an MO drive, and a floppy (registered trademark) disk drive, and inputs / outputs data such as images.
The communication control unit 14 includes a communication control device, a communication port, and the like, and is a communication interface that mediates communication between a computer and a network, and performs communication control between other devices via the network.

The input unit 15 performs data input and includes, for example, a controller having a lever or button that moves up and down, left and right, a keyboard, a pointing device such as a mouse, and an input device such as a numeric keypad. The input data is output to the control unit 11.
The display unit 16 includes, for example, a display device such as a CRT monitor or a liquid crystal panel, and a logic circuit (a video adapter or the like) for executing display processing in cooperation with the display device, and is input under the control of the control unit 11. The displayed display information is displayed on the display device.
The input unit 15 and the display unit 16 may be, for example, a display with a touch panel in which those functions are integrated.

The peripheral device I / F unit (interface) 17 is a port for connecting a peripheral device to the computer, and the computer transmits and receives data to and from the peripheral device via the peripheral device I / F unit 17. The peripheral device I / F unit 17 is configured by USB, IEEE 1394, RS-232C, or the like, and usually includes a plurality of peripheral devices I / F. The connection form with the peripheral device may be wired or wireless.
The bus 18 is a path that mediates transmission / reception of control signals, data signals, and the like between the devices.

  FIG. 2 is a plan view showing an example of a facility hall where the viewpoint position moves according to the present embodiment. The video output device 1 according to the present embodiment is a device having a function of displaying an image corresponding to an arbitrary line-of-sight direction viewed from a viewpoint position that continuously moves along a predetermined route. The route along which the viewpoint position moves according to the present embodiment is, for example, in a space such as a company floor or a facility such as a museum. It may be outdoors as long as it can capture a video on the route. In the present embodiment, an example will be described in which a facility in the facility having the floor plan 20 shown in FIG. 2 is presented to the user. The floor map in the present invention is a floor plan 20.

  The video output device 1 cuts out a visual field image based on a viewpoint position and a line-of-sight direction input by a user from a frame image that constitutes a video imaged in advance along a route, and outputs it as a moving image. By presenting the video in the hall while freely moving the viewpoint position and the line-of-sight direction, the user can have an experience of walking around and seeing.

  In addition, the video output device 1 presents the floor plan 20 to the user and designates a point on the floor plan 20 by the user (for example, a click operation) in the middle of the operation of outputting the video, thereby specifying the specified point. It is provided with a function to discontinuously move (jump) the viewpoint position to a point closest to.

  As shown in FIG. 2, the route in the floor plan 20 is composed of a node 21 and a branch 22. The node 21 is a path joint, and the branch 22 is a unit path that connects a node 21 (hereinafter referred to as a start node 21a) as a starting point and a node 21 (hereinafter referred to as an end point node 21b) as an end point. It is. In the example shown in FIG. 2, seven nodes N1 to N7 and six branches B1 to B6 are defined.

  As shown in the drawing, each branch 22 is represented by a straight line, and the direction of the arrow of each branch 22 indicates that the starting point of the arrow is the starting node 21a and the ending point of the arrow is the ending node 21b. That is, when the start node 21a and the end node 21b are specified, the branch 22 is specified. The direction of the branch 22 coincides with the traveling direction when the inside of the facility is photographed along the route when the omnidirectional image stored in the storage unit 12 is generated.

  It is up to the designer of the device to define the position of the node 21 on the route, but in general, the node 21 is defined at an end point (dead end), a corner, or a branch point on the route. To do. In the example shown in FIG. 2, N3, N4, N5, and N7 are nodes 21 that indicate end points on the route, N6 is a node 21 that indicates a corner on the route, and N1 and N2 indicate branch points on the route. Node 21.

  FIG. 3 is a diagram illustrating an example of a data structure stored in the storage unit 12 of the video output device 1. The storage unit 12 of the video output device 1 stores (a) node information 25, (b) branch information 26, (c) video information 27, (d) various parameters 28, and the cut out image as video. An application (not shown) for output is stored in advance.

  The node information 25 shown in (a) is information related to each of the plurality of nodes 21 existing on the path where the viewpoint position can move, and “node ID”, “position information X”, “position information Y”, “ Stores attribute.

  “Position information X” and “Position information Y” are position information of the node 21 and store “X coordinate value” and “Y coordinate value” when the node 21 is represented on XY two-dimensional coordinates, respectively. . The XY two-dimensional coordinates are coordinates (coordinate system) in which the video output apparatus 1 manages the nodes displayed on the floor plan 20. The distance between the nodes 21 can be calculated as an actual size value by converting the coordinates into distance units.

  The “attribute” is an order that represents the feature of the node 21 on the route, and is a value that indicates the number of branches 22 derived from each node 21. For example, “1” is stored if the node 21 is at an end point (dead end) on the route, “2” is stored if the node 21 is at a corner on the route, and “3” is stored if the node 21 is on a three-way on the route. 3 ”is stored, and“ 4 ”is stored if the node 21 is on a crossroad on the route.

  The branch information 26 shown in (b) is information relating to each of the plurality of branches 22 connecting the start node 21a and the end node 21b, and includes “branch ID”, “start node”, “end node”, and “video ID”. Is stored.

  The “starting node” stores the node ID on the shooting start side among the end points of the branch 22, and the “end point node” stores the node ID on the shooting end side among the end points of the branch 22. The direction from the start node 21a to the end node 21b is defined as the direction of the branch 22. In “Video ID”, the video ID corresponding to the branch 22 is stored.

  The video information 27 shown in (c) is information relating to an omnidirectional image taken on each branch 22, and stores “video ID”, “image data”, and “number of frames”.

  The “image data” stores an omnidirectional video sequence. The omnidirectional image sequence is a group of image data composed of a plurality of continuous frames in which an omnidirectional image constituting an omnidirectional video is one frame. An omnidirectional video is a video taken using an omnidirectional camera while moving at a constant speed along the branch 22. Since shooting is performed while moving at a constant speed, each omnidirectional image frame is associated with each point defined on the branch 22. The “number of frames” stores the value of the number of frames constituting the omnidirectional image sequence.

  The various parameter information 28 shown in (d) stores “cut-out angle” and the like used to cut out a panoramic image in the visual field range corresponding to the user's line-of-sight direction from the omnidirectional image. Details will be described later.

  FIG. 4 is a diagram illustrating an example of the viewpoint position 52 on the route represented on the XY two-dimensional coordinates. A pair of coordinates indicated by parentheses in the symbols of the nodes N1 to N4 are the X coordinate value and the Y coordinate value of the node. As shown in the figure, the direction of the branch B1 coincides with the direction of the X axis, the angle formed by the branch B2 with respect to the X axis is 60 ° counterclockwise, and the angle formed by the branch B3 with respect to the X axis is It is 60 ° clockwise.

  Here, the branch angle is an angle formed by the X axis of the illustrated XY coordinate system and the direction of the branch 22, and a counterclockwise direction is a positive direction. The branch angle can be calculated from the position coordinates of the start point node 21a and the position coordinates of the end point node 22b of the branch 22. In the example shown in FIG. 4, the branch angle of the branch B1 is 0 °, the branch angle of the branch B2 is 60 °, and the branch angle of the branch B3 is 300 ° (−60 °).

FIG. 4 shows an example of the viewpoint position 52 and the line-of-sight direction 54 on the branch B1. The line-of-sight angle θ _L 55 is an angle (horizontal angle) formed by the X-axis and the line-of-sight direction 54, and a counterclockwise direction is a positive direction. In this case, the display unit 16 of the video output device 1 displays a user input screen 61 (see FIG. 8) in which an image in a visual field range that is visible from the viewpoint position 52 in the line-of-sight direction 54 is arranged.

FIG. 5 is a diagram showing node information 25a and branch information 26a for storing the route shown in FIG.
As shown in FIG. 5A, x1 is stored in the position coordinate X of the node N1, y1 is stored in the position coordinate Y, and 3 which is the number of branches 22 derived from the node N1 is stored in the attribute. The X2 is stored in the position coordinate X of the node N2, y1 is stored in the position coordinate Y, and 3 which is the number of branches 22 derived from the node N2 is stored in the attribute. X3 is stored in the position coordinate X of the node N3, y3 is stored in the position coordinate Y, and 1 which is the number of branches 22 derived from the node N3 is stored in the attribute. X4 is stored in the position coordinate X of the node N4, y4 is stored in the position coordinate Y, and 1 which is the number of branches 22 derived from the node N4 is stored in the attribute.

  As shown in FIG. 5B, the video ID corresponding to the branch B1 is stored in the video ID of the branch B1, N1 is stored in the start node, and N2 is stored in the end node. The video ID corresponding to the branch B2 is stored in the video ID of the branch B2, N2 is stored in the start node, and N3 is stored in the end node. A video ID corresponding to the branch B3 is stored in the video ID of the branch B3, N2 is stored in the start node, and N4 is stored in the end node.

(Video output processing)
Subsequently, a video output process according to the present embodiment will be described with reference to FIGS.
The video output process shown in FIG. 6 is a process of receiving an input of continuous movement in the viewpoint position 52 and the line-of-sight direction 54 and outputting an image of the user's visual field range. The jump process (FIG. 6) executed in step S103. 11) is a characteristic process of the present invention.

  FIG. 7 is a diagram illustrating an example of retained data 60 that is temporarily retained by the image output apparatus 1 in the image output process. The retained data 60 may be stored in the RAM of the control unit 11 instead of being stored in the storage unit 12 of the video output device 1.

The “branch ID of the target branch” is the branch ID of the branch 22 where the viewpoint position 52 is located (hereinafter referred to as the target branch 22a). The branch ID of the target branch is updated only when the viewpoint position 52 moves to the next branch 22. “Progress rate P” is a value that identifies the relative position of the viewpoint position 52 within the target branch 22a.
That is, the viewpoint position 52 is specified by the “branch ID of the target branch” and the “progress rate P” of the target branch 22a.

The “line-of-sight angle θ _L ” is an angle (horizontal angle) that the line-of-sight direction 54 makes with the reference axis (X axis), and a counterclockwise direction is a positive direction. The holding data 60 may further store “line-of-sight height”. The “line-of-sight height” is an angle (vertical angle) between the line-of-sight direction 54 and the reference plane (XY plane).
That is, the line-of-sight direction 54 is specified by “line-of-sight angle θ _L ” and “line-of-sight height” if necessary.
“Progress rate P” and “line-of-sight angle θ _L ” (and “line-of-sight height”) are values that are updated each time the video output process is executed.

The “branch ID of the destination branch”, “degree of change in line-of-sight angle r”, “degree of change in blend ratio r ₁ ”, and “blend ratio α” are blend image display processing executed in step S110 (see FIG. 12). It is used in. The “gaze angle change degree r” is a fixed value stored in advance, for example, “1 °”. The “branch ID of the destination branch” and the “degree of blend ratio change r ₁ ” are stored each time the blend image display process is executed. “Blend ratio α” is a value that is updated each time Steps S305 to S311 of the blend image display process are executed. Details will be described later.

FIG. 6 is a flowchart showing the flow of the video output process according to the present embodiment.
The control unit 11 of the video output device 1 displays the user input screen 61 on the display unit 16 (step S101).

  FIG. 8 is a diagram for explaining an example of the user input screen 61. On the user input screen 61, a visual field image area 62, a route display area 63, and an input reception area 74 are displayed. In the visual field image area 62, the image displayed in step S112 or in steps S207 and S310 is arranged.

  In the route display area 63, a floor plan 20 in the facility, a route in which the viewpoint position 52 is movable, and a line-of-sight direction indicator 54e representing the current viewpoint position 52 and the line-of-sight direction 54 are displayed. In the illustrated example, a double circle is drawn as the viewpoint position 52, and a sector is drawn as the line-of-sight direction indicator 54e.

  The user can specify an arbitrary position of the floor plan 20 displayed in the route display area 63 by clicking, for example.

  Based on the visual field image displayed in the visual field image region 62, the viewpoint position 52 on the route displayed in the route display region 63, and the line-of-sight direction indicator 54e, the user instructs to move the viewpoint position 52 from the input reception region 74. And an instruction regarding movement in the line-of-sight direction 54 are input.

  In the input reception area 74, when the viewpoint position 52 is moved, when the line-of-sight direction 54 is moved, various change operations (for example, changing the moving speed, the viewing angle of the visual field image displayed in the visual field image area 62). When a change or the like is performed, a symbol printed on a key top of a keyboard (not shown) pressed by the user is displayed. The functions of various buttons displayed in the input reception area 74 are assigned to any key of a keyboard (not shown).

  Here, the forward button 75a and the backward button 75b are used for an operation for inputting the movement of the viewpoint position 52, and while the forward button 75a is pressed, the viewpoint position 52 moves in a predetermined traveling direction on the route at a predetermined speed. The progress rate P is increased or decreased so as to move at Further, while the backward button 75b is being pressed, the progress rate P is increased or decreased so that the viewpoint position 52 moves at a predetermined speed in a predetermined traveling direction on the route. Here, the predetermined speed is, for example, a speed suitable for the speed at which the user walks. The predetermined speed is stored in the storage unit 12 in advance, and can be changed according to a user instruction.

  As another method, the forward button 75a and the backward button 75b are used only to reverse the traveling direction on the path along which the viewpoint position 52 moves, and the viewpoint position 52 always moves at a predetermined speed in a predetermined traveling direction. It may be set to. In this case, when the stop button 75c is pressed, the movement of the viewpoint position 52 is stopped.

On the other hand, the right-pointing button 76b and the left-pointing button 76d are used for inputting the movement of the line-of-sight direction 54, and the line-of-sight direction 54 moves rightward with respect to the traveling direction at a predetermined speed while the right-pointing button 76b is pressed. reducing the viewing angle theta _L to. Further, while the left button 76d is pressed, the line-of-sight direction 54 increases the viewing angle theta _L to be described later to move to the left with respect to the traveling direction at a predetermined speed. The upward button 76a and the downward button 76c are pressed when changing the line-of-sight height.

  For example, functions such as changing the moving speed and changing the viewing angle of the visual field image displayed in the visual field image area 62 are assigned to the change operation button 77. When making the change, a key on a keyboard (not shown) corresponding to the change operation button 77 is pressed by the user.

  Returning to the description of FIG. The control unit 11 of the video output device 1 determines whether or not an input of a point on the floor plan 20 (hereinafter referred to as a point P) is received via the route display area 63 of the user input screen 61 (step S102). ). When the input of the point P is accepted (YES in step S102), the control unit 11 of the video output device 1 executes a jump process described later (step S103), and proceeds to step S104.

When the input of the point P is not received (NO in step S102), the control unit 11 of the video output device 1 moves the viewpoint position 52 and the line-of-sight direction 54 via the input reception area 74 of the user input screen 61. The movement input is accepted (step S104). Subsequently, the control unit 11 of the video output device 1 calculates the line of sight angle theta _L (step S105).

Video output device 1 updates the line-of-sight angle theta _L by calculating the moving distance of the line of sight direction 54 received in Step S104, in the storage unit 12. Specifically, right button 76 b (see FIG. 8) is pressed by the user, when the rightward movement of the viewing direction 54 is instructed, the video output apparatus 1 reduces a predetermined amount gaze angle theta _L , left button 76d (see FIG. 8) is pressed, when the leftward movement of the viewing direction 54 is instructed, the video output device 1 is increased a predetermined amount viewing angle theta _L, viewing angle theta _L Update and memorize.

Subsequently, the control unit 11 of the image output device 1, based on the line-of-sight angle theta _L calculated in step S105, identifies the traveling direction of the viewpoint position 52 (step S106). It is assumed that the “branch ID of the target branch 22a” is stored in advance in the retained data 60.

As a method for identifying the traveling direction, for example, the video output device 1 searches the branch information 26 and the node information 25 for the position information of the start node 21a and the end node 21b of the target branch 22a, and the branch angle θ of the target branch 22a. _b is calculated. If the line-of-sight angle θ _L is within the range of θ _b −90 ° <θ _L ≦ θ _b + 90 °, the traveling direction matches the direction of the target branch 22a. Matches backwards.

  The control unit 11 of the video output device 1 calculates the progress rate P of the viewpoint position 52 on the target branch 22a (step S107).

  The control unit 11 of the video output device 1 calculates the movement amount of the viewpoint position 52 received in step S104, and calculates the existing progress rate if the traveling direction of the viewpoint position 52 is the positive direction of the target branch 22a. The storage unit 12 stores the sum of the amount of movement and the amount of movement as a new progress rate. Further, if the traveling direction of the viewpoint position 52 is the negative direction of the target branch 22a, the value obtained by subtracting the calculated movement amount from the existing traveling rate is stored in the storage unit 12 as a new traveling rate.

  When the calculated progress rate is less than “0”, the progress rate is “0”, and when the calculated progress rate exceeds “1”, the progress rate is “1”. "

  The control unit 11 of the video output device 1 determines whether or not the viewpoint position 52 is located at the end point of the target branch 22a (step S108). Specifically, it is determined whether or not the progress rate stored in the retained data 60 is “0” or “1”.

  If it is not at the end point (NO in step S108), the control unit 11 of the video output apparatus 1 searches the branch information 26 for the video ID of the target branch 22a, and the number of frames of the video ID searched from the video information 27. And the target frame is specified by multiplying the number of frames by the progress rate P. A field-of-view image is generated by cutting out an image corresponding to the line-of-sight direction 54 and the line-of-sight height from the omnidirectional image of the target frame (step S111).

  FIG. 9 is a diagram for explaining the visual field range with respect to the line-of-sight direction 54. The video output device 1 sets the viewing angle “line-of-sight angle (line-of-sight height) −Δ / 2” to “line-of-sight angle” from the omnidirectional image 90 of the specified target frame, where Δ is the cut-out angle stored in the various parameters 28. An image in the visual field range of “line-of-sight angle (line-of-sight height) + Δ / 2” is cut out and generated. The user is presented with an image with a viewing angle 54e as shown in the drawing centered on the viewing direction 54 from the viewpoint position 52.

FIG. 10 is a diagram illustrating an operation of cutting out an image 91 corresponding to the line-of-sight direction from the omnidirectional image 90 of the specified target frame. In the illustrated example, in viewing angle theta _L is 90 °, delta represents an image 91 of 90 °.

  Returning to the description of FIG. If it is at the end point (YES in step S108), the control unit 11 of the video output device 1 determines whether the end point is a branch point or a corner (step S109). Specifically, the video output device 1 searches the node information 25 for the attribute of the starting node 21a of the target branch 22a when the progress rate is “0”, and when the progress rate is “1”. Retrieves the attribute of the end node 21b of the target branch 22a, and determines whether or not the retrieved attribute is “2” or more.

  If it is at a branch point or a turning point (YES in step S109), the control unit 11 of the video output device 1 executes a blend image display process described later (step S110), and returns to step S102. If it is not at the branch point or the turning point (NO in step S109), the control unit 11 of the video output device 1 proceeds to step S111.

  The control unit 11 of the video output device 1 arranges the generated visual field image in the visual field image area 62 of the user input screen 61 and outputs it to the display unit 15 (step S112), and ends the process.

The video output device 1, based on the progress rate P and viewing angle theta _L of the target branch 22a, arranging the viewpoint position 52 and the line-of-sight direction indicator 54e in the path of the path display area 63.

  The video output device 1 performs step S102 at a short time interval of, for example, 0.01 to 0.5 seconds so that the user can recognize the image 91 corresponding to the line-of-sight direction arranged in the visual field image region 52 as a video. Step S112 is repeated to update the image arranged in the visual field image area 52.

  As described above, the video output process can automatically output a video that can be seen in an arbitrary line-of-sight direction 54 from the viewpoint position 52 that freely moves on a route in the space.

(Jump processing)
Next, the jump process executed in step S103 (FIG. 6) of the video output process will be described with reference to FIG. In the jump process, a floor plan 20 in which a path where the viewpoint position 52 is movable is presented to the user, and when the position (point P) on the floor plan 20 is designated by the user, the path (branch 22 or node) 21) A process of specifying a point closest to the point P from among the above points, updating the current viewpoint position 52 to the specified point, and presenting an image seen from the new viewpoint position 52 to the user.

  The control unit 11 of the video output device 1 converts the coordinates in the path display area 63 of the point P received in step S102 into coordinates in XY two-dimensional coordinates that the video output device 1 manages nodes and the like inside. (Step S201).

  For example, in the user input screen 61 of FIG. 8, the route display area 63 is managed in a coordinate system in which the upper left corner is the origin, the X axis is positive in the right direction, and the Y axis is positive in the downward direction. And In step S102, when the user inputs (clicks) a point P in the floor plan 20 in the floor plan display area 63, the control unit 11 of the video output device 1 sets the coordinates of the point P in the coordinate system in the path display area 63. get. Then, the control unit 11 of the video output device 1 converts the coordinates of the point P into the coordinates of the XY two-dimensional coordinate system from the ratio of the coordinate system and the XY two-dimensional coordinate system in the route display area 63.

  The control unit 11 of the video output device 1 calculates the coordinate value of a point that is a search candidate (hereinafter referred to as a point R) (step S202). The point R is a point on the route (branch 22 or node 21).

  Specifically, the video output apparatus 1 calculates the coordinate values of 99 points excluding both ends of the branch 22 at the break points generated when the branches 22 stored in the branch information 26 are divided into, for example, 100 equal parts. . If the total number of branches 22 stored in the branch information 26 is n and the total number of nodes stored in the node information 25 is m, the total number of points R is 99n + m (pieces). Each coordinate value on the branch 22 can be calculated from the position coordinate of the start node 21a and the position coordinate of the end node 21b of each branch 22.

Note that the branch point setting method for each branch 22 in the video output apparatus 1 may be divided into equal parts, or may be arbitrarily divided instead of equally divided.
Further, the video output device 1 calculates the coordinate value of the point to be a search candidate in advance and stores it in the storage unit 12 or the RAM, and the control unit 11 of the video output device 1 may use this. Good.

  The control unit 11 of the video output device 1 calculates the Euclidean distance d between the point P and the point R (step S203). The Euclidean distance between the point P and the point R is calculated by Equation (1).

  The control unit 11 of the video output device 1 determines whether or not the Euclidean distance d has been calculated for all the points R calculated in S202 (step S204). If not calculated (NO in step S204), the control unit 11 of the video output apparatus 1 returns to step S203.

  When the calculation has been completed (YES in step S204), the control unit 11 of the video output device 1 identifies the point R that minimizes the Euclidean distance d from all the search candidate points R, and identifies the identified point. The branch 22 to which R belongs is set as the target branch 22a, and the branch ID is stored in the retained data 60. Further, the coordinate value of the specified point R is converted into the progress rate P in the branch, and the converted value is stored in the holding data 60 (step S205). Here, the line-of-sight angle (horizontal angle) stored in the holding data 60 may be updated to an initial value (for example, 0) or the like.

  The control unit 11 of the video output device 1 identifies the target frame based on the branch ID of the target branch 22a and the progress rate P, and cuts out an image corresponding to the line-of-sight direction 54 and the line-of-sight height from the omnidirectional image of the target frame. To generate a visual field image (step S206).

  The control unit 11 of the video output device 1 arranges the generated visual field image in the visual field image region 62 of the user input screen 61 and outputs it to the display unit 15 (step S207), and ends the process.

  As described above, by executing the jump process, the video output apparatus 1 can start the route from the designated position when the user designates a desired position on the floor plan area 20 in the floor plan display area 63. It is possible to resume the output of the video seen from the viewpoint position that continuously moves up. Further, the point designated by the user may be on the floor plan 20, and the point on the route (branch 22 or node 21) may not be clicked accurately.

(Blend image display processing)
Next, the blend image display process executed in step S110 of the video output process will be described with reference to FIGS. The blend image display process is a process executed when it is determined in step S110 of the video output process that the viewpoint position 52 is located at a branch point or a corner. The blend image display process is a process for automatically and continuously displaying images that are visible at a branch point or a corner when the viewpoint position 52 transitions from the target branch 22a to the destination branch.

  The control unit 11 of the video output device 1 identifies the destination branch (step S301), and stores the branch ID of the destination branch in the storage unit 12 (holding data 60). When the viewpoint position 52 is located at a corner, the movement destination branch that transitions from the target branch 22a is specified as one. As a method of specifying the movement destination branch when the viewpoint position 52 is located at the branch point, for example, an input from the user is accepted by a movement destination route selection button (not shown) on the user input screen 61.

Control unit 11 of the video output device 1 calculates a branch angle theta _E of the destination branch identified in step S301 (step S302). A method for calculating a branch angle theta _E are as described above with reference to FIG. Next, the control unit 11 of the video output device 1 specifies the moving direction of the line-of-sight direction 54 (step S303).

  FIG. 13 is a diagram for explaining the movement of the viewpoint position 31. When the viewpoint position 52 is located at the end node 21b of the target branch 22a and the line-of-sight direction 54 coincides with the direction of the target branch 22a, the line-of-sight direction 54 is a white arrow until reaching the direction of the destination branch 22c. Move gradually in the direction of. In the example shown in FIG. 13, the movement direction of the line-of-sight direction 54 is counterclockwise.

The control unit 11 of the video output apparatus 1 calculates the degree of change r ₁ of the blend ratio (step S304) and stores it in the storage unit 12 (holding data 60). As a method of calculating the blend ratio change degree r ₁ , for example, the video output apparatus 1 determines the change in the line-of-sight angle r of the retained data 60 as the difference between the branch angle θ _E and the line-of-sight angle θ _L calculated in step S302. The value obtained by dividing by is set as the blend ratio change degree r ₁ .

In the example shown in FIG. 13, the blend ratio change degree r ₁ is calculated as shown in the figure.

The control unit 11 of the video output device 1 stores “0” in the blend ratio α and stores it in the storage unit 12 (holding data 60) (step S305). Next, the control unit 11 of the video output device 1 updates the blend ratio α stored in the storage unit 12 to α + r ₁ and updates the line-of-sight angle θ _L to θ _L + r (step S306). Moving direction of the viewing direction 54 identified in step S303 is in the case of clockwise, and updates the viewing angle theta _L to theta _L -r.

The control unit 11 of the video output device 1 searches the branch information 26 for the video ID of the target branch 22a, and identifies the target frame corresponding to the node 21 where the viewpoint position 52 is located. From the omnidirectional image of the target frame to generate a field image by cutting out an image corresponding to the viewing angle theta _L (step S307).

The control unit 11 of the video output apparatus 1 searches the branch information 26 for the video ID of the destination branch 22c, and identifies the target frame corresponding to the node 21 where the viewpoint position 52 is located. From the omnidirectional image of the target frame to generate a field image by cutting out an image corresponding to the viewing angle theta _L (step S308).

  The control unit 11 of the video output device 1 generates a blend image obtained by blending the visual field image generated in step 307 and the visual field image generated in step 308 based on the mathematical formula (2) (step 309).

  The control unit 11 of the video output device 1 arranges the generated blend image in the visual field image area 62 of the user input screen 61 and outputs it to the display unit 15 (step S310).

  The control unit 11 of the video output device 1 determines whether or not the blend ratio α stored in the storage unit 12 satisfies α ≧ 1 (step S311). When α ≧ 1 is not satisfied (NO in step S311), the control unit 11 of the video output apparatus 1 returns to step S306.

  When α ≧ 1 is satisfied (YES in step S311), the control unit 11 of the video output device 1 updates the branch ID of the target branch 22a stored in the storage unit 12 to the branch ID of the destination branch 22c. (Step S312), the process ends, and the process returns to step S102 of the video output process.

As described above, by executing the blending image display processing, the line-of-sight angle theta _L toward the branch angle theta _E of the destination branch, while gradually changing, and, gradually blend ratio α to 0 to 1 By repeating the processing of step S306 to step S310 while changing, a series of blended images is output as a moving image, and an image in which the viewpoint position 52 continuously moves from the target branch 22a to the destination branch 22c can be presented. it can.

  The preferred embodiments of the video output device 1 and the like according to the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes or modifications can be conceived within the scope of the technical idea disclosed in the present application, and these naturally belong to the technical scope of the present invention. Understood.

DESCRIPTION OF SYMBOLS 1 ......... Video output device 11 ......... Control part 12 ......... Storage part 13 ......... Media input / output part 14 ......... Communication control part 15 ......... Input part 16 ......... Display part 17 ......... Peripheral device I / F unit 20... Floor plan 21... Node 21 a... Origin node 21 b... End node 22... Branch 22 a ... Target branch 25 ... Node information 26. Branch information 27 ......... Video information 52 ......... Viewpoint position 54 ......... Gaze direction 60 ......... Holding data 61 ......... User input screen 90 ......... Omnidirectional image

Claims (7)

An image output device that outputs an image in a line-of-sight direction at a viewpoint position that moves in a route arranged in a coordinate space,
Route storage means for storing a sequence of frames of omnidirectional images in association with the route;
Floor map display means for displaying the route as a floor map;
An input means for receiving movement of the viewpoint position and the line-of-sight direction and receiving an input of a position on the floor map;
Viewpoint position moving means for moving the viewpoint position to a position in the route in the coordinate space corresponding to the position when the input of the position on the floor map is accepted;
Video generation means for generating a video of a visual field range in the viewpoint position and the line-of-sight direction by cutting out from the omnidirectional image;
Video output means for outputting the generated video;
A video output device comprising: The path is composed of nodes and directed branches,
The video output apparatus according to claim 1, wherein a sequence including frames of the omnidirectional image is associated with each directed branch. In the floor map, a two-dimensional coordinate system with a predetermined position as an origin is set,
The viewpoint position moving means converts the position on the floor map into a position in the coordinate space, and moves the viewpoint position to a position in the route closest to the converted position. Item 3. The video output device according to Item 2. The viewpoint position moving means calculates a distance between the converted position and each candidate point set in the branch constituting the route, and moves the viewpoint position to the candidate point having the smallest calculated distance. The video output device according to claim 3, wherein: The video output device according to claim 4, wherein the candidate point is a break point that is generated when the branch is equally divided into a predetermined number. Video performed by a computer having route storage means for storing a sequence of frames of omnidirectional images, which outputs a video in the line-of-sight direction at a viewpoint position moving in a route arranged in a coordinate space, in association with the route. Output method,
A floor map display step for displaying the route as a floor map;
An input step of receiving movement of the viewpoint position and the line-of-sight direction and receiving an input of a position on the floor map;
A viewpoint position moving step of moving the viewpoint position to a position in the path in the coordinate space corresponding to the position when receiving the input of the position on the floor map;
A video generation step of generating a video of a visual field range in the viewpoint position and the line-of-sight direction by cutting out from the omnidirectional image; and
A video output step for outputting the generated video;
A video output method comprising: A program for causing a computer to function as an image output device that outputs an image in a line-of-sight direction at a viewpoint position that moves in a path arranged in a coordinate space,
The computer,
Route storage means for storing a sequence of frames of omnidirectional images in association with the route;
Floor map display means for displaying the route as a floor map;
An input means for receiving movement of the viewpoint position and the line-of-sight direction and receiving an input of a position on the floor map;
Viewpoint position moving means for moving the viewpoint position to a position in the path in the coordinate space corresponding to the position when receiving the input of the position on the floor map;
Video generation means for cutting out and generating a video of a visual field range in the viewpoint position and the line-of-sight direction from the omnidirectional image;
Video output means for outputting the generated video;
Program to function as.

Images