JP4241708B2 - Video image presentation system, video image transmission device, video image presentation device, video image processing program - Google Patents

Description

The present invention relates to a video image presentation system, and more particularly to a video image presentation system, a video image transmission device, a video image presentation device, and a video image that can respond to a user's visual field change request even when accumulated video images are presented. It relates to a processing program.

Conventionally, moving images are set continuously in streets and buildings, and video images are continuously displayed in the same way as when a person observes while walking or moving by car. Recorded and recorded in the same way as when playing a video image according to the user's position from the recorded video image, as if a person walked and observed or moved by car Video video presentation devices that provide video are known.

Hereinafter, a video image that records the movement in a street or a building is referred to as a walk-through image.

“NTT R & D, Vol.49, No.1 PP.25-30, published in 2000” as an example of a video image presentation device that provides walkthrough images, shows walkthrough images that move in actual streets and buildings. An example of a video video presentation device to be provided has been reported, and Japanese Patent Laid-Open No. 10-214069 discloses a video video presentation that provides a walk-through video that moves in a virtual space constructed using a three-dimensional computer graphics technology. An example of a device has been reported.

Furthermore, these video image presentation devices can provide walk-through images to users over a network by using a VOD (Video On Demand) system.

The VOD system stores video images in a server machine in an environment where the server machine and client machine are connected via a network, and distributes the video images stored in the server machine via a network in response to a request from the client machine. A system that presents video images on a client machine is disclosed in, for example, “NEC Technical Bulletin, Vol. 51, No. 8, (August 1998)”.

However, the above-described conventional technique has the following problems.

First, when the user is presented with a walk-through video moving on the route, there is a problem in that the visual field cannot be changed by moving the line of sight freely up and down and left and right.

This is a problem caused by the characteristic of the system in which a video image is recorded in advance and a walk-through image is presented to the user by reproducing the video image.

In order to give a degree of freedom in the line of sight, video images in which various directions are observed when moving on the same route are recorded, and when presenting a walk-through video, it is played back according to the user's request. A method of switching video images is also conceivable.

However, for example, in order to realize a line-of-sight movement in increments of 1 when moving the line of sight 180 degrees in the horizontal direction, 180 video images must be prepared. Considering the storage location of video images, it can be said that it is practically difficult to realize.

Second, for example, a state in which a plurality of routes are moved is divided for each route and stored as a video image, and a video image is switched according to a user's route selection operation to present a walk-through image. In addition, when a service is provided to a user by combining a plurality of video images, when the video image is switched, the image is temporarily stopped at the joint, giving an unnatural impression to the user. There was a problem.

This is because it is necessary to first confirm the location of the video image, read the video image, and start playback before starting to display the video image. This is particularly noticeable when video images stored on a server machine are acquired and displayed via a network in the form of streaming video.

This is because, in the streaming video system, it is generally necessary to establish one virtual connection between the server and the client for one video reproduction. Therefore, when switching video images, the overhead of discarding and establishing the connection In addition, in the streaming video system, a buffering process of about 5 seconds is performed to maintain smooth playback even when the network communication status changes. Even if this occurs, it is necessary to wait for the buffering time to elapse before the actual reproduction starts.

The first object of the present invention is to solve the above-mentioned drawbacks of the prior art and to perform video line presentation system, video picture transmission apparatus, video picture presentation apparatus, and video picture processing capable of moving the line of sight freely and smoothly. To provide a program.

The second object of the present invention is to solve the above-mentioned drawbacks of the prior art, particularly to solve these problems that become prominent when connected via a network. The present invention provides a video image presentation system, a video image transmission device, a video image presentation device, and a video image processing program capable of smoothly displaying video images without being conscious of the joints between the two.

In order to achieve the above object, the present invention provides a video image presentation system for presenting captured video images, which accepts designation of a gaze direction of a user who views the video images, and corresponds to the gaze direction in the video images. A cutout range is calculated, the video image is cut out based on the calculated cutout range, and presented as a new video image.

According to a second aspect of the present invention, there is provided a video image presentation system for presenting the video image, and a video image transmission device for cutting out the video image and transmitting a new video image to the image presentation device. It is characterized by providing.

In the video image presentation system of the present invention according to claim 3, the video image transmission device calculates the cutout range in the video image corresponding to the line-of-sight direction of the user notified from the image presentation device, A video image is cut out based on the calculated cut-out range, and is sent to the video presentation device as a new video image.

5. The video image presentation system according to claim 4, wherein the image presentation device calculates the cutout range in the video image corresponding to the direction of the user's line of sight, and the video image is transmitted from the video image transmission device. To cut out and present a new video image based on the cutout range.

6. The video image presentation system according to claim 5, wherein the video image transmission device calculates a cutout range in the video image corresponding to the line-of-sight direction of the user notified from the image presentation device, and the video Of each block obtained by dividing the video screen into predetermined sizes, the block area in the range including the cutout range in the video video is cut out as a new video video and sent to the video presentation device. It is characterized by that.

In the video image presentation system of the present invention according to claim 6, the image presentation device calculates a first cutout range in the video image corresponding to the user's line-of-sight direction, and the image transmission device Based on one cut-out range, a second cut-out range that is larger than the first cut-out range is calculated, and based on the second cut-out range, the video image is cut out, and the video as a new video image The video presentation device that is sent to a presentation device and receives the new video image cuts out and presents the new video image based on the first cutout range.

In the video image presentation system according to the seventh aspect of the present invention, the image presentation device receives the second cutout range from the image sending device, and compares the first cutout range with the second cutout range. Thus, when the first cutout range is included in the second cutout range, the new video image is cut out based on the first cutout range, and the first cutout range is the second cutout range. If it is not included in the range, the first cutout range is changed to be included in the second cutout range, and the new video image is cut out based on the changed first cutout range. Features.

The video image presentation system of the present invention according to claim 8 is characterized in that the video image transmission device transmits an image of the video image to be transmitted to the image presentation device based on the number of the image presentation devices to which the video image is transmitted. By changing the size to a predetermined enlargement ratio, the communication amount of the video image is suppressed below the communication band between the video image transmission device and the image presentation device.

The video image presentation system of the present invention according to claim 9 is configured such that the video image transmission device determines the size of the video image image to be transmitted to the image presentation device based on an instruction from the image presentation device. By changing to, the communication amount of the video image is suppressed below the communication band between the video image transmission device and the image presentation device.

The video image presentation system of the present invention according to claim 10 is characterized in that the image presentation device determines the size of the image of the video image transmitted by the video image transmission device to the video image transmission device, and the reception state of the video image. It is characterized by instructing based on.

The video image presentation system of the present invention according to claim 11 is characterized in that the image presentation device determines the size of the image of the video image transmitted by the video image transmission device to the video image transmission device. It is specified based on the processing status.

A video image presentation system according to a twelfth aspect of the present invention is characterized in that the video image is transmitted from the image presentation device to the image presentation device via a network.

14. The video image presentation system according to claim 13, wherein the image presentation device compresses and transmits the video image to be transmitted, and the image presentation device receives the compressed video received from the image presentation device. The video is decoded and presented.

The video image presentation system of the present invention according to claim 14 receives a plurality of video images cut out simultaneously from the image transmission device, selectively switches the video images, and presents the image to the image presentation device. It is characterized by comprising a switching device that transmits without changing the connection with the device.

The video image presentation system of the present invention according to claim 15 is characterized in that the video image is a walk-through image in which a state of moving in a street or a building is recorded.

The video image presentation system of the present invention according to claim 16 is characterized in that the video image is a wide-field video image including an image in a range in which the user moves the line of sight.

According to a seventeenth aspect of the present invention, in the video image transmitting device for transmitting a video image to the image presenting device, the video image corresponding to the viewing direction of the user viewing the video image notified from the image presenting device. And a cutout unit for cutting out the cutout range of the video image as a new video image and sending it to the video presentation device.

The video image transmitting device according to the present invention of claim 18 calculates the video image screen corresponding to the line-of-sight direction of the user in the video image within each block obtained by dividing the screen of the video image by a predetermined size. Means further comprising means for cutting out the area of each block in the range including the cutout range as a new video video and sending it to the video presentation device.

The video image sending device of the present invention according to claim 19 is configured such that the size of the image of the video image sent to the video presentation device is set to a predetermined enlargement ratio based on the number of the video presentation devices to which the video image is sent. By providing means for changing, the communication amount of the video image is suppressed below the communication band with the image presentation device.

The video video transmission device according to the present invention of claim 20 comprises means for changing the size of the video video image transmitted to the video presentation device to a predetermined enlargement ratio based on an instruction from the video presentation device. Thus, the communication amount of the video image is suppressed to be equal to or less than the communication band with the image presentation device.

According to a twenty-first aspect of the present invention, there is provided a video image transmitting apparatus according to the present invention, comprising means for compressing and transmitting the video image.

According to a twenty-second aspect of the present invention, there is provided a video image transmitting device according to the present invention, wherein the video image is transmitted to the image presentation device via a network.

According to a twenty-third aspect of the present invention, there is provided a video presentation device for presenting a video image transmitted from a video image transmission device, and means for instructing the video image transmission device to specify a line-of-sight direction of a user who views the video image. The video image sending device receives and presents a new video image cut out from the video image in a cut-out range corresponding to the line-of-sight direction.

25. The video presentation apparatus according to the present invention of claim 24, wherein said video transmission is based on means for calculating a first cutout range in said video video corresponding to said user's line-of-sight direction, and said first cutout range. A new video image cut out in a second cutout range that is larger than the first cutout range calculated by the apparatus is received, and the new video image is cut out and presented based on the first cutout range. A cutting-out means is provided.

In the video presentation device of the present invention according to claim 25, the cutout unit receives the first cutout range from the image sending device, and compares the first cutout range with the second cutout range, When the first cutout range is included in the second cutout range, the new video image is cut out based on the first cutout range, and the first cutout range becomes the second cutout range. When not included, the first cutout range is changed so as to be included in the second cutout range, and the new video image is cut out based on the changed first cutout range. To do.

The video presentation apparatus of the present invention of claim 26 is a means for instructing the video video transmission device to change the size of the video video image to a predetermined enlargement ratio based on the reception status of the video video. It is characterized by providing.

The video presentation device according to a twenty-seventh aspect of the present invention instructs the video video transmission device to change the size of the video video image to a predetermined enlargement ratio based on the processing status of the received video video. Means are provided.

The video presentation device of the present invention according to claim 28 is characterized in that the video video is received from the video video transmission device via a network.

A video presentation apparatus according to a twenty-ninth aspect of the present invention comprises means for simultaneously receiving a plurality of video videos from the video video transmission device and selectively switching and presenting the video videos.

A video presentation apparatus according to a thirty-third aspect of the present invention is characterized in that the video video compressed and transmitted from the video video transmission device is decoded and presented.

According to a thirty-first aspect of the present invention, in a video processing program for transmitting a video image from a video transmission device to a video presentation device by controlling a computer, a user who views the video video notified from the video presentation device A function for calculating a cutout range in the video image corresponding to the line-of-sight direction, and a function for cutting out the video image based on the calculated cutout range and presenting the video image as a new video image. Features.

According to a thirty-second aspect of the present invention, in a video processing program for sending and presenting a video image from a video transmitting device to a video presenting device by controlling a computer, designation of the line-of-sight direction of a user who views the video image And a function for receiving and presenting a new video image cut out from the video image in a cut-out range corresponding to the line-of-sight direction in the video image sending device. It is characterized by.

A video processing program according to a thirty-third aspect of the present invention is based on the function of calculating a first cutout range in the video video corresponding to the user's line-of-sight direction, and the video transmission based on the first cutout range. A new video image cut out in a second cutout range that is larger than the first cutout range calculated by the apparatus is received, and the new video image is cut out and presented based on the first cutout range. It is characterized by having a function.

In the video processing program of the present invention according to claim 34, the cutout function receives the first cutout range from the image sending device, and compares the first cutout range with the second cutout range, When the first cutout range is included in the second cutout range, the new video image is cut out based on the first cutout range, and the first cutout range becomes the second cutout range. When not included, the first cutout range is changed so as to be included in the second cutout range, and the new video image is cut out based on the changed first cutout range. To do.

In the present invention, based on the designation of the gaze direction of the user viewing the video image, the region corresponding to the gaze direction is cut out from the video image and presented, thereby continuously changing the gaze direction in the vertical and horizontal directions. The video image can be switched according to the change.

Further, in the present invention, in the video image sending device, an area corresponding to the line-of-sight direction is roughly cut out from the video image in units of blocks, and the video presentation device receives the roughly cut-out video image, so that a precise line-of-sight is received. A range corresponding to the direction can be cut out and displayed.

Further, according to the present invention, it is possible to control and distribute the video image size based on the network bandwidth between the video image transmission device and the image presentation device.

Further, in the present invention, when a delay in transmission of a video image caused by a change in the network situation or a frame loss thereof is detected, the size of the image of the video image transmitted by the video image transmission apparatus in order to reduce the situation. By controlling the above, QoS control can be realized.

Further, in the present invention, smooth video switching can be realized by switching video video in one connection.

As described above, according to the present invention, the following effects are achieved.

First, it can be used by playing back video footage that records a walk-through in a virtual space built using 3D computer graphics technology, or a video footage that records a walk-through in an actual street. In the system that presents the movement in the space to the user, the user's free and smooth line-of-sight movement can be realized.

Second, in a system that presents movement in space to a user by combining a plurality of video images, the video image can be smoothly displayed without making the user aware of the joint between the video images. . That is, by smoothing the connection between video images, the user can move naturally in space.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing a configuration of a video image presentation system according to the first embodiment of the present invention.

Referring to FIG. 1, the video image presentation system according to the present embodiment shows a walk-through image presentation device 1 (walk-through image presentation client) that presents a walk-through image to a user, and a situation in which it moves in a street or a building. A video image storage device 2 for storing continuously recorded video images, and a video image stored in the video image storage device 2 are processed to generate a video image to be displayed on the walk-through image presentation device 1. A video video transmission device 3 (video video transmission server) for transmission to the walk-through video presentation device 1 is provided.

The walk-through video presentation device 1 and the video video transmission device 3 are connected to each other via a network 10 as shown in FIG. 2, and the walk-through video transmitted from the video video transmission device 3 is sent to each walk-through video presentation device. 1 to receive and play.

The walk-through video presentation device 1 according to the present embodiment is realized by a program-controlled CPU or the like, and includes a video video selection unit 11 that selects a video video to be played back according to an input from the user, an input from the user, and the like. , A line-of-sight input unit 12 that determines the line-of-sight direction to be observed, a video image decoding unit 13 that decodes a video image received from the video image transmission device 3 according to an encoding method, and a video decoded by the video image decoding unit 13 A video image display unit 14 that displays images to the user is provided.

The video image sending device 3 according to the present embodiment is realized by a program-controlled CPU or the like, holds a video image name received from the walk-through image presentation device 1, and continuously applies the corresponding video image from the video image storage device 2. A video image reading unit 31 that reads out automatically, a cutout region calculation unit 32 that calculates which region in the video image is to be cut out and displayed according to the line-of-sight direction received from the walk-through image presentation device 1, and a cutout region calculation unit 32 A cutout part storage unit 33 that stores the calculated cutout part information S3 and a part to be displayed on the walk-through video presentation device 1 from the video image read by the video image reading unit 31 according to the cutout part stored in the cutout part storage unit 33 A block cutout unit 34 for cutting out is provided.

Next, the operation of the video image presentation system according to this embodiment will be described.

The video image storage device 2 stores a video image that records a walk-through in a virtual space constructed using 3D computer graphics technology, or a video image that records a walk-through in an actual street. ing. Note that the video image stored in the video image storage device 2 is a wide-field video image including a vertical and horizontal range in which the user moves the line-of-sight direction.

First, the flow of the walk-through video display will be described with reference to the flowchart of FIG.

The video image selection unit 11 determines a video image (video image name) to be reproduced in accordance with an instruction input from the user. In determining the video image, for example, a list of video image names is presented, a determination method for selecting a video image name from the list, a determination method for directly specifying the URL of the video image name, and the like can be considered. Any method of determination may be used. The video image name selected by the video image selection unit 11 is transmitted to the video image transmission device 3 as video image information S1.

In the video image transmission device 3, the transmitted video image information S1 is held by the video image reading unit 31, and the video images stored in the video image storage device 2 are sequentially read out for each frame according to the video image information S1. (Step 301), the data is sent to the block cutout unit 34.

When the user's line-of-sight direction is input from the line-of-sight method input unit 12 of the walk-through video presentation device 1 (step 302), the input line-of-sight direction information S2 (that is, the direction in which the user's line of sight is directed); Based on the display area information S5 indicating the shape and size of the video image display area acquired from the video image display unit 14, the cutout part calculation unit 32 determines which part of the video image is cut out and displayed as a walk-through image. The designated cutout site information S3 is calculated (step 303).

The line-of-sight direction input unit 12 receives a direction in which the user's line of sight (line-of-sight direction) is input. The line-of-sight direction is a value indicating which part of the video image the user is gazing at and can be specified by any method, but as a difference value (number of pixels in the horizontal and vertical directions) from the center of the video image An example is a method of designating. The line-of-sight direction is sent from the line-of-sight direction input unit 12 to the cut-out site calculation unit 32 as line-of-sight direction information S2.

Further, as the line-of-sight direction input unit 12, for example, a cross direction instruction key or the like that is operated by the user to input the line-of-sight direction on the display screen of the video image display unit 14 of the walk-through video presentation device 1 may be displayed. It is feasible.

On the other hand, the cutout site storage unit 33 stores the cutout site information S3 calculated by the cutout site calculation unit 32 (step 304).

When the line-of-sight direction is not input from the line-of-sight method input unit 12 (in the initial state or when there is no change in the line-of-sight direction) (step 302), the cut-out part calculation unit 32 does not calculate cut-out part information and cuts out in advance as an initial value. The cutout part information stored in the part storage unit 33 or the cutout part information calculated last time and stored in the cutout part storage unit 33 is sent to the block cutout unit 34 described later.

The cutout part information stored in the cutout part storage unit 33 as an initial value is information indicating a predetermined range (whole or part) of a video image, for example.

As the cutout part information S3 stored in the cutout part storage unit 33, the coordinate value (pixel value) in the video image of the part to be cut out, the ratio in the video image of the part to be cut out, and the like can be considered. Other information may be used as long as the information can be specified.

In the block cutout unit 34, for each frame of the video image sent from the video image reading unit 31, the video image to be displayed is cut out in accordance with the cutout portion information S 3 stored in the cutout portion storage unit 33. Perform (step 305). The cut-out video image is sent to the walk-through image presentation device 1 in the form of streaming video as a walk-through image (step 306).

The walk-through video presentation device 1 appropriately decodes the sent walk-through video (video video) according to the encoding method and sends it to the video video display unit 14. The video image display unit 14 presents video images to the user.

4 and 5 show the relationship between the video image stored in the video image storage device 2 and the walk-through image actually presented by the user.

A video image 101 shown in FIG. 4 is a scene of the video image stored in the video image storage device 2. The video image sending device 3 determines the cutout range 102 (cutout site information) according to the user's line-of-sight direction, cuts out the video image 103 as shown in FIG. 5, and sends it to the walk-through video presentation device 1. The walk walkthrough video presentation device 1 presents the video video 103 to the user as a walkthrough video.

Here, an example of a video video clipping method in the block clipping unit 34 will be described for each type of video video stored in the video video storage device 2.

(1) When the video image stored in the video image storage device 2 is in an uncompressed format, the following method can be considered.

The data of each frame sent from the video image reading unit 31 to the block cutout unit 34 is a sequence of uncompressed bitmap images. In this case, the part stored in the cutout part storage unit 33 is cut out from the bitmap image of each frame that has been sent.

(2) When the video image stored in the video image storage device 2 is an MPEG4 video image, the following method can be considered.

The MPEG4 video image is encoded in units of 16 × 16 pixel blocks (macroblocks). Therefore, if the cutout unit is cut out in units of macroblocks, cutout processing can be performed without decoding, cutting out, and re-encoding for each frame. In order to reduce the amount of calculation at the time of clipping, it is recommended that the clipping process be performed in units of macroblocks in the case of MPEG4 video images. Similar to the MPEG4 format, a video image divided into macroblocks and encoded is MPEG2 or H.264. 263. In the case of these video images, the clipping process is performed in the same manner as the MPEG4 video image.

(3) When the video image stored in the video image storage device 2 does not correspond to the above (1) and (2), the block cutout unit 34 decodes the transmitted video image and decodes the uncompressed image. Then, cut out and encode again.

Next, a procedure for changing the cutout part will be described.

When the line-of-sight direction is changed in accordance with an instruction input from the user or the like in the line-of-sight direction input unit 12, the changed line-of-sight direction information S <b> 2 is sent to the cutout site calculation unit 32. The cutout part calculation unit 32 calculates cutout part information S3 according to the received gaze direction change information and updates the information about the cutout part stored in the cutout part storage unit 33 (steps 303 and 304 in FIG. 3). ). When the cutout part information S3 stored in the cutout part storage unit 33 is changed, the range of the video image cut out from the video image by the block cutout unit 34 is changed.

As a result, the video part that is cut out from the video image as a walk-through image is changed, so that the same visual field change as when the sight line direction is changed is obtained, and the video image is changed in accordance with the change of the sight line direction of the user. Is possible.

FIGS. 6 to 8 are schematic diagrams showing changes in the walk-through video when the cutout part is changed. Here, for simplification of description, the same video image is always sent from the video image reading unit 31 to the block cutout unit 34.

In FIG. 6, a video image 111 is a video image sent from the video image reading unit 31. Now, it is assumed that the cutout site information S3 indicated by the site 112 is stored in the cutout site storage unit 33. At this time, the walk-through video cut out by the block cut-out unit 34 and displayed on the walk-through video presentation device 1 is a video video 113 shown in FIG.

Next, it is assumed that the cutout part stored in the cutout part storage unit 33 is changed to the cutout part indicated by the part 114. At this time, the walk-through video cut out by the block cut-out unit 34 and displayed on the walk-through video presentation device 1 is a video video 115 shown in FIG.

By changing the walk-through video presented by the walk-through video presentation device 1 from the video video 113 to the video video 115, the user can obtain a display as if the direction of the line of sight has been moved to the right.

Here, when the cutout part information S3 stored in the cutout part storage unit 33 is changed in the cutout part calculation unit 32, the cutout part is not changed from the cutout part before the change to the cutout part after the change. An example is also conceivable in which the cutout part calculation unit 32 is provided with a function of changing the above in a stepwise manner so that the walk-through video presented to the user can be smoothly moved.

FIG. 9 is a schematic diagram illustrating changes in the cutout site when the cutout site calculation unit 32 that implements this function is used. In FIG. 9, it is assumed that a cutout part 116 is set for the video image 111 in the initial state.

At this time, when the cutout part stored in the cutout part storage unit 33 is changed to the part 117 by the signal from the line-of-sight direction input unit 12, the cutout part calculation unit 32 sets the cutout parts to the parts 118, 119, 120, Change sequentially to 117 and 4 steps. That is, until the video image of the portion 117 is presented, the video images whose positions are moved little by little like the portions 118, 119, and 120 are sequentially presented. Thereby, the walk-through video displayed on the walk-through video presentation device 1 can be smoothly scrolled.

In this example, the case of changing to four stages has been described, but the number of stages may be any number.

As described above, in the video image presentation system according to the present embodiment, the video image stored in the video image storage device 2 is a wide-field video image including a vertical and horizontal range in which the user moves the line of sight, and the video image In the sending device 3, a part of the wide-field video image is cut out and sent as a walk-through video, and each time the user performs a change operation of the gaze direction, the new gaze direction is the center. By changing the video cutout position in the region, the user can walk through and simultaneously change the line-of-sight direction vertically and horizontally.

Next, a second embodiment of the present invention will be described. FIG. 10 is a block diagram showing a configuration of a video image presentation system according to the second embodiment of the present invention.

Referring to FIG. 10, the video image presentation system according to the present embodiment continuously records the walk-through image presentation device (client) 1a that presents the walk-through image to the user and the movement in the street or the building. A video image storage device 2 for storing the video image, and processing the video image stored in the video image storage device 2 to generate a video image to be displayed by the walk-through image presentation device 1a, and presenting the walk-through image A video image sending device (server) 3 for sending to the device 1a is provided. Also in this embodiment, the video image transmission device 3 and the walk-through image presentation device 1a are connected to each other via the network 10 as shown in FIG.

The walk-through video presenting apparatus 1a according to the present embodiment includes a video image selection unit 11 that selects a video image to be reproduced according to an input from a user, and a line-of-sight direction that determines a line-of-sight direction to be observed according to an input from the user. Input part 12, display part calculation part 15 that calculates a part to be displayed according to the line-of-sight direction determined by line-of-sight direction input part 12, and display part storage part 16 that stores cut-out part information S3 calculated by display part calculation part 15 A video video decoding unit 13 that decodes the video video received from the video video transmission device 3 in accordance with the encoding format, and the video video decoded by the video video decoding unit 13 according to the display site stored in the display site storage unit 16. A display part cutout unit 17 that cuts out a part to be displayed and a display part cutout unit 17 Ri comprises a video display unit 14 for displaying to the user a video image that issued.

The video image transmission device 3 of the present embodiment holds the video image information S1 (video image name) received from the walk-through image presentation device 1a, and continuously applies the corresponding video image from the video image storage device 2. A video image reading unit 31 to be read, a cutout site calculation unit 32 that calculates a site to be sent to the walkthrough video presentation device 1a according to the display site calculated by the walkthrough video presentation device 1a, and a cutout site calculated by the cutout site calculation unit 32 And a block cutout unit for cutting out a part to be sent to the walk-through video presentation device 1a from the video image read out by the video image reading unit 31 according to the cutout part stored in the cutout part storage unit 33 34 is provided.

Next, the operation of the video image presentation system according to this embodiment will be described.

The video image storage device 2 stores a video image that records a walk-through in a virtual space constructed using 3D computer graphics technology or a video image that records a walk-through in an actual street. ing. Note that the video image stored in the video image storage device 2 is a wide-field video image including a vertical and horizontal range in which the user moves the line-of-sight direction.

First, the flow of the walk-through video display will be described with reference to FIG. 11 showing the processing of the video video transmission device 3 and the flowchart of FIG. 12 showing the processing of the walk-through video presentation device 1a.

The video image selection unit 11 determines a video image (video image name) to be reproduced in accordance with an instruction input from the user. For the determination, for example, as in the first embodiment, a determination method for selecting a video image name from a list of video image names, a determination method for directly specifying a URL of a video image name, and the like can be considered. The video image name selected by the video image selection unit 11 is transmitted to the video image transmission device 3 as video image information S1.

In the video image transmission device 3, the transmitted video image information S1 is held by the video image reading unit 31, and the video images stored in the video image storage device 2 are sequentially read out for each frame according to the video image information S1. (Step 1101), the data is sent to the block cutout unit 34.

On the other hand, in the walk-through video presentation device 1a, the direction of the user's line of sight (the line of sight) is input by the line-of-sight direction input unit 12 (step 1201). The line-of-sight direction is a value indicating which part of the video image the user is gazing at and can be specified by any method, but as a difference value (number of pixels in the horizontal and vertical directions) from the center of the video image An example is a method of expressing. The line-of-sight direction is sent from the line-of-sight direction input unit 12 to the display part calculation unit 15 as line-of-sight direction information S2.

The display part calculation unit 15 displays the display part based on the line-of-sight direction information S2 received from the line-of-sight direction input unit 12 and the display area information S5 indicating the shape and size of the video image display area acquired from the video image display unit 14. Information S4 is calculated (step 1202) and stored in the display part storage unit 16 (step 1203). The display part information S4 is represented by a set of the line-of-sight direction and the shape and size of the video display area.

At the same time, the display part calculation unit 15 calculates a circumscribing rectangle of the display part information as the display part circumscribing rectangle information (step 1202), and the cutout part calculation unit 32 of the video image transmission device 3 as the display part circumscribing rectangle information S6. (Step 1203). The display part circumscribing rectangle information S6 includes a viewing direction and the vertical and horizontal sizes of the circumscribing rectangle.

When the display part circumscribing rectangle information S6 is sent from the display part calculating unit 15 (step 1102), the cutout part calculating unit 32 of the video image transmission device 3 determines the sent display part circumscribing rectangle information S6 in advance. In accordance with the enlargement rate, the calculation is performed to enlarge α times in the horizontal direction and β times in the vertical direction (step 1103), and the result is stored in the cutout part storage unit 33 as cutout part information S3 (step 1104). Here, the enlargement ratios α and β can be set to arbitrary values of 1 or more, but values of about 1.1 to 1.5 are preferable.

Further, when the display part circumscribing rectangle information S6 is not input from the display part calculation unit 15 (when there is no change in the initial state or the line-of-sight direction) (step 1102), the cut-out part calculation unit 32 does not calculate the cut-out part information S3, Cutout part information stored in advance in the cutout part storage unit 33 as an initial value or cutout part information S3 calculated in the previous time and stored in the cutout part storage unit 33 is sent to the block cutout part 34 described later.

The cutout part information stored in the cutout part storage unit 33 as an initial value is information indicating a predetermined range (whole or part) of a video image, for example.

FIG. 13 is a schematic diagram showing the relationship between the line-of-sight direction, display part information, display part circumscribed rectangle information, and cutout part information.

In the schematic diagram of FIG. 13, a point 200 represents the line-of-sight direction, an area 201 represents display part information, an area 202 represents display part circumscribing information that is a circumscribed rectangle of the area 201, and an area 203 represents the area 202 in the horizontal direction and In the vertical direction, the cut-out parts are α-fold and β-fold respectively. In the schematic diagram of FIG. 13, the video image display area is illustrated as an elliptical shape, but the shape of the video image display area may be other shapes such as a rectangle.

On the other hand, the block cutout unit 34 of the video video transmission device 3 displays each frame of the video video sent from the video video reading unit 31 according to the cutout part information S3 stored in the cutout part storage unit 33. The video image to be cut out is processed (step 1105). The cut-out video image is sent as a walk-through image to the video image decoding unit 13 of the walk-through image presenting apparatus 1a in the form of streaming video (step 1106). At the same time, the cutout part information S3 is sent from the block cutout part 34 to the display part cutout part 17 (step 1107).

The video video decoding unit 13 of the walk-through video presentation device 1a decodes each frame of the video video sequentially sent as a video stream from the block cutout unit 34 according to the encoding format, and displays the decoded video video This is sent to the part cutout unit 17.

The display part cutout unit 17 receives the cutout part information S3 sent from the block cutout part 34 and the display part information S4 stored in the display part storage unit 16 and sends it from the video image decoding unit 13. Which part of the video image should be displayed is calculated (step 1204), and the video image to be displayed is cut out (step 1205). The cut video image is sent to the video image display unit 14.

The video image display unit 14 displays the video image as a walk-through image (step 1206).

Here, the processing content of the display part cutout unit 17 described above will be described in detail with reference to the flowchart of FIG.

When the video image is received from the video image decoding unit 13 (step 1401), the display part information S4 is acquired from the display part storage unit 16 and the cutout part information S3 is acquired from the block cutout part 34 (step 1402).

Next, by comparing the display part information S4 and the cut-out part information S3, it is determined whether or not the area indicated by the display part information S4 is completely included in the area indicated by the cut-out part information S3 (step 1403). ).

If it is completely included, the area indicated by the display part information S4 is cut out from the received video image (step 1404), and the cut out video image is sent to the video image display unit 14 (step 1405).

If the area indicated by the display part information S4 is not completely included in the area indicated by the cut-out part information S3 in step 1403, the area indicated by the display part information S4 is changed according to a predetermined rule described below (step 1406), the received video image is cut out based on the changed display part information.

The display part information S4 is changed according to the following rules (1) and (2). (1) The area shape and size are not changed. (2) The position of the area is changed to a position that fits in the area indicated by the cut-out part information and is closest to the area indicated by the display part information.

15 to 17, the video image stored in the video image storage device 2, the video image sent from the block clipping unit 34 to the display part clipping unit 17 via the video image decoding unit 13, and the video image display 14 Shows the relationship with the walk-through video actually observed by the user.

The video image 211 is a scene of the video image stored in the video image storage device 2. A range 212 is a range indicated by the display part information S4 calculated by the display part calculation unit 15, and a range 213 is a range indicated by the cutout part information S3 calculated by the cutout part calculation unit 32 (FIG. 15). The video image 214 is a video image cut out by the block cutout unit 34 in accordance with the cutout part information S3 indicating the range 213 (FIG. 16).

The block cutout unit 34 cuts out the video image 214 in FIG. 16 from the video image 211 in FIG. 15 based on the range 213. The video image 215 is a video image in which the video image 214 is cut out as a walk-through image by the display part cutout unit 17 (FIG. 17).

In this example, since the range 212 of the front part part information S4 is completely included in the range 213 of the cutout part information S3, the display part cutout unit 17 cuts out the video image 215 based on the range 212. This video image 215 is presented to the user by the video image display unit 14.

Here, the clipping performed by the block clipping unit 34 when the video image stored in the video image storage device 2 is encoded in the MPEG4 format will be described.

Video images encoded in the MPEG4 format are encoded in units of 16 × 16 pixel blocks (called macroblocks). Therefore, if the block cutout unit 34 specifies that the cutout is cut out in units of macroblocks, the cutout process can be performed without decoding, cutting out, and re-encoding for each frame, thereby reducing the amount of calculation and increasing the processing speed. Can be planned.

Therefore, in the case of a video image in the MPEG4 format, the block cutout unit 34 lists macroblocks including the area indicated by the cutout part stored in the cutout part storage unit 33 and cuts out all the listed macroblocks. I will do it. That is, an area slightly larger than the area indicated by the cutout part information S3 stored in the cutout part storage unit 33 is cut out.

FIG. 18 and FIG. 19 are schematic diagrams showing cutout areas when a video image is cut out in units of macroblocks.

An area 221 indicates an area of the video image at a certain time of the video image stored in the video image storage device 2. An area 222 indicates an area of the cutout part information S3 stored in the cutout part storage unit 33. When the macroblock and cutout area of the video image have the relationship as shown in FIG. 18, the video image area actually cut out by the block cutout unit 34 is an area 224 shown in FIG. That is, in this example, a region 224 surrounded by the macroblocks MB5, MB6, MB7, MB9, MB10, and MB11 including the region 222 of the cutout site information S3 is cut out.

Similar to the MPEG4 format, a video image divided into macro blocks and encoded is H.264. 263. In the case of these video images, the processing can be performed in the same manner as the MPEG4 video image. The video image stored in the video image storage device 2 is preferably encoded in this type of format.

Next, a processing procedure when the line-of-sight direction input to the line-of-sight direction input unit 12 is changed will be described.

When the line-of-sight direction is changed in accordance with an instruction input from the user or the like in the line-of-sight direction input unit 12, the changed line-of-sight direction information S <b> 2 is sent to the display part calculation unit 15. The display part calculation unit 15 calculates the display part information S4 and the display part circumscribing rectangle information S6 according to the above-described procedure according to the received gaze direction change information (step 1202 in FIG. 12). Then, it is sent to the cutout site calculation unit 32 (step 1203 in FIG. 12).

The cutout part calculation unit 32 calculates cutout part information S3 according to the procedure described above in accordance with the sent display part circumscribed rectangle information S6 (step 1103 in FIG. 11), and changes the cutout part. By changing the cutout part information S3, the range of the video image cut out from the video image by the block cutout unit 34 is changed.

On the other hand, as the display part information S4 stored in the display part storage unit 16 is changed, the video image cut out by the display part cutout unit 17 is changed. As a result, the video part that is cut out from the video image as a walk-through image is changed, so that the same visual field change as when the line-of-sight direction is changed is obtained, and the video image according to the change of the user's line-of-sight direction Can be changed.

FIG. 21 to FIG. 23 are schematic diagrams showing changes in the walk-through video when the line-of-sight direction is changed. Here, for simplification of description, the same video image is always sent from the video image reading unit 31 to the block cutout unit 34.

In FIG. 21, a video image 251 is a video image sent from the video image reading unit 31. When the part indicated by the display part information S4 stored in the display part storage unit 16 is the part 252 and the part indicated by the cutout part information S3 stored in the cutout part storage unit 33 is the part 253, The video image finally displayed on the video image display unit 14 through the block cutout unit 34 and the display part cutout unit 17 is a video image 254 shown in FIG.

Next, the line-of-sight direction is changed in the line-of-sight direction input unit 12, and as shown in FIG. 21, the part indicated by the display part information S4 stored in the display part storage part 16 is the part 255, and the cut-out part storage part 33 Assume that the part indicated by the cutout part information S3 stored in is changed to the part 256. At this time, the video image finally displayed on the video image display unit 14 through the block cutout unit 34 and the display part cutout unit 17 becomes a video image 257 shown in FIG.

Finally, the line-of-sight direction is changed in the line-of-sight direction input unit 12, and as shown in FIG. 21, the part indicated by the display part information S4 stored in the display part storage part 16 is the part 258, and the cut-out part storage part 33 Assume that the part indicated by the cut-out part information S3 stored in is changed to part 259. At this time, the video image finally displayed on the video image display unit 14 through the block cutout unit 34 and the display part cutout unit 17 becomes a video image 260 shown in FIG.

As shown in FIGS. 21 to 24, the walk-through video presented by the walk-through video presentation device 1a changes from the video video 254 to the video video 257 and further to the video video 260, so that the user looks as if the direction of the line of sight Can be obtained as if it was moved to the right.

FIG. 25 and FIG. 26 show schematic views illustrating other examples of the change of the walk-through video when the line-of-sight direction is changed. Here, for simplification of description, the same video image is always sent from the video image reading unit 31 to the block cutout unit 34.

In FIG. 25, a video image 251 is a video image sent from the video image reading unit 31. Now, assume that the part indicated by the display part information S4 stored in the display part storage unit 16 is the part 271 and the part indicated by the cut-out part information S3 stored in the cut-out part storage unit 33 is the part 272.

In this case, since the part 271 indicated by the display part information S4 is not completely included in the part 272 indicated by the cutout part information S3, the display part information S4 is changed by the display part cutout unit 17 in accordance with the rules described above. . As a result, the part indicated by the changed display part information S4 is the part 273 in FIG.

Thereby, the video image finally displayed on the video image display unit 14 through the display part cutout unit 17 becomes a video image 274 shown in FIG.

As shown in FIGS. 25 and 26, even when the update of the cutout part of the video image is delayed, the video image sent to the walk-through image presentation device 1a is stored in the display part storage unit 16. Since the size is larger than the part, it is possible to move the line of sight without delaying the cutout by the display part cutout unit 17.

Note that when the display part calculation unit 15 calculates the display part, the display part calculation unit 15 has a function of changing the display part step by step instead of suddenly changing from the display part before change to the display part after change. For example, an embodiment that can smoothly change the change in the walk-through video accompanying the line-of-sight change is also conceivable.

FIG. 20 is a schematic diagram for explaining the change of the display part when the display part cutout unit 17 that implements this function is used. In FIG. 20, it is assumed that a display part 242 is set for the video image 241 in the initial state.

At this time, when the display part stored in the display part storage unit 16 is changed to the part 243 by the signal from the line-of-sight direction input unit 12, the display part calculation unit 15 sets the display part 242 to the parts 244, 245, and 246. 243 and 4 steps in order. In other words, before the video of the part 243 is presented, videos in which the positions are moved little by little like the parts 244, 245, and 246 are sequentially presented. Thereby, the walk-through video displayed on the walk-through video presentation device 1a can be smoothly scrolled. In this example, the case of changing to four stages has been described, but the number of stages may be any number.

Here, when implementing the video image presentation system of the present embodiment, there is an example in which the walk-through image presentation device 1a and the video image transmission device 3 are connected via a network such as a LAN or a WAN. Conceivable.

In the case of this embodiment, the video image transmission device 3 and the video image storage device 2 are installed on the server machine, and the walk-through image presentation device 1a is provided on the client machine. Can provide services.

In this way, when the walk-through video presentation device 1a and the video video transmission device 3 are connected via a network, the display part circumscribing rectangle information S6 is transmitted from the display part calculation unit 15 to the cutout part calculation unit 32. There is a case where a slight time delay occurs, and due to this, it is expected that the part of the video image sent from the block cutout unit 34 is delayed.

That is, although the display part information S4 stored in the display part storage unit 16 is updated, a situation may occur where the part of the video image sent from the block cutout unit 34 remains unchanged. In such a situation, there is a possibility that the display part information S4 is not included in the cutout part information S3. In this case, the display part cutout unit 17 changes the display part information S4 as described with reference to FIG. I do.

Even when the update of the part of the video image sent in this way is delayed, in the video image presentation system according to the present embodiment, the video image (that is, the block cutout unit) sent to the walk-through image presentation device 1a. Since the video image cut out at 34 is larger in size than the display part stored in the display part storage unit 16, it is possible to realize line-of-sight movement without delaying the cutout by the display part cutout unit 17. is there.

As described above, in addition to the effects of the first embodiment, the video image presentation system according to the present embodiment provides a network from one video image transmission device 3 to a plurality of walkthrough image presentation devices 1a. Video images can be distributed efficiently. For example, the video image transmission device 3 and the video image storage device 2 are installed on the server machine, and the walk-through image presentation device 1a is provided on the client machine, so that the video image can be transmitted from one server machine to a plurality of client machines. Services can be provided efficiently.

Next, a third embodiment of the present invention will be described. FIG. 27 is a block diagram showing a configuration of a video image presentation system according to the third embodiment of the present invention.

The video image presentation system of the present embodiment is determined in order to expand the display part circumscribed rectangle information S6 by the cutout part calculation unit 32 in addition to the configuration of the video image presentation system in the second embodiment shown in FIG. The video image transmitting apparatus 3a is newly provided with an expansion rate changing unit 35 for changing the expansion rate.

The enlargement factor changing unit 35 changes the value of the enlargement factor for enlarging the display part circumscribed rectangle information S6 stored in the cutout part calculating unit 32 in accordance with signals from other modules. When the enlargement ratio is changed, the size of the part indicated by the cutout part information S3 calculated by the cutout part calculation unit 32 is changed. Then, the size of the video image cut out by the block cutout unit 34 is changed.

That is, the size of the video image sent from the video image sending device 3a to the walk-through image presenting device 1a by the enlargement ratio changing unit 35, that is, the amount of data sent from the video image sending device 3a to the walk-through image presenting device 1a. Therefore, it is possible to change the network bandwidth necessary for the video image presentation system.

As an example of the video image presentation system according to the present embodiment, an example in which the enlargement ratio is changed according to the number of walk-through image presentation devices 1a connected to one server machine having the video image transmission device 3a. Is mentioned.

In this embodiment, when the number of walk-through video presentation devices 1a is large, the enlargement ratio is lowered to reduce the network bandwidth used by one session. When the number of sessions is small, the enlargement ratio is increased and a large-sized video image is transmitted to the walk-through image presentation device 1a, so that it can immediately respond to the gaze direction change request received by the gaze direction input unit 12. Like that.

However, even when the enlargement rate is lowered, in order to make the effect of the second embodiment effective, it is preferable to keep the cutout portion larger by reducing the reduction rate of the enlargement rate as much as possible.

As described above, in this embodiment, in addition to the effects of the first and second embodiments, for example, the video image transmission device 3a is operated by providing the enlargement rate changing unit 35 that changes the enlargement rate. The network bandwidth that can be used on the server machine that is running can be calculated at any time, and video images can be distributed with the amount of data communication that matches the bandwidth, and the video image presentation system can be operated in various environments. .

Next, a fourth embodiment of the present invention will be described. FIG. 28 is a block diagram showing a configuration of a video image presentation system according to the fourth embodiment of the present invention.

In the fourth embodiment, in addition to the configuration of the video image presentation system in the third embodiment shown in FIG. 27, the quality of the video image transmitted to the walk-through image presentation device 1b, its processing status, etc. The QoS control unit 18 that requests the enlargement rate changing unit 35 to change the enlargement rate is newly provided.

The following example is given as one example of the video image presentation system according to the present embodiment.

The QoS control unit 18 monitors data delay, video frame loss, etc. for the video image sent from the video image transmission device 3a, and expands when data delay, video frame loss, etc. occur at a certain frequency. It instructs the rate changing unit 35 to reduce the enlargement rate.

As a result, the size of the video image sent from the video image sending device 3a to the walk-through image presenting device 1b is reduced, so that the video image is transmitted and received between the video image sending device 3a and the walk-through image presenting device 1b. The network bandwidth required to do this is reduced. That is, data delay and video frame loss can be prevented.

Moreover, the following example is given as another example of the video image presentation system according to the present embodiment.

The QoS control unit 18 monitors the processing state in the display part cutout unit 17 and instructs the enlargement ratio change unit 35 to reduce the enlargement ratio when the processing in the display part cutout unit 17 falls below the frame rate of the video image. To do.

As a result, the size of the video image transmitted from the video image transmission device 3a to the walk-through image presentation device 1b is reduced, thereby reducing the processing in the display part cutout unit 17. That is, it is possible to prevent a decrease in the frame rate of the presented video image.

As described above, in the video image presentation system according to the present embodiment, in addition to the effects of the first to third embodiments, in the walk-through image presentation device 1b, from the server caused by the fluctuation of the network situation. When a video video delay or frame loss is detected, so-called network QoS (Quality of Service) control for controlling the data communication amount of the video video transmitted for the purpose of mitigating those phenomena becomes possible. That is, it is possible to provide the video image presentation system to the user with quality according to the network state.

Further, even when the processing for the received video image exceeds the processing capability of the walk-through image presentation device 1b, the video image transmission device 3a is instructed to reduce the video image enlargement rate in the same manner, thereby The video can be processed stably.

Next, a fifth embodiment of the present invention will be described. FIG. 29 is a block diagram showing a configuration of a video image presentation system according to the fifth embodiment of the present invention.

The video image presentation system according to the present embodiment includes a walk-through image presentation device 1c that presents a walk-through image to a user, and a video image storage that stores a video image that continuously records the state of moving in a street or a building. Two or more devices that process the video image stored in the device 2 and the video image storage device 2, generate a video image to be displayed on the walk-through image presentation device 1c, and send it to the walk-through image presentation device 1c .., 3-n and the walk-through video presentation device 1c are always established with one virtual connection between the n video video transmission devices 3-1, 3-2,. Video images sent from the video image sending devices 3-1, 3-2,..., 3-n are selectively switched and sent to the walk-through image presenting device 1c through the same connection. A video for instructing the video image switching unit 4 to display a video image to be displayed from the video images read by the n video image transmission units 3-1, 3-2, ..., 3-n. A video determination unit 5 is provided.

The walk-through video presentation device 1c, the video video storage device 2, and the video video transmission devices 3-1, 3-2,..., 3-n of the present embodiment are the first, second, third, and fourth, respectively. Any configuration of the embodiment is applied.

That is, as the walkthrough video presentation device 1c of the present embodiment, the walkthrough video presentation devices 1, 1a, and 1b of the first, second, third, and fourth embodiments can be used. As the video video transmission devices 3-1, 3-2,..., 3-n, the video video transmission devices 3, 3a of the first, second, third, and fourth embodiments can be used. Further, the number n of the video image transmission apparatuses 3-1, 3-2,..., 3-n according to the present embodiment is not particularly limited, and can be implemented by an arbitrary number of two or more.

In addition, video video transmission devices 3-1, 3-2,..., 3-n and a video video storage device 2 are provided on the server machine, and a walk-through video presentation device 1c and a video video determination unit 5 are provided on the client machine. Provided. The video image switching unit 4 is located on the network independently of the server and the client machine.

Next, the operation of the video image presentation system according to the present embodiment will be described with reference to the flowchart of FIG.

First, the video image determination unit 5 determines a plurality of video image candidates to be displayed on the walk-through image presentation device 1c, and names the video images as separate video image transmission devices 3-1, 3-2,. n (step 3001).

Receiving the video image name, the video image transmission devices 3-1, 3-2,..., 3-n read out the corresponding video image from the video image storage device 2 and perform the transmission including the extraction processing in the block extraction unit 34. Preparation is performed (step 3002). The operations in the video image transmitting apparatuses 3-1, 3-2,..., 3-n are performed according to any of the methods described in the first, second, third, and fourth embodiments.

On the other hand, the video image determination unit 5 determines the video image to be displayed first from the video image candidates and transmits it to the video image switching unit 4 (step 3003). The video image switching unit 4 transmits the video image to the walk-through image presenting device 1c according to the instruction from the video image determining unit 5 (step 3004). The walk-through video presentation device 1c presents the transmitted video video to the user (step 3005).

Here, when the video image to be displayed is changed in the video image determination unit 5 (step 3006), the changed video image name is transmitted to the video image switching unit 4. Upon receiving the signal from the video image determination unit 5, the video image switching unit 4 immediately switches the video image to be transmitted to the walk-through image presentation device 1c to the video image and transmits it (step 3007).

Note that, in the video image presentation system according to the present embodiment, the video image after switching is waiting in a state where preparation for transmission has already been completed, so when switching the video image, the video image before and after the switching is connected. Can be sent continuously without interruption.

Further, the video image determination unit 5 determines candidate video images as needed according to the situation at any time, and transmits the video image names to the video image transmission apparatuses 3-1, 3-2,..., 3-n. And instructing the preparation of video transmission. At the same time, for video images that are out of reproduction candidates, the video image transmission devices 3-1, 3-2,.

Next, some specific examples in the present embodiment will be given. First, a first example of the present embodiment will be described.

In this embodiment, the video image determined as a candidate by the video image determination unit 5 is a video image in which the same scene is recorded at different bit rates. Then, in accordance with the network environment connecting the video image switching unit 4 and the walk-through image presentation device 1c, a video image having a bit rate suitable for transmission is selected and displayed.

In this way, in this embodiment, when the available bandwidth of the network is wide, a detailed video image is displayed at a high bit rate, and when the available bandwidth of the network is narrowed, it is immediately reduced. It is possible to display video images by switching to bit rate video images.

Next, a second example of the present embodiment will be described.

The video image determined as a candidate by the video image determination unit 5 is a video image that is moving along the same route but is recorded in a different direction. For example, for the same route, video images that record the front, rear, left, and right are determined as candidates.

For this reason, in this embodiment, the user can move the line-of-sight direction more freely by appropriately switching the video image to be displayed by the user's instruction input.

Next, a third example of the present embodiment will be described.

FIG. 31 is a diagram for explaining a viewpoint movement path in a three-dimensional space. Here, the route 511 is connected to three routes, routes 512, 513, and 514 at the point 515. These video images that record the movement of the paths 511, 512, 513, and 514 are referred to as video images 521, 522, 523, and 524, respectively.

At this time, the video image determination unit 5 determines four videos of video images 521, 522, 523, and 524 as candidates, and each of them is determined as a video image transmission device 3-1, 3-2, 3-3, 3- 4, the video image transmission devices 3-1, 3-2, 3-3 and 3-4 read the video image and prepare for transmission.

When actually presenting the walk-through video to the user, the video video determination unit 5 first determines the reproduction of the video video 521 and transmits it to the video video switching unit 4, and the walk-through video presentation device 1 c uses the video video. 521 is reproduced. That is, a state of walking along the route 511 is presented to the user.

Next, a route to be taken next to the route 511 is selected from the routes 512, 513, and 514 until the reproduction of the video image 521 reaches the point 515.

Here, it is assumed that the path 512 is selected, and the video image to be reproduced next is the video image 522. The video image name determined by the video image determination unit 5 is transmitted to the video image switching unit 4. In the video image switching unit 4, after the video image 521 is reproduced to the last frame, the reproduction of the video image 522 is started immediately.

As described above, in this embodiment, it is possible to present to the user a state in which a plurality of routes are continuously and selectively moved.

As described above, in the video image presentation system according to the present embodiment, by appropriately switching the video image to be displayed, display at an appropriate bit rate, the user can move the line-of-sight direction more freely, A state in which the route is continuously and selectively moved can be presented.

Further, this embodiment is combined with each of the first to fourth embodiments (that is, as walk-through video presentation device 1c and video video transmission devices 3-1, 3-2,..., 3-n). By using those of the first to fourth embodiments), the effects of the first to fourth embodiments can be realized together.

Note that the video image presentation system according to each of the above embodiments includes the video image selection unit 11, the line-of-sight input unit 12, the video image decoding unit 13, the video image display unit 14, and the display part calculation unit 15 in the walk-through image presentation device 1. , The display part storage unit 16, the display part cutout unit 17, the QoS control unit 18, the video video reading unit 31 in the video video transmission devices 3, 3a, 3-1, 3-2,. The functions of the part calculation unit 32, the cutout part storage unit 33, the block cutout unit 34, and the enlargement ratio changing unit 35, the video image switching unit 4, the video image determining unit 5, and other functions are implemented in hardware. Of course, the video processing programs 90, 90a, 90b, 90c, 91, 9 which are computer programs having respective functions are realized. The a, can be implemented by being loaded into the memory of the computer processor. The video processing programs 90, 90a, 90b, 90c, 91, 91a are stored in a magnetic disk, a semiconductor memory, or other recording medium. Each function described above is realized by being loaded from the recording medium into the computer processing apparatus and controlling the operation of the computer processing apparatus.

Although the present invention has been described with reference to the preferred embodiments and examples, the present invention is not necessarily limited to the above-described embodiments and examples, and various modifications can be made within the scope of the technical idea. Can be implemented.

It is a block diagram which shows the structure of the video image presentation system by the 1st Embodiment of this invention. It is a block diagram which shows the network structure to which the video image presentation system by the 1st Embodiment of this invention is applied. It is a flowchart explaining operation | movement of the video image transmission apparatus of the video image presentation system by the 1st Embodiment of this invention. It is a schematic diagram which shows an example of the video image | video of the 1st Embodiment of this invention. It is a schematic diagram which shows the user presentation image | video by the video image | video of FIG. 2 of the 1st Embodiment of this invention. It is a schematic diagram which shows the video image change at the time of the user's eyes | visual_axis movement in the 1st Embodiment of this invention. It is a schematic diagram which shows the video image change at the time of the user's eyes | visual_axis movement in the 1st Embodiment of this invention. It is a schematic diagram which shows the video image change at the time of the user's eyes | visual_axis movement in the 1st Embodiment of this invention. It is a schematic diagram which shows the change of the cut-out site | part in the 1st Embodiment of this invention. It is a block diagram which shows the structure of the video image presentation system by the 2nd Embodiment of this invention. It is a flowchart explaining operation | movement of the video image transmission apparatus in the 2nd Embodiment of this invention. It is a flowchart explaining operation | movement of the walk through image | video presentation apparatus in the 2nd Embodiment of this invention. It is a schematic diagram about the relationship of the gaze direction in the 2nd Embodiment of this invention, display site | part information, display site | part circumscribed rectangle information, and cut-out site | part information. It is a flowchart explaining operation | movement of the display part cutout part in the 2nd Embodiment of this invention. It is a schematic diagram which shows the video image | video and user presentation image | video in the 2nd Embodiment of this invention. It is a schematic diagram which shows the video image | video and user presentation image | video in the 2nd Embodiment of this invention. It is a schematic diagram which shows the video image | video and user presentation image | video in the 2nd Embodiment of this invention. It is a schematic diagram which shows the cutout site | part calculation method in the 2nd Embodiment of this invention. It is a schematic diagram which shows the cutout site | part calculation method in the 2nd Embodiment of this invention. It is a schematic diagram which shows the change of the cut-out site | part in the 2nd Embodiment of this invention. It is a schematic diagram which shows the video image change at the time of the user's eyes | visual_axis movement in the 2nd Embodiment of this invention. It is a schematic diagram which shows the video image change at the time of the user's eyes | visual_axis movement in the 2nd Embodiment of this invention. It is a schematic diagram which shows the video image change at the time of the user's eyes | visual_axis movement in the 2nd Embodiment of this invention. It is a schematic diagram which shows the video image change at the time of the user's eyes | visual_axis movement in the 2nd Embodiment of this invention. It is a schematic diagram which shows the other example of the video image change at the time of a user's eyes | visual_axis movement in the 2nd Embodiment of this invention. It is a schematic diagram which shows the other example of the video image change at the time of a user's eyes | visual_axis movement in the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the video image presentation system by the 3rd Embodiment of this invention. It is a block diagram which shows the structure of the video image presentation system by the 4th Embodiment of this invention. It is a block diagram which shows the structure of the video image presentation system by the 5th Embodiment of this invention. It is a flowchart explaining operation | movement of the video image presentation system by the 5th Embodiment of this invention. It is explanatory drawing of the movement path | route of the viewpoint in the three-dimensional space in the 5th Embodiment of this invention.

Explanation of symbols

1, 1a, 1b, 1c Walk-through video presentation device 11 Video video selection unit 12 Gaze direction input unit 13 Video video decoding unit 14 Video video display unit 15 Display site calculation unit 16 Display site storage unit 17 Display site cutout unit 18 QoS control Unit 2 Video video storage devices 3, 3a, 3-1, 3-2,..., 3-n Video video transmission device 31 Video video reading unit 32 Cutout site calculation unit 33 Cutout site storage unit 34 Block cutout unit 35 Expansion rate change Part 4 Video picture switching part 5 Video picture determination part 90, 90a, 90b, 90c, 91, 91a Video processing program S1 Video picture information S2 Line-of-sight input information S3 Cutout part information S4 Display part information S5 Display area information S6 Display part circumscribed rectangle information

Claims (34)

A video presentation system that presents the captured video image,
And means for accepting an designated by the user of the line-of-sight direction of viewing the video image,
Means for calculating a cutout range corresponding to the line-of-sight direction in the video image;
Means for generating a presentation video is new video images for presentation by Succoth cut out the cut-out area in which the calculated from the video image,
Means for presenting the presented video image;
A video image presentation system comprising: A video image presentation device for presenting video images;
A video image sending device for sending a video image to the video image presenting device;
The video image presentation system according to claim 1, comprising: The video image sending device is:
Means for calculating a cutout range corresponding to the line-of-sight direction in the video image ;
Means for generating a presentation video is new video images for presentation by Succoth cut out the cut-out area in which the calculated from the video image,
Means for sending the presentation video image to the video presentation device;
The video image presentation system according to claim 2, comprising: The video image presentation device includes :
Means for generating a presentation video is new video images for presentation by Succoth cut out the cut-out area in which the calculated from the video image,
Means for presenting the presented video image to the video presenting device;
Comprises, video presentation system according to claim 2. The means for generating the presented video image includes:
Among the blocks divided by size defined screens of the video image, cuts out an area of each of the blocks in the range containing the cutout range in the video image as a new video, claims 1 to 3 The video image presentation system according to any one of the above. A video image presentation system for presenting captured video images,
A video image presentation device for presenting video images;
A video image sending device for sending a video image to the video image presenting device;
With
The video image presentation device includes:
Means for accepting designation of the viewing direction of the user who views the video image;
Means for calculating a first cutout range in the video image corresponding to the line-of-sight direction of the user;
Means for cutting out and presenting a new video image sent from the video image sending device based on the first cutout range;
Have
The video image sending device is:
Means for calculating a second cutout range that is larger than the first cutout range based on the first cutout range;
Means for generating a presentation video image that is a new video image for presentation by cutting out the calculated second cutout range from the video image;
Means for sending the presentation video image to the video presentation device;
A video image presentation system. The video image presentation device includes :
Further comprising a means that receive said second cutout range from the video image delivering apparatus,
The means for cutting out and presenting based on the first cutout range compares the first cutout range with the second cutout range, so that the first cutout range is included in the second cutout range. The presentation video image is cut out based on the first cutout range, and if the first cutout range is not included in the second cutout range, it is included in the second cutout range. Changing the first cutout range to cut out the presentation video image based on the changed first cutout range ,
The video image presentation system according to claim 6. The video image transmitting device changes the size of the image of the presented video image to be transmitted to the video image presenting device to a predetermined enlargement ratio based on the number of the video image presenting devices to which the video image is transmitted. 8. The apparatus according to claim 1, further comprising: a unit that suppresses a communication amount of the presented video image below a communication band between the video image transmitting device and the video image presenting device. Video display system. The video image sending device is
By changing the image size of the video image to be transmitted to the video image presentation device to a predetermined enlargement ratio based on an instruction from the video image presentation device, the video image transmission device, the video image presentation device, The video image presentation system according to claim 8 , further comprising a unit that suppresses a communication amount of the presented video image below a communication band between the two. It said video image presentation device, to said video delivery device comprises means for indicating based on the size of the image of the presentation video image in which the video sending apparatus sends out, the reception state of the presentation video, wherein Item 10. The video image presentation system according to Item 9. The video image presentation device has means for designating a size of an image of the presentation video image transmitted by the video image transmission device to the video image transmission device based on a processing state of the received presentation video image. , video presentation system according to claim 9. Means for sending to the video presentation device, the said video image presentation device from the video transmitting apparatus, via the network sends the presentation video, according to claims 3 to any one of claims 11 Video image presentation system. The video image presentation device further includes compression transmission means for compressing and transmitting the video image to be transmitted ,
The video image presentation device further includes means for decoding and presenting the video image compressed by the compression transmission unit received from the video image presentation device.
The video image presentation system according to claim 12. A plurality of the presented video images are simultaneously received from the video image transmitting device, and each of the presented video images is selectively switched to change the connection between the video image presenting device and the video image presenting device. comprising a switching unit for transmitting without,
Video presentation system according to claim 1 1 or claim 13. The video image is a walkthrough video recorded how moving in streets and buildings, video presentation system according to any one of claims 14 claim 1. The video image, the user is a wide view video image including an image of the range of movement gaze, video presentation system according to claim 15. A video image sending device for sending a video image to a video image presenting device,
Means for calculating a cutout range in the video image corresponding to the line- of-sight direction corresponding to the line-of-sight direction of a user who views the video image notified from the video image presentation device ;
Means for generating a presentation video is new video images for presentation by Succoth cut out the cut-out area in which the calculated from the video image,
A video image sending device comprising: The means for generating the presented video image includes the cutout range calculated corresponding to the line-of-sight direction of the user in the video image in each block obtained by dividing the screen of the video image by a predetermined size. the area of each said block range including to cut out as a new video image,
The video image transmission device according to claim 17. Based on the number of video image presentation device of the destination of the presentation video image, means for changing the size of the image of the presentation video to be sent to the video image presentation device at a predetermined magnification, further comprising, wherein Item 19. A video image transmitting apparatus according to Item 17 or Item 18. The size of the image of the presentation video to be sent to the video image presentation device further comprises means for changing the predetermined magnification based on an instruction from the video image presentation device, according to claim 17 or claim 18 Video video transmission device. Means for compressing and transmitting the video image ;
21. The video image transmission device according to claim 17, wherein the video image transmission device is a video image transmission device. The video image transmission device according to any one of claims 17 to 21 , further comprising means for transmitting the video image to the video image presentation device via a network. A video presentation device that presents a video image to be transmitted from the video transmitting apparatus,
Means for instructing the video image sending device to specify the line-of-sight direction of the user who views the video image ;
Means for receiving and presenting a presentation video image that is a new video image cut out from the video image in a cutout range corresponding to the line-of-sight direction in the video image sending device;
A video image presentation device comprising: Means for calculating a first cutout range in the video image corresponding to the line-of-sight direction of the user;
Receiving a presentation video image that is a new video image cut out in a second cutout range that is larger than the first cutout range calculated by the video image sending device based on the first cutout range; further comprising a clipping means for presenting cut out on the basis of the first cut-out area the presentation video image,
The video image presentation apparatus according to claim 23. The cutout unit receives the first cutout range from the image sending device, and compares the first cutout range with the second cutout range, so that the first cutout range becomes the second cutout range. If it is included in the range, the new video image is cut out based on the first cutout range. If the first cutout range is not included in the second cutout range, the second cutout is performed. Changing the first cutout range to be included in a range, and cutting out the new video image based on the changed first cutout range ;
The video image presentation apparatus according to claim 24. Wherein to the video image delivering apparatus, the size of the image of the presentation video, an instruction to change based on the reception state of the presentation video in a predetermined enlargement ratio means,
Video presentation device according to 請 Motomeko 24 or claim 25 Ru comprising a. The video to the video delivery device, means for performing an instruction to change the size of the image of the presentation video, at a predetermined magnification based on the processing status of the presentation video image received,
Video presentation device according to 請 Motomeko 24 or claim 25 Ru comprising a. Receiving the video image from the video image transmitting device via a network ;
The video image presentation device according to any one of claims 23 to 27. Means for simultaneously receiving a plurality of the presented video images from the video image transmitting device and selectively switching the presented video images ;
Video presentation device according to 請 Motomeko 28 Ru with a. Wherein said presenting video images sent compressed from the video image transmitting apparatus, presented in terms of the decrypted video presentation device according to any one of claims 29 claim 23. To a computer functioning as a video image sending device for sending video images to a video image presentation device,
A process of calculating a cutout range in the video image corresponding to the line-of-sight direction of the user who views the video image notified from the video image presentation device ;
And generating a presentation video is new video images for presentation by Succoth cut out the cut-out area in which the calculated from the video image,
A program that executes In a computer functioning as a video image presentation device for presenting video images transmitted from a video image transmission device ,
A process for instructing the video image sending device to specify the line-of-sight direction of the user who views the video image ;
Processing for receiving and presenting a presentation video image that is a new video image cut out from the video image in a cutout range corresponding to the line-of-sight direction in the video image sending device;
A program that executes Processing to calculate a first cutout range in the video image corresponding to the user's line-of-sight direction to the computer ;
Receiving a presentation video image that is a new video image cut out in a second cutout range that is larger than the first cutout range calculated by the video image sending device based on the first cutout range; Cutout processing for cutting out and presenting the presented video image based on the first cutout range ;
And to further execute, program of claim 32. In the cutout process , the first cutout range is received from the image sending device, and the first cutout range is compared with the second cutout range, so that the first cutout range becomes the second cutout range. If it is included in the range, the new video image is cut out based on the first cutout range. If the first cutout range is not included in the second cutout range, the second cutout is performed. Changing the first cutout range to be included in a range, and cutting out the new video image based on the changed first cutout range ;
Program of claim 33.

Images