JP6389540B2 - Movie data generation device, display system, display control device, and program - Google Patents

Description

  The present invention relates to a moving image data generation device, a display system, a display control device, and a program.

There has been known an HMD (head mounted display) that holds a portable terminal such as a smartphone in a detachable manner and displays an image on the portable terminal (see, for example, Patent Document 1).
[Prior art documents]
[Patent Literature]
[Patent Document 1] JP-A-2006-206512

  There has been a demand for a technique that enables a VR (Virtual Reality) image to be appropriately viewed by the HMD.

  According to the first aspect of the present invention, a moving image data generation device is provided. The moving image data generating device generates moving image data that includes a VR image in a first region in a moving image constituent image, and includes moving image data including a superimposed image that is superimposed on the VR image in a second region other than the first region in the moving image constituent image. A generation unit may be provided. The moving image data generation device may include a moving image data transmission unit that transmits the moving image data generated by the moving image data generation unit.

  The VR image may be an image to be displayed by texture mapping to a sphere, and the superimposed image may be an image to be displayed by mapping to a plane located inside the sphere. The VR image may be an image for display by texture mapping on a sphere, and the superimposed image may be an image for display by texture mapping on a part of the sphere. The VR image may be an image to be displayed by texture mapping onto a sphere, and the superimposed image may be an image to be arranged and displayed on an upper layer of the VR image. The VR image may be an image for display by texture mapping onto a hemisphere.

  The VR image may be a stereoscopic image including a right eye image and a left eye image. The VR image may be a panoramic image. The superimposed image may include a plurality of images. The plurality of images may be a plurality of divided images obtained by dividing one image. The moving image data generation unit arranges the right-eye image and the left-eye image adjacent to each other in the horizontal direction in the first area, and horizontally generates a plurality of divided images obtained by dividing the superimposed image in the second area. The moving image data may be generated by arranging the video data adjacent to each other. The aspect ratio of the moving image constituent image may be different from the aspect ratio of the VR image. The moving image constituent image may have an aspect ratio of 16: 9, and the VR image may have an aspect ratio of 2: 1. The resolution of the moving image constituent image may be 3840 × 2160, and the resolution of the VR image may be 3840 × 1920.

  According to the 2nd aspect of this invention, the program for functioning a computer as said moving image data generation apparatus is provided.

  According to a third aspect of the present invention, there is provided a display system including the moving image data generation device and a display control device that controls to display the moving image data generated by the moving image data generation device.

  According to a fourth aspect of the present invention, a display control device is provided. The display control device receives the moving image data including the VR image in the first region in the moving image composition image and including the superimposed image to be superimposed on the VR image in the second region other than the first region in the moving image composition image. A receiving unit may be provided. The display control apparatus may include a display control unit that performs texture mapping of the VR image on the sphere and controls the superimposed image to be displayed on the VR image.

  The display control unit may texture-map the VR image on a sphere, and may map the superimposed image on a plane located inside the sphere. The display control unit may texture-map the VR image to a sphere, and texture-map the superimposed image to a part of the sphere. The display control unit may texture-map the VR image on a sphere, and may display the superimposed image arranged on an upper layer of the VR image. The display control unit may texture-map the VR image to a hemisphere.

  The display control device maps the superimposed image to a plane located inside the sphere, or maps the superimposed image to a part of the sphere according to an input to the input receiving unit that receives input and the input to the input receiving unit. And a selection unit that selects whether to perform texture mapping or to display the superimposed image arranged on an upper layer of the VR image. The superimposed image may include a plurality of images, and the display control unit may perform control so that the plurality of images are superimposed on the VR image. The superimposed image may be a plurality of divided images obtained by dividing one image, and the display control unit performs control so as to superimpose and display the plurality of divided images divided from the superimposed image on the VR image. It's okay. The display control unit may superimpose and display a combined image obtained by combining the plurality of divided images divided from the superimposed image on the VR image.

  According to the fifth aspect of the present invention, there is provided a program for causing a computer to function as the display control device.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

An example of display system 100 is shown roughly. An example of functional composition of animation data generation device 200 is shown roughly. An example of functional composition of HMD300 is shown roughly. The production | generation process of the moving image structure image 610 by the moving image data generation part 210 is shown roughly. An example of animation data display processing by HMD300 is shown roughly. The production | generation process of the moving image structure image 620 by the moving image data generation part 210 is shown roughly. An example of animation data display processing by HMD300 is shown roughly. An example of the hardware constitutions of the computer 900 which functions as a smart phone with which HMD300 is mounted | worn is shown roughly. An example of the hardware constitutions of the computer 1000 which functions as the moving image data generation apparatus 200 is shown schematically.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 schematically illustrates an example of a display system 100. The display system 100 includes a moving image data generation device 200 and an HMD 300. The moving image data generation device 200 generates moving image data including a VR moving image and transmits the moving image data to the HMD 300 via the network 10. The HMD 300 displays the moving image data received from the moving image data generation device 200 to the user 40 who is a wearer of the HMD 300. The HMD 300 may be an example of a display control device. Here, the type of HMD 300 to which a smartphone is attached will be mainly described as an example.

  The network 10 may be any network. For example, the network 10 includes a telephone network. For example, the network 10 includes the Internet. For example, the network 10 includes a LAN (Local Area Network).

  The VR moving image may be a moving image for displaying on the user 40 as if the user 40 is in a place different from the actual place. For example, the VR video is texture-mapped inside the sphere by the HMD 300, and the HMD 300 assumes that the user 40 is located at the center point of the sphere, and a part of the VR image texture-mapped to the sphere is set to the posture of the user 40. Display accordingly.

  The moving image data generation device 200 may receive the VR moving image captured and generated by the camera system 20 via the network 10. The camera system 20 has one or a plurality of cameras. The camera system 20 has, for example, one or a plurality of fisheye lens cameras.

  The camera system 20 generates, for example, a so-called equirectangular VR video by stitching images taken by a plurality of fisheye lens cameras and performing projective transformation. Even if the camera system 20 transmits images captured by a plurality of fisheye lens cameras to the moving image data generation device 200, the moving image data generation device 200 performs stitching and projective conversion on the images to generate a VR image. Good. In addition, the camera system 20 may acquire, for example, an image with a field angle of 180 degrees or more, which is captured by one fisheye lens camera, as a VR image.

  The moving image data generation device 200 according to the present embodiment generates moving image data including a VR moving image and a superimposed moving image that is superimposed on the VR moving image. The moving image data generation device 200 may generate moving image data including a VR image and a superimposed image in each of a plurality of moving image constituent images. The moving image constituent image is, for example, a frame. The moving image constituent image may be a field.

  The moving image data generation device 200 may include a superimposed moving image related to the content of the VR moving image in the moving image data. For example, the moving image data generation device 200 generates moving image data including a VR moving image captured and generated by a camera arranged in the front row of a sports spectator seat such as baseball and basketball, and a television relay image of the sports. To do. In addition, for example, the moving image data generation device 200 generates moving image data including a VR moving image related to sports and an advertisement image related to the sports.

  The moving image data generation device 200 may receive a superimposed image from the image providing device 30. For example, when the VR image to be displayed is determined, the moving image data generation device 200 provides the superimposed image related to the VR image by requesting the image providing device 30 for the superimposed image related to the VR image. You may receive from the apparatus 30. The image providing device 30 may store arbitrary images such as a television relay image and an advertisement image.

  The HMD 300 superimposes and displays the superimposed image included in the moving image data on the VR moving image included in the moving image data received from the moving image data generation device 200. For example, the HMD 300 captures and displays a television broadcast image of the sport superimposed on the generated VR image, which is taken by a camera arranged in the front row of the sport spectator seat. Accordingly, it is possible to provide the user 40 with a viewing environment in which the user watches the sports in the front row of the spectator seat and also browses the television broadcast on the television at hand. Further, for example, the HMD 300 displays an advertisement image related to the sport superimposed on the generated VR image, which is shot by a camera arranged in the front row of the sport spectator seat. Accordingly, it is possible to provide the user 40 with a browsing environment for browsing advertisements such as goods related to the sport while watching the sport in the front row of the audience seats.

  FIG. 2 schematically illustrates an example of a functional configuration of the moving image data generation device 200. The moving image data generation apparatus 200 includes a VR image storage unit 202, a superimposed image storage unit 204, a VR image acquisition unit 206, a superimposed image acquisition unit 208, a moving image data generation unit 210, and a moving image data transmission unit 212.

  The VR image storage unit 202 stores VR moving images. The VR image storage unit 202 may store a VR moving image generated by the camera system 20. The VR image storage unit 202 may store a VR moving image generated by the moving image data generation device 200 from an image captured by the camera system 20. The VR image storage unit 202 may store a VR image received by the moving image data generation device 200 from an arbitrary device via the network 10. The VR image storage unit 202 may store a VR moving image read from the storage medium by the moving image data generation device 200.

  The VR moving image is a three-dimensional moving image including a right eye image and a left eye image, for example. The VR video may be a panoramic video. A VR video is composed of a plurality of VR images. The VR image is, for example, a stereoscopic image including a right eye image and a left eye image. The VR image may be a panoramic image.

  The superimposed image storage unit 204 stores a superimposed image. The superimposed image storage unit 204 may store the superimposed image received from the image providing device 30. The superimposed image storage unit 204 may store a superimposed image received from an arbitrary device by the moving image data generation device 200 via the network 10. The superimposed image storage unit 204 may store the superimposed image read from the storage medium by the moving image data generation device 200.

  The VR image acquisition unit 206 acquires a VR moving image from the VR image storage unit 202. The VR image acquisition unit 206 may acquire a VR moving image corresponding to the request from the moving image data generation unit 210 from the VR image storage unit 202.

  The superimposed image acquisition unit 208 acquires a superimposed image from the superimposed image storage unit 204. The superimposed image acquisition unit 208 may acquire a superimposed image corresponding to the request from the moving image data generation unit 210 from the superimposed image storage unit 204.

  The moving image data generation unit 210 generates moving image data including the VR image acquired by the VR image acquisition unit 206 and the superimposed image acquired by the superimposed image acquisition unit 208. In response to a request from the HMD 300, the moving image data generation unit 210 may receive a VR image and a superimposed image corresponding to the request from the VR image acquisition unit 206 and the superimposed image acquisition unit 208, and generate moving image data.

  The moving image data transmission unit 212 transmits the moving image data generated by the moving image data generation unit 210. The moving image data transmission unit 212 may transmit moving image data to the HMD 300.

  FIG. 3 schematically shows an example of the functional configuration of the HMD 300. The HMD 300 includes a moving image data receiving unit 302, a moving image data dividing unit 304, a display control unit 306, a display unit 308, an input receiving unit 310, and a selecting unit 312. It is not essential that the HMD 300 has all of these configurations. These configurations may be the configuration of a smartphone attached to the HMD 300.

  The moving image data receiving unit 302 receives the moving image data transmitted by the moving image data transmitting unit 212. For example, the moving image data receiving unit 302 transmits a request received from the user 40 to the moving image data generation device 200 and receives the moving image data transmitted by the moving image data generation device 200 in response to the request.

  The moving image data dividing unit 304 divides the moving image data received by the moving image data receiving unit 302. The moving image data dividing unit 304 may divide moving image data into a VR image and a superimposed image. The moving image data dividing unit 304 may divide the superimposed image into a plurality of divided images.

  The display control unit 306 texture maps the VR image divided by the moving image data division unit 304 onto a sphere and causes the display unit 308 to display the VR image. For example, the display control unit 306 may texture-map the VR image into a hemisphere and display the VR image on the display unit 308.

  The display control unit 306 may superimpose the superimposed image 500 on the VR image and display it on the display unit 308. The display control unit 306 may cause the display unit 308 to display a plurality of divided images obtained by dividing the superimposed image 500 on the VR image. The display control unit 306 may display a plurality of divided images on the display unit 308 by superimposing them on different positions of the VR image. The display control unit 306 may cause the display unit 308 to display a combined image obtained by combining a plurality of divided images obtained by dividing the superimposed image 500 on the VR image.

  The display control unit 306 may map the superimposed image 500 on a plane located inside the sphere obtained by texture mapping the VR image and display the superimposed image 500 on the display unit 308. Thereby, it is possible to provide a display as if the display is floating in the VR space. The display control unit 306 may texture-map the superimposed image 500 on a part of a sphere obtained by texture-mapping a VR image and display the superimposed image 500 on the display unit 308. Thereby, it is possible to provide a display in which a display is fitted in the VR image. The display control unit 306 may arrange the superimposed image 500 on the upper layer of the VR image and display it on the display unit 308. Thereby, it is possible to provide a display in which a display having a form clearly distinguished from the VR image exists.

  The input receiving unit 310 receives an input from the user 40. For example, the input receiving unit 310 maps the superimposed image 500 on a plane located inside the sphere obtained by texture mapping the VR image, or performs texture mapping of the superimposed image 500 on a part of the sphere obtained by texture mapping the VR image, An input for selecting whether to display the superimposed image 500 on the upper layer of the VR image is received.

  The selection unit 312 maps the superimposed image 500 on a plane located inside the sphere on which the VR image is texture-mapped, or the superimposed image on a part of the sphere on which the VR image is texture-mapped, according to an input to the input receiving unit 310. Whether to perform texture mapping 500 or to display the superimposed image 500 on the upper layer of the VR image is selected. The moving image data dividing unit 304 may divide the VR image and the superimposed image 500 from the moving image data in accordance with the selection by the selection unit 312. The display control unit 306 may display the superimposed image 500 on the VR image on the display unit 308 according to the selection by the selection unit 312.

  FIG. 4 schematically shows a generation process of the moving image composition image 610 by the moving image data generation unit 210. Here, a case will be described as an example where a right-eye image 410 and a left-eye image 420 having an aspect ratio of 1: 1 are arranged with respect to a moving image composition image 610 having an aspect ratio of 16: 9. In FIG. 4, as a specific example, the resolution of the moving image constituent image 610 is 3840 × 2160, the resolution of the right-eye image 410 and the left-eye image 420 is 1920 × 1920, and the resolution of the superimposed image 500 is 1280 × 720. Show the case.

  The moving image data generation unit 210 divides the superimposed image 500 acquired by the superimposed image acquisition unit 208, and generates a divided image 510, a divided image 520, and a divided image 530 each having a resolution of 1280 × 240. The moving image data generation unit 210 arranges the right-eye image 410 and the left-eye image 420 adjacent to the first region 612 in the moving image composition image 610 in the horizontal direction. The moving image data generation unit 210 arranges the divided image 510, the divided image 520, and the divided image 530 in the second region 614 other than the first region 612 in the moving image constituent image 610 so as to be adjacent in the horizontal direction.

  Through the above processing, the moving image data generation unit 210 generates the moving image composition image 610. The moving image data generation unit 210 generates moving image data including a plurality of moving image configuration images 610 by performing the same processing on the plurality of moving image configuration images 610. The moving image data generation unit 210 may include additional data indicating the arrangement configuration of the right-eye image 410, the left-eye image 420, the divided image 510, the divided image 520, and the divided image 530 in the moving image constituent image 610 in the moving image data.

  FIG. 5 schematically shows an example of the moving image data display process performed by the HMD 300. Here, a display process when the moving image data receiving unit 302 receives moving image data composed of a plurality of moving image constituent images 610 generated by the moving image data generation device 200 will be described.

  The moving image data dividing unit 304 divides the right-eye image 410, the left-eye image 420, and the superimposed image 500 from the moving image constituent image 610 with reference to the additional data included in the moving image data. The moving image data dividing unit 304 divides the superimposed image 500 into a divided image 510, a divided image 520, and a divided image 530 with reference to the additional data.

  The display control unit 306 generates a combined image 540 obtained by combining the divided image 510, the divided image 520, and the divided image 530 with reference to the additional data. The display control unit 306 texture maps the right eye image 410 to the hemisphere 710 and texture maps the left eye image 420 to the hemisphere 720. In addition, the display control unit 306 superimposes the combined image 540 on the right eye image 410 and the left eye image 420.

  For example, the display control unit 306 maps the combined image 540 on a plane located inside the hemisphere 710 and a plane located inside the hemisphere 720. Further, the display control unit 306 may texture-map the combined image 540 on a part of the hemisphere 710 and a part of the hemisphere 720. In addition, the display control unit 306 may arrange the combined image 540 on the upper layer of the hemisphere 710 and the hemisphere 720.

  The display control unit 306 may cause the display unit 308 to display the right-eye image 410 texture-mapped to the hemisphere 710, the left-eye image 420 texture-mapped to the hemisphere 720, and the combined image 540 superimposed thereon.

  As shown in FIGS. 4 and 5, the display system 100 according to the present embodiment includes a moving image constituent image 610 having an aspect ratio of 16: 9, a right-eye image 410 and a left-eye image having an aspect ratio of 1: 1, respectively. The moving image data is generated by arranging the divided image 510, the divided image 520, and the divided image 530 obtained by dividing the superimposed image 500 in the surplus area. Then, the right-eye image 410 and the left-eye image 420 are texture-mapped to the hemisphere 710 and the hemisphere 720, respectively, and the combined image 540 that combines the divided image 510, the divided image 520, and the divided image 530, and the right-eye image 410 and the left-eye image 420. Superimposed on the display.

  Thus, by performing texture mapping of the right-eye image 410 and the left-eye image 420 to the hemisphere, the respective resolutions can be increased compared to texture mapping to the entire sphere. Further, the resolution can be increased by arranging the right-eye image 410 and the left-eye image 420 having the aspect ratio of 1: 1 in the moving image composing image 610 so that the texture mapping is performed on the hemisphere. Further, by dividing and arranging the superimposed image 500 in the surplus area where the right-eye image 410 and the left-eye image 420 are arranged, the surplus area can be used effectively, and effective use of resources can be realized. .

  Here, the case where the additional data is included in the moving image data has been described as an example, but the present invention is not limited to this. The moving image data generation device 200 and the HMD 300 may grasp the format of the moving image composition image 610 in advance.

  In this example, the right-eye image 410 and the left-eye image 420 are arranged on the upper side of the moving image composition image 610, and the divided image 510, the divided image 520, and the divided image 530 are arranged on the lower side of the moving image composition image 610. Although it has been described, it is not limited to this. Arrangements in the right-eye image 410, the left-eye image 420, the divided image 510, the divided image 520, and the divided image 530 may be arbitrary. For example, the right-eye image 410 and the left-eye image 420 may be disposed below the moving image composition image 610, and the divided image 510, the divided image 520, and the divided image 530 may be disposed above the moving image composition image 610. Further, the number of divisions of the superimposed image 500 is not limited to three and may be other numbers.

  Here, the case where the combined image 540 is a two-dimensional image has been described as an example. However, the present invention is not limited thereto, and the combined image 540 may be a three-dimensional image, that is, a stereoscopic image. For example, the moving image data generation device 200 generates moving image data including divided images obtained by dividing a right eye superimposed image and a left eye superimposed image, and the HMD 300 generates a right eye combined image and a left eye combined image. And the combined image for the right eye may be superimposed on the image for the right eye 410, and the combined image for the left eye may be superimposed on the image for the left eye 420. Accordingly, for example, it is possible to provide a viewing environment in which a stereoscopic image is viewed at hand while watching a sport in the front row of the audience seat.

  FIG. 6 schematically shows a generation process of the moving image composition image 620 by the moving image data generation unit 210. Here, a case where a panoramic image 430 having an aspect ratio of 2: 1 is arranged with respect to a moving image composition image 620 having an aspect ratio of 16: 9 will be described as an example. In FIG. 4, as a specific example, a case where the resolution of the moving image constituent image 620 is 3840 × 2160, the resolution of the panoramic image 430 is 3840 × 1920, and the resolution of the superimposed image 500 is 1280 × 720 is shown.

  The moving image data generation unit 210 divides the superimposed image 500 acquired by the superimposed image acquisition unit 208, and generates a divided image 510, a divided image 520, and a divided image 530 each having a resolution of 1280 × 240. The moving image data generation unit 210 arranges the panoramic image 430 in the first area 622 in the moving image constituent image 620. The moving image data generation unit 210 arranges the divided image 510, the divided image 520, and the divided image 530 in the second region 624 other than the first region 622 in the moving image constituent image 620 adjacent to each other in the horizontal direction.

  Through the above processing, the moving image data generation unit 210 generates the moving image composition image 620. The moving image data generation unit 210 generates moving image data composed of a plurality of moving image composition images 620 by performing the same processing on the plurality of moving image composition images 620. The moving image data generation unit 210 may include additional data indicating the arrangement configuration of the panoramic image 430, the divided image 510, the divided image 520, and the divided image 530 in the moving image constituent image 620 in the moving image data.

  FIG. 7 schematically shows an example of the moving image data display process by the HMD 300. Here, a display process when the moving image data receiving unit 302 receives moving image data composed of a plurality of moving image composition images 620 generated by the moving image data generation device 200 will be described.

  The moving image data dividing unit 304 divides the panoramic image 430 and the superimposed image 500 from the moving image constituent image 620 with reference to the additional data included in the moving image data. The moving image data dividing unit 304 divides the superimposed image 500 into a divided image 510, a divided image 520, and a divided image 530 with reference to the additional data.

  The display control unit 306 generates a combined image 540 obtained by combining the divided image 510, the divided image 520, and the divided image 530 with reference to the additional data. The display control unit 306 texture maps the panoramic image 430 to the whole globe 730. In addition, the display control unit 306 superimposes the combined image 540 on the panoramic image 430.

  For example, the display control unit 306 maps the combined image 540 on a plane located inside the whole sphere 730. Further, the display control unit 306 may texture-map the combined image 540 on a part of the whole globe 730. In addition, the display control unit 306 may arrange the combined image 540 on the upper layer of the whole globe 730.

  The display control unit 306 may cause the display unit 308 to display the panoramic image 430 texture-mapped on the whole globe 730 and the combined image 540 superimposed on the panoramic image 430.

  As shown in FIGS. 6 and 7, the display system 100 according to the present embodiment allows a panoramic image 430 with an aspect ratio of 2: 1 to be accommodated to the maximum extent in a moving image constituent image 620 with an aspect ratio of 16: 9. The moving image data in which the divided image 510, the divided image 520, and the divided image 530 obtained by arranging and dividing the superimposed image 500 in the surplus area are generated is generated. Then, the panorama image 430 is texture-mapped to the whole globe 730, and the divided image 510, the divided image 520, and the combined image obtained by combining the divided images 530 are displayed superimposed on the panoramic image 430.

  In many cases, the aspect ratio of the equirectangular panoramic image is 2: 1. However, according to the display system 100 according to the present embodiment, the moving image constituent image is set so that the equirectangular panoramic image can be accommodated to the maximum extent. By disposing at 620, the resolution of the panoramic image can be increased. Furthermore, by dividing and arranging the superimposed image 500 in the surplus area where the panoramic image is arranged, the surplus area can be used effectively, and effective use of resources can be realized.

  In the above embodiment, the divided image 510 obtained by dividing the superimposed image 500, the divided image 520, the position where the divided image 530 is superimposed, and the position where the divided image 510, the divided image 520, and the combined image 540 obtained by combining the divided images 530 are superimposed. May be determined in any manner. For example, the superposition position may be set by default. Further, it may be determined according to the setting by the user 40. Further, it may be determined by the moving image data generation device 200. In this case, for example, the moving image data generation apparatus 200 transmits the moving image data to the HMD 300 including the information on the superimposed position.

  4 to 7, the case where the aspect ratio of the moving image constituent image is 16: 9 and the aspect ratio of the VR image is 2: 1 has been described as an example. However, the present invention is not limited thereto. Other ratios may be used as long as the aspect ratio of the moving image constituent image is different from the aspect ratio of the VR image.

  FIG. 8 schematically illustrates an example of a hardware configuration of a computer 900 that functions as a smartphone attached to the HMD 300. The computer 900 according to this embodiment includes an SoC 910, a main memory 922, a flash memory 924, an antenna 932, an antenna 934, a display 940, a microphone 942, a speaker 944, a USB port 952, and a card slot 954.

  The SoC 910 operates based on a program stored in the main memory 922 and the flash memory 924, and controls each unit. The antenna 932 is a so-called cellular antenna. The antenna 934 is a so-called WiFi (registered trademark) antenna. The SoC 910 may realize various communication functions using the antenna 932 and the antenna 934. For example, the SoC 910 may receive a program used by the SoC 910 using the antenna 932 or the antenna 934 and store the program in the flash memory 924.

  The SoC 910 may realize various display functions using the display 940. The display 940 may be an example of the display unit 308. The SoC 910 may implement various voice input functions using the microphone 942. The SoC 910 may implement various audio output functions using the speaker 944.

  The USB port 952 implements USB connection. The card slot 954 realizes connection with various cards such as an SD card. The SoC 910 may receive a program used by the SoC 910 from a device or memory connected to the USB port 952 and a card connected to the card slot 954 and store the program in the flash memory 924.

  A program that is installed in the computer 900 and causes the computer 900 to function as a smartphone according to the present embodiment may work on the SoC 910 or the like to cause the computer 900 to function as each unit of the smartphone. The information processing described in these programs is read by the computer 900, whereby the moving image data receiving unit 302, the moving image data dividing unit, which are specific means in which the software and the various hardware resources described above cooperate with each other. 304, functions as a display control unit 306, an input reception unit 310, and a selection unit 312. And the specific smart phone according to the intended use is constructed | assembled by implement | achieving the calculation or processing of the information according to the intended use of the computer 900 in this embodiment by these specific means.

  In the above-described embodiment, the case where the HMD 300 is a type to which a smartphone is attached has been mainly described as an example, but the present invention is not limited thereto. The HMD 300 itself may function as the moving image data receiving unit 302, the moving image data dividing unit 304, the display control unit 306, the display unit 308, the input receiving unit 310, and the selection unit 312.

  FIG. 9 schematically illustrates an exemplary hardware configuration of a computer 1000 that functions as the moving image data generation apparatus 200. The computer 1000 according to this embodiment includes a CPU peripheral unit including a CPU 1010, a RAM 1030, and a graphic controller 1085 that are connected to each other by a host controller 1092; a ROM 1020 that is connected to the host controller 1092 by an input / output controller 1094; An input / output unit having F1040, hard disk drive 1050, DVD drive 1070, and input / output chip 1080 is provided.

  The CPU 1010 operates based on programs stored in the ROM 1020 and the RAM 1030 and controls each unit. The graphic controller 1085 acquires image data generated by the CPU 1010 or the like on a frame buffer provided in the RAM 1030 and displays the image data on the display 1090. Alternatively, the graphic controller 1085 may include a frame buffer that stores image data generated by the CPU 1010 or the like.

  The communication I / F 1040 communicates with other devices via a network by wire or wireless. The communication I / F 1040 functions as hardware that performs communication. The hard disk drive 1050 stores programs and data used by the CPU 1010. The DVD drive 1070 reads a program or data from the DVD-ROM 1072 and provides it to the hard disk drive 1050 via the RAM 1030.

  The ROM 1020 stores a boot program executed when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like. The input / output chip 1080 connects various input / output devices to the input / output controller 1094 via, for example, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

  A program provided to the hard disk drive 1050 via the RAM 1030 is stored in a recording medium such as a DVD-ROM 1072 or an IC card and provided by a user. The program is read from the recording medium, installed in the hard disk drive 1050 via the RAM 1030, and executed by the CPU 1010.

  A program that is installed in the computer 1000 and causes the computer 1000 to function as the moving image data generation device 200 may work on the CPU 1010 or the like to cause the computer 1000 to function as each unit of the moving image data generation device 200. The information processing described in these programs is read by the computer 1000, so that the VR image storage unit 202, the superimposed image storage unit, which are specific means in which the software and the various hardware resources described above cooperate. 204, a VR image acquisition unit 206, a superimposed image acquisition unit 208, a moving image data generation unit 210, and a moving image data transmission unit 212. The specific moving image data generating apparatus 200 corresponding to the purpose of use is constructed by realizing calculation or processing of information according to the purpose of use of the computer 1000 in this embodiment by these specific means.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that it can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first,” “next,” etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

10 network, 20 camera system, 30 image providing device, 100 display system, 200 moving image data generating device, 202 VR image storage unit, 204 superimposed image storage unit, 206 VR image acquisition unit, 208 superimposed image acquisition unit, 210 moving image data generation Unit, 212 video data transmission unit, 300 HMD, 302 video data reception unit, 304 video data division unit, 306 display control unit, 308 display unit, 310 input reception unit, 312 selection unit, 410 right eye image, 420 left eye image 430 panoramic image, 500 superimposed image, 510 divided image, 520 divided image, 530 divided image, 540 combined image, 610 moving image component image, 620 moving image component image, 710 hemisphere, 720 hemisphere, 730 whole ball, 900 computer, 910 SoC, 922 Mei Memory, 924 Flash memory, 932 antenna, 934 antenna, 940 display, 942 microphone, 944 speaker, 952 USB port, 954 card slot, 1000 computer, 1010 CPU, 1020 ROM, 1030 RAM, 1040 communication I / F, 1050 hard disk drive 1070 DVD drive, 1072 DVD-ROM, 1080 input / output chip, 1085 graphic controller, 1090 display, 1092 host controller, 1094 input / output controller

Claims (18)

A moving image data generating unit that generates a moving image data including a VR image in a first region in the moving image constituent image and including a superimposed image to be superimposed on the VR image in a second region other than the first region in the moving image constituent image; ,
A moving image data transmission unit that transmits the moving image data generated by the moving image data generation unit,
The moving image data is composed only of the VR image and the superimposed image,
The VR image is a stereoscopic image including a right-eye image and a left-eye image,
The moving image data generation unit arranges the right-eye image and the left-eye image adjacent to each other in the horizontal direction in the first region, and horizontally generates a plurality of divided images obtained by dividing the superimposed image in the second region. A moving image data generation device that generates the moving image data by arranging the image data adjacent to each other. The VR image is an image for display by texture mapping to a sphere,
The moving image data generation device according to claim 1, wherein the superimposed image is an image to be displayed by mapping to a plane located inside the sphere. The VR image is an image for display by texture mapping to a sphere,
The moving image data generation device according to claim 1, wherein the superimposed image is an image to be displayed by texture mapping onto a part of the sphere.   4. The moving image data generation device according to claim 1, wherein the VR image is an image to be displayed by texture mapping onto a hemisphere. 5.   The moving image data generation device according to claim 4, wherein an aspect ratio of the moving image constituent image is different from an aspect ratio of the VR image. The aspect ratio of the moving image constituent image is 16: 9,
The moving image data generation device according to claim 5, wherein an aspect ratio of the VR image is 2: 1. The resolution of the moving image constituent image is 3840 × 2160,
The moving image data generation device according to claim 5 or 6, wherein the resolution of the VR image is 3840 x 1920. The program for functioning a computer as a moving image data generation apparatus as described in any one of Claim 1 to 7 . The moving image data generation device according to any one of claims 1 to 7 ,
A display control device that controls to display the moving image data generated by the moving image data generation device. A moving picture data receiving unit for receiving moving picture data including a VR image in a first area in the moving picture constituent image and including a superimposed image to be superimposed on the VR image in a second area other than the first area in the moving picture constituent image; ,
A display control unit configured to texture-map the VR image on a sphere and control to display the superimposed image superimposed on the VR image;
The moving image data is composed only of the VR image and the superimposed image,
The VR image is a stereoscopic image including a right-eye image and a left-eye image,
The superimposed image includes a plurality of divided images obtained by dividing one image,
In the first area, the right-eye image and the left-eye image are arranged adjacent to each other in the horizontal direction, and in the second area, the plurality of divided images are arranged adjacent to each other in the horizontal direction. ,
The display control unit is configured to perform texture mapping of the right-eye image and the left-eye image on the sphere, and to control to display the plurality of divided images superimposed on the VR image. The display control apparatus according to claim 10 , wherein the display control unit texture maps the VR image to a sphere and maps the superimposed image to a plane located inside the sphere. The display control apparatus according to claim 10 , wherein the display control unit texture maps the VR image to a sphere and texture maps the superimposed image to a part of the sphere. The display control apparatus according to any one of claims 10 to 12 , wherein the display control unit texture-maps the VR image to a hemisphere. The display control apparatus according to claim 13 , wherein an aspect ratio of the moving image constituent image is different from an aspect ratio of the VR image. The aspect ratio of the moving image constituent image is 16: 9,
The display control apparatus according to claim 14 , wherein an aspect ratio of the VR image is 2: 1. The resolution of the moving image constituent image is 3840 × 2160,
Resolution of the VR image is 3840 × 1920, the display control device according to claim 14 or 15. The display control device according to any one of claims 10 to 16 , wherein the display control unit superimposes and displays a combined image obtained by combining the plurality of divided images divided from the superimposed image on the VR image. A program for causing a computer to function as the display control device according to any one of claims 10 17.

Images