JP6660622B2 - Information processing device, information processing method, camera, receiving device, receiving method - Google Patents

Description

  The present invention relates to an information processing technology, and more particularly to a video technology or a related technology that enables blind spot prevention, high presence, efficient communication, and efficient three-dimensional shape restoration.

  In recent years, monitoring systems have become more intelligent and the area to be monitored has been widened. With the widening of the monitored space, the number of cameras installed tends to increase in order to reduce the blind spot and photograph the whole monitored space.

  Also, in a video distribution system such as a television broadcast, the number of cameras used for shooting tends to increase, especially for shooting powerful scenes in sports broadcasting and the like.

JP 2011-120283 A

Torii et al .: “Instant 3D Restoration by Sequential SfM and Surface Generation”, Proceedings of the 19th Image Sensing Symposium (SSII2013), p. IS 2-33-1-4, June 2013

  However, even in a surveillance system, it is sometimes difficult to completely eliminate blind spots even when a large number of cameras are installed.

  Also, in a video distribution system such as a television broadcast, even if a large number of cameras are installed, it is sometimes difficult to completely eliminate blind spots and shoot a powerful scene without missing.

  To address this problem, studies have been made on a free viewpoint video system that reproduces a video image at an arbitrary position by installing a large number of fixed cameras. For example, Patent Literature 1 discloses an information processing method that does not excessively increase the amount of information processing in a transmission side system when generating an image of an arbitrary viewpoint based on a plurality of camera images.

  By the way, if it is a facility such as a stadium, it is possible to install a large number of fixed cameras in various places, but for an outdoor event, for example, the installation place is limited, and it is difficult to generate a free viewpoint video. .

  Further, the conventional free viewpoint video system is assumed to be viewed by a user outside the shooting location, and a user inside the shooting location cannot receive the benefit of the free viewpoint video.

  This application relates to a video processing device, a video capturing device such as a camera, a video receiving device, and a video receiving device, which are effective in a system that handles a plurality of cameras and a plurality of video data, which are created while considering the solution of the above-described problem. It is intended to provide a method to be performed.

  As an example of a more specific object, an object is to provide an information processing apparatus and an information processing method for preventing a blind spot or prohibiting photographing of a specific area. Further, as another example, it is an object to provide an information processing apparatus and an information processing method for delivering a video at a free or low rate to a user who has transferred a captured video. Further, as another example, it is an object to provide an information processing apparatus and an information processing method that give a sense of realism to a user in a shooting place. Another object is to provide an information processing apparatus and an information processing method for efficiently performing communication within a facility or the entire stadium. Another object is to provide an information processing apparatus and an information processing method for efficiently performing three-dimensional shape restoration.

  Another object of the present invention is to provide a camera, a video receiving device, and a video receiving method that can be connected to the information processing device and the information processing method.

  One embodiment of the present invention that achieves one of the above objects is an information processing device, which includes a storage unit that stores a plurality of pieces of captured video data, and detects a blind spot from the plurality of captured video data stored in the storage unit. And an analysis unit that generates an instruction signal to prevent blind spots and outputs the signal to a camera that generates captured video data.

  According to the above-described invention, it is possible to provide an information processing apparatus that prevents a blind spot by detecting a blind spot and outputting an instruction signal to prevent the blind spot.

FIG. 1 shows a configuration of a video information processing apparatus 100 according to Embodiment 1. FIG. 2 shows a configuration of a video information processing apparatus 101 according to the first embodiment. FIG. 7 is a diagram showing an example of an instruction displayed on a screen of the user camera 151 based on a direction instruction signal in the first embodiment. FIG. 2 shows a configuration of a video information processing apparatus 101 according to the first embodiment. FIG. 2 shows a configuration of a video information processing apparatus 102 according to Embodiment 1. FIG. 2 shows a configuration of a video information processing apparatus 103 according to Embodiment 1. FIG. 2 shows a configuration of a video information processing apparatus 103 according to Embodiment 1. FIG. 2 shows a configuration of a video information processing apparatus 104 according to the first embodiment. FIG. 2 shows a configuration of a video information processing apparatus 105 according to Embodiment 1. FIG. 14 is a diagram illustrating an example of a notification displayed on the screen of the user camera 152 based on a shooting prohibition notification signal in the second embodiment. FIG. 2 shows a configuration of a video information processing apparatus 105 according to Embodiment 1. FIG. 9 shows a configuration of a video information processing apparatus 200 according to the second embodiment. FIG. 9 shows a configuration of a video information processing apparatus 200 according to the second embodiment. FIG. 9 shows a configuration of a video information processing apparatus 201 according to the second embodiment. FIG. 9 shows a configuration of a video information processing apparatus 201 according to the second embodiment. FIG. 14 shows a configuration of a video information processing apparatus 300 according to the third embodiment. FIG. 15 is a diagram illustrating an example of a screen of the user camera 351 when soccer is played in a stadium in the third embodiment. FIG. 14 shows a configuration of a video information processing apparatus 300 according to the third embodiment. FIG. 14 shows a configuration of a video information processing apparatus 301 according to Embodiment 3. In the third embodiment, a diagram showing an example of a screen of the user camera 352 when a soccer game is played in the stadium. FIG. 14 shows a configuration of a video information processing apparatus 301 according to Embodiment 3. FIG. 14 shows a configuration of a video information processing apparatus 302 according to Embodiment 3. In the third embodiment, a diagram showing an example of a screen of the user camera 352 when a soccer game is played in the stadium. FIG. 14 shows a configuration of a video information processing apparatus 302 according to Embodiment 3. 9 shows a configuration of a video information processing apparatus 400 according to Embodiment 4. FIG. FIG. 14 is a diagram illustrating an example of a method for generating viewpoint information in the user camera 451 according to the fourth embodiment. 9 shows a configuration of a video information processing apparatus 400 according to Embodiment 4. FIG. 15 shows a configuration of a video information processing apparatus 500 according to Embodiment 5. FIG. 15 shows a configuration of a video information processing apparatus 500 according to Embodiment 5. FIG. 15 shows a configuration of a video information processing device 501 according to Embodiment 5. FIG. 15 shows a configuration of a video information processing device 501 according to Embodiment 5. FIG. 15 shows a configuration of a video information processing apparatus 600 according to Embodiment 6. FIG. 15 shows a configuration of a video information processing apparatus 600 according to Embodiment 6. FIG. 15 shows a configuration of a video information processing device 601 according to Embodiment 6. FIG. 15 shows a configuration of a video information processing device 601 according to Embodiment 6. FIG. 15 shows a configuration of a video information processing apparatus 602 according to Embodiment 6. FIG. 15 shows a configuration of a video information processing apparatus 602 according to Embodiment 6. FIG. 15 shows a configuration of a video information processing apparatus 603 according to Embodiment 6. FIG. FIG. 15 is a diagram illustrating an example of a notification displayed on the screen of the user camera 652 when the user camera 652 is activated in the sixth embodiment. 15 shows a configuration of a video information processing apparatus 603 according to Embodiment 6. FIG.

  Prior to the description of each embodiment of the present invention, some of the aspects of the present invention included in the following description of the embodiments and the effects thereof will be described below.

  An invention according to an aspect 1 of the present application is an information processing apparatus, comprising: a storage unit that stores a plurality of pieces of captured video data, and detects a blind spot from the plurality of captured video data pieces stored in the storage unit to prevent the blind spot. And an analysis unit that generates an instruction signal and outputs the instruction signal to a camera that generates the photographed video data.

  According to a second aspect of the present invention, in the information processing apparatus according to the first aspect, the analysis unit generates, as the instruction signal, a signal instructing movement of the camera and / or a signal instructing a shooting direction of the camera. It is a feature.

  According to a third aspect of the present invention, in the information processing apparatus according to the first aspect, the information processing apparatus further includes a video distribution unit, and the analysis unit converts a free viewpoint video from the captured video data stored in the storage unit. The storage unit stores the free viewpoint video generated by the analysis unit, and the video distribution unit converts the video information data corresponding to the input visual field information and / or the visual field information into the visual field information and / or Alternatively, the image information is output to a video receiving device that generates viewpoint information, and the analysis unit generates an instruction signal to prevent a blind spot when the video information data output from the video distribution unit has a blind spot. It is characterized by the following.

  The invention according to a fourth aspect of the present invention is a camera, wherein the camera performs movement and / or displays an instruction on a screen based on an instruction signal output from the information processing apparatus according to the first aspect.

  An invention according to a fifth aspect of the present invention is an information processing apparatus, comprising: a storage unit for storing captured video data; and detecting whether or not a shooting prohibited area is captured from the captured video data stored in the storage unit. And an analysis unit that generates a notification signal so as to prohibit shooting when shooting, and outputs the notification signal to a camera that generates the shot video data.

  According to a sixth aspect of the present invention, in the information processing apparatus according to the fifth aspect, the analysis unit generates, as the notification signal, a signal for instructing the camera to prohibit shooting and / or a signal for notifying the camera to prohibit shooting. It is characterized by the following.

  An invention according to a seventh aspect of the present invention is a camera, characterized by stopping shooting and / or displaying a notification content on a screen based on a notification signal output from the information processing apparatus according to the fifth aspect. .

  The invention according to aspect 8 of the present application is an information processing apparatus, comprising: a storage unit that stores captured video data; an analysis unit that generates a free viewpoint video from the captured video data stored in the storage unit; A video management unit that manages charging; the storage unit stores the free viewpoint video generated by the analysis unit; and the video distribution unit stores input visual field information and / or Or, the video information data corresponding to the viewpoint information is read from the storage unit and output to the video receiving device that generates the visual field information and / or the viewpoint information, and the charging management unit includes a camera of the same user as the video receiving device. And charging management of the video information data output from the video distribution unit based on whether or not the captured video data has been transferred.

  The invention according to an aspect 9 of the present application is an information processing apparatus, comprising: a storage unit configured to store captured video data; an analysis unit configured to generate a free viewpoint video from the captured video data stored in the storage unit; And a point management unit that manages points for the camera user that generates the image.

  An invention according to an aspect 10 of the present application is an information processing apparatus, wherein: a storage unit that stores captured video data; an analysis unit that analyzes the captured video data stored in the storage unit; And an additional information generation unit that generates additional information to be superimposed on the captured video data and outputs the additional information to a camera that generates the captured video data.

  An invention according to an eleventh aspect of the present invention is the information processing device according to the tenth aspect, wherein the additional information is advertising information.

  An invention according to an aspect 12 of the present application is a camera, wherein the additional information output from the information processing apparatus according to the aspect 10 is displayed so as to be superimposed on the captured video data.

  An invention according to aspect 13 of the present application is an information processing apparatus, comprising: a storage unit that stores captured video data; an analysis unit that generates a free viewpoint video from the captured video data stored in the storage unit; Having a past video distribution unit to output, the storage unit stores the free viewpoint video generated by the analysis unit, the analysis unit analyzes the captured video data stored in the storage unit, The past video distribution unit reads past video information data from the storage unit based on the analysis result of the analysis unit, and outputs the past video information data to a camera that generates the captured video data.

  An invention according to an aspect 14 of the present application is a camera, characterized by displaying past video information data output from the information processing apparatus according to the aspect 13.

  An invention according to aspect 15 of the present application is an information processing apparatus, comprising: a storage unit that stores captured video data and viewpoint information; an analysis unit that generates a free viewpoint video from the captured video data stored in the storage unit; A video distribution unit that outputs information data, the video distribution unit reads video information data corresponding to the viewpoint information from a storage unit, and outputs the video information data to a camera that generates the captured video data and the viewpoint information. It is a feature.

  An invention according to an aspect 16 of the present application is a camera, characterized by displaying video information data output from the information processing apparatus according to the aspect 15.

  An invention according to aspect 17 of the present application is an information processing apparatus, comprising: a storage unit that stores captured video data; an analysis unit that analyzes the captured video data stored in the storage unit; and a communication speed management that manages a communication speed. Wherein the communication speed management unit generates an instruction signal indicating the communication speed of the camera that generates the captured video data based on the analysis result of the analysis unit, and outputs the instruction signal to the camera. It is assumed that.

  An invention according to an aspect 18 of the present application is the information processing device according to the aspect 17, wherein the information processing device further includes a video distribution unit, and the analysis unit converts a free viewpoint video from the captured video data stored in the storage unit. The storage unit stores the free viewpoint video generated by the analysis unit, and the video distribution unit reads out video information data corresponding to the input visual field information and / or viewpoint information from the storage unit. Outputting the visual field information and / or viewpoint information to a video receiving device that generates the visual field information and / or the viewpoint information. The analysis unit determines the importance of the captured video data based on the visual field information and / or the viewpoint information, and performs the communication speed management. The unit generates the instruction signal based on the importance.

  An invention according to a nineteenth aspect of the present invention is a camera, wherein the camera transfers video image data to the information processing device based on an instruction signal output from the information processing device according to the aspect.

  An invention according to aspect 20 of the present application is an information processing apparatus, comprising: a storage unit that stores captured video data; an analysis unit that generates a free viewpoint video from the captured video data stored in the storage unit; An image delivery unit for outputting the image data, wherein the storage unit saves the free viewpoint image generated by the analysis unit, and the image delivery unit stores image information data corresponding to the input visual field information and / or viewpoint information. Is read from the storage unit and output to the video receiving device that generates the visual field information and / or viewpoint information, and the analysis unit preferentially frees the area indicated by the visual field information and / or viewpoint information. It is characterized by generating a viewpoint video.

  An invention according to a twenty-first aspect of the present application is a video receiving device, which displays video information data output from the information processing device according to the twentieth aspect.

  An invention according to aspect 22 of the present application is an information processing device, wherein a storage unit that stores captured video data, a pre-video storage unit that stores a previously generated free viewpoint video, and the imaging that is stored in the storage unit. An analysis unit for updating and generating a free viewpoint video from the video data and the free viewpoint video stored in the preliminary video storage unit.

  An invention according to an aspect 23 of the present application is an information processing apparatus, comprising: a storage unit configured to store captured video data; and an analysis unit configured to generate a free viewpoint video from the captured video data stored in the storage unit. The unit stores the free viewpoint video generated by the analysis unit, and the analysis unit uses only data at a close time among the captured video data, and / or restricts to only high-quality data or high-quality data. Data is preferentially used.

  An invention according to an aspect 24 of the present application is an information processing apparatus, which includes a storage unit for storing captured video data and camera attribute information, and a free viewpoint video from the captured video data stored in the storage unit and the camera attribute information. An analysis unit for generating the camera attribute information is provided, wherein the camera attribute information is at least one of a shooting position, a shooting angle, a shooting time, and a zoom.

  An invention according to an aspect 25 of the present application is a camera, wherein the camera attribute information used by the information processing apparatus according to the aspect 24 is generated.

  The invention of Aspect 26 of the present application is characterized in that in the camera of Aspect 25, the camera attribute information is generated by matching a shooting position with a certain reference point when the camera is activated.

  The invention according to an aspect 27 of the present application is the camera according to the aspect 25, wherein the photographing position is generated using at least one of GPS, WiFi, and information on a base station as the camera attribute information. It is.

  The invention of Aspect 28 of the present application is characterized in that, in the camera of Aspect 25, the shooting time is generated as the camera attribute information by synchronizing a clock on a communication network with a clock in the camera.

  The invention according to aspect 29 of the present application is an information processing method, comprising: a storage step of storing captured video data; detecting a blind spot from the captured video data stored in the storage step; and outputting an instruction signal to prevent the blind spot. An analysis step of generating and outputting the captured video data to a camera that generates the captured video data is included.

  An invention according to an aspect 30 of the present application is an information processing method, comprising: a storage step of storing captured video data; and detecting whether or not a shooting prohibited area is captured from the captured video data stored in the storage step. And an analysis step of generating a notification signal so as to prohibit shooting when shooting and outputting the notification signal to a camera that generates the shot video data.

  The invention according to aspect 31 of the present application is an information processing method, comprising: a storage step of storing captured video data; an analysis step of generating a free viewpoint video from the captured video data stored in the storage step; A video distribution step for outputting, and a charging management step for managing charging, wherein the storing step stores the free viewpoint video generated in the analyzing step, and the video distribution step includes input visual field information and / or Or, the video information data corresponding to the viewpoint information is read out from the storage step, and is output to a video receiving device that generates the view information and / or the viewpoint information. The charging management step includes a camera of the same user as the video receiving device. The video information data output in the video distribution step based on whether or not the captured video data has been transferred from It is characterized in that to perform billing management.

  An invention according to an aspect 32 of the present application is an information processing method, comprising: a storage step of storing captured video data; an analysis step of generating a free viewpoint video from the captured video data stored in the storage step; And a point managing step of managing points for a user of the camera that generates the image.

  An invention according to an aspect 33 of the present application is an information processing method, comprising: a storage step of storing captured video data; an analysis step of analyzing the captured video data stored in the storage step; and an analysis result of the analysis step. And generating additional information to be superimposed on the captured video data, and outputting the additional information to a camera that generates the captured video data.

  An invention according to an aspect 34 of the present application is an information processing method, comprising: a storage step of storing captured video data; an analysis step of generating a free viewpoint video from the captured video data stored in the storage step; Having a past video distribution step of outputting, the storing step stores the free viewpoint video generated in the analyzing step, the analyzing step analyzes the captured video data stored in the storing step, In the past video distribution step, the past video information data is read from the storage step based on the analysis result of the analysis step, and is output to a camera that generates the photographed video data.

  An invention according to an aspect 35 of the present application is an information processing method, comprising: a storage step of storing captured video data and viewpoint information; an analysis step of generating a free viewpoint video from the captured video data stored in the storage step; A video distribution step of outputting information data, wherein the video distribution step reads out video information data corresponding to the viewpoint information from a storage step, and outputs the read video information data to a camera that generates the captured video data and viewpoint information. It is a feature.

  An invention according to an aspect 36 of the present application is an information processing method, wherein a storing step of storing captured video data, an analyzing step of analyzing the captured video data stored in the storing step, and a communication speed management for managing a communication speed Wherein the communication speed management step generates an instruction signal indicating the communication speed of the camera that generates the captured video data based on the analysis result of the analysis step, and outputs the instruction signal to the camera. It is assumed that.

  An invention according to an aspect 37 of the present application is an information processing method, comprising: a storage step of storing captured video data; an analysis step of generating a free viewpoint video from the captured video data stored in the storage step; An image distribution step for outputting, wherein the storage step stores the free viewpoint image generated in the analysis step, and wherein the image distribution step includes image information data corresponding to the input visual field information and / or viewpoint information. Is read from the storage step, and is output to a video receiving device that generates the visual field information and / or the viewpoint information, and the analyzing step preferentially frees the area indicated by the visual field information and / or the viewpoint information. It is characterized by generating a viewpoint video.

  An invention according to an aspect 38 of the present application is the information processing method, wherein a storage step of storing captured video data, a pre-video storage step of storing a free viewpoint video generated in advance, and the shooting stored in the storage step An analysis step of updating and generating a free viewpoint video from the video data and the free viewpoint video stored in the pre-video storage step.

  The invention according to an aspect 39 of the present application is an information processing method, comprising: a storing step of storing captured video data; and an analyzing step of generating a free viewpoint video from the captured video data stored in the storing step. Storing the free viewpoint video generated in the analysis step, wherein the analysis step uses only data at a close time among the photographed video data, and / or restricts only high-quality data to high-quality data. Data is preferentially used. An invention according to an aspect 40 of the present application is an information processing method, comprising: a storage step of storing captured video data and camera attribute information; and a free viewpoint video from the captured video data stored in the storage step and the camera attribute information. The method further includes an analyzing step of generating the camera attribute information, wherein the camera attribute information is at least one of a shooting position, a shooting angle, a shooting time, and a zoom.

  According to the first aspect of the present invention, it is possible to provide an information processing apparatus for preventing a blind spot by detecting a blind spot and outputting an instruction signal to prevent the blind spot.

  According to the second aspect of the present invention, in the first aspect, the information processing for preventing the blind spot is performed by generating, as the instruction signal, the signal instructing the movement of the camera and / or the signal instructing the photographing direction, particularly in the first aspect An apparatus can be provided.

  According to the third aspect of the present invention, when the video information data output from the video distribution unit has a blind spot, the analyzing unit generates an instruction signal to prevent the blind spot, thereby providing information for preventing the blind spot. A processing device can be provided.

  According to the fourth aspect of the present invention, it is possible to provide a camera that moves based on an instruction signal output from the information processing apparatus according to the first aspect and / or displays an instruction on a screen.

  According to the fifth aspect of the present invention, there is provided an information processing apparatus for prohibiting photographing of a certain area by outputting a notification signal to a camera so as to prohibit photographing by an analysis unit and prohibiting photographing. can do.

  According to the sixth aspect of the present invention, in the fifth aspect, the information processing apparatus that prevents blind spots by generating, as the notification signal, the signal instructing the prohibition of photographing of the camera and / or the signal for notifying in the fifth aspect. Can be provided.

  According to the seventh aspect of the invention, it is possible to provide a camera that stops shooting and / or displays a notification content on a screen based on a notification signal output from the information processing apparatus of the fifth aspect.

  Further, according to the invention of the above aspect 8, the charging management unit performs charging management based on whether or not the user has transferred the captured video, so that different charging is performed for the user who has transferred the captured video and the user who has not. , An information processing apparatus that performs video distribution can be provided.

  Further, according to the ninth aspect of the present invention, it is possible to provide an information processing apparatus that gives points to a user who has transferred a captured video by performing point management by the point management unit.

  According to the tenth aspect of the present invention, there is provided an information processing apparatus that gives a sense of realism to a user in a shooting location by generating additional information by an analysis unit and an additional information generation unit based on captured video data of a user. be able to.

  According to the invention of the above aspect 11, the analysis unit and the additional information generation unit generate the advertisement information based on the photographed video data of the user, so that the information processing can distribute the advertisement information based on the photographed video data of the user An apparatus can be provided.

  According to the twelfth aspect of the present invention, it is possible to provide a camera that displays additional information output from the information processing apparatus according to the tenth aspect in a manner superimposed on captured video data.

  Further, according to the invention of the thirteenth aspect, it is possible to provide an information processing apparatus that gives a sense of realism to a user in a shooting location by distributing a past video based on captured video data of the user.

  According to the fourteenth aspect, it is possible to provide a camera that displays past video information data output from the information processing apparatus according to the thirteenth aspect.

  Further, according to the invention of the above aspect 15, it is possible to provide an information processing apparatus that provides a sense of reality to a user in a facility or a stadium by providing a video from a favorite viewpoint.

  According to the sixteenth aspect, it is possible to provide a camera that displays video information data output from the information processing apparatus according to the fifteenth aspect.

  Further, according to the invention described in the above aspect 17, it is possible to provide an information processing apparatus for efficiently performing communication within the facility or the entire stadium.

  Further, according to the invention described in the above aspect 18, the analysis unit determines the importance of the captured video data, and instructs the communication speed based on the importance, thereby efficiently communicating information in the facility or the entire stadium. A processing device can be provided.

  According to the nineteenth aspect, it is possible to provide a camera that transfers captured video data to the information processing apparatus based on an instruction signal output from the information processing apparatus according to the seventeenth aspect.

  Further, according to the invention of the above aspect 20, the analysis unit preferentially generates a free viewpoint video for an area indicated by the visual field information and / or the viewpoint information input from the video receiving device, thereby restoring the three-dimensional shape. An information processing device that performs the above efficiently can be provided.

  Further, according to the invention of the above-mentioned aspect 21, it is possible to provide a video receiving apparatus which displays the video information data outputted from the information processing apparatus of the aspect 20.

  Further, according to the invention of the above-mentioned aspect 22, it is possible to provide an information processing device that efficiently restores a three-dimensional shape by using a free viewpoint video previously stored in the preliminary video storage unit.

  According to the twenty-third aspect of the present invention, the analysis unit uses only the data at a close time among the captured video data and / or restricts only to the high-quality data or preferentially uses the high-quality data, thereby achieving 3 It is possible to provide an information processing device that efficiently restores a dimensional shape.

  Further, according to the invention of the above-mentioned aspect 24, it is possible to provide an information processing apparatus that efficiently performs three-dimensional shape restoration by using the camera attribute information by the analysis unit.

  According to the invention of the above aspect 25, it is possible to provide a camera for generating camera attribute information used by the information processing apparatus of the aspect 24.

  Further, according to the invention of the twenty-sixth aspect, it is possible to provide a camera generated as the camera attribute information used by the information processing apparatus of the twenty-fourth aspect by adjusting a shooting position to a certain reference point when the camera is activated.

  According to the invention of the above-mentioned aspect 27, there is provided a camera which generates, as the camera attribute information used by the information processing apparatus of the aspect 24, at least one of GPS, WiFi, and information on a base station. can do.

  Further, according to the invention of the twenty-eighth aspect, it is possible to provide a camera which generates, as camera attribute information used by the information processing apparatus of the twenty-fourth aspect, a shooting time by synchronizing a clock on a communication network with a clock in the camera. .

  Further, according to the invention of the above-mentioned aspect 29, it is possible to provide an information processing method for preventing a blind spot by detecting a blind spot in the analyzing step and outputting an instruction signal to prevent the blind spot.

  Further, according to the invention of the above aspect 30, an information processing method is provided in which the analysis step determines the prohibition of photographing and outputs a notification signal to the camera to prohibit the photographing, thereby prohibiting the photographing of a certain area. can do.

  According to the invention of the above-mentioned aspect 31, the charging management step performs the charging management based on whether or not the user has transferred the captured video, so that different charging is performed for the user who has transmitted the captured video and the user who has not. Can provide an information processing method for performing video distribution.

  Further, according to the invention of the above aspect 32, the point management step performs the point management, thereby providing an information processing method for giving points to the user who has transferred the captured video.

  Further, according to the invention of the above-mentioned aspect 33, there is provided an information processing method for giving a sense of reality to a user in a shooting place by generating additional information in an analysis step and an additional information generation step based on photographed video data of a user. be able to.

  Further, according to the invention of the above-mentioned aspect 34, it is possible to provide an information processing method of giving a sense of reality to a user in a shooting place by distributing a past video based on the shot video data of the user.

  Further, according to the invention of the above-mentioned aspect 35, it is possible to provide an information processing method for giving a sense of realism to a user in a facility or a stadium by providing an image from a favorite viewpoint.

  Further, according to the invention described in the above-mentioned aspect 36, it is possible to provide an information processing method for efficiently performing communication within the facility or the entire stadium.

  Further, according to the invention described in the above-mentioned aspect 37, it is possible to provide an information processing method for efficiently performing communication within the facility or the entire stadium.

  Further, according to the invention of the above-mentioned aspect 38, it is possible to provide an information processing method for efficiently performing the three-dimensional shape restoration by using the free viewpoint video previously stored in the preliminary video storage step.

  Further, according to the invention of the above-mentioned aspect 39, the analysis step uses only the data at a close time among the photographed video data and / or restricts only the high-quality data or preferentially uses the high-quality data. An information processing method for efficiently restoring a dimensional shape can be provided.

  Further, according to the invention of the above-mentioned aspect 40, it is possible to provide an information processing method for efficiently performing the three-dimensional shape restoration by using the camera attribute information in the analysis step.

  Hereinafter, each embodiment of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a diagram showing a configuration of a video information processing apparatus 100 according to Embodiment 1 of the present invention. In the present embodiment, a description will be given of a video information processing apparatus that prevents the occurrence of blind spots. Also, a video information processing apparatus that prohibits shooting of a specific area will be described. The video information processing apparatus according to the present embodiment can be applied to a monitoring system, a video distribution system, and the like.

  The video information processing apparatus 100 illustrated in FIG. 1 includes a storage unit 131 and an analysis unit 141. Each of the N cameras 111 and the M mobile cameras 121 has a function of capturing an image and a function of transferring captured image data to the image information processing apparatus 100.

  Although the functional block diagrams of the camera 111 and the mobile camera 121 are not shown, the camera 111 and the mobile camera 121 are provided with an image capturing unit that captures an image, and video data generated from the image captured by the image capturing unit. And a transmission unit for transmitting to the device 100. Further, the camera 111 and the moving camera 121 may include a display unit that displays an image being captured. It should be noted that the video data generated by the camera 111 and the moving camera 121 is obtained by converting a video signal captured by a capturing unit into an HEVC or H.264 video signal. The data may be data encoded using an encoding method such as H.264, or may be unencoded data. When the video data is encoded data, the camera 111 and the moving camera 121 further include an encoder that generates video data by encoding video captured by the capturing unit and outputs the video data to the transmission unit. The mobile camera 121 also receives a movement instruction signal transmitted from the video information processing apparatus 100, and changes a position and a shooting direction of the mobile camera itself according to the movement instruction signal received by the reception unit. And a control unit for performing such control. The control unit may have a function of transmitting video data, receiving a movement instruction signal, and controlling a screen or the like displayed on the display unit.

  The storage unit 131 of the video information processing apparatus 100 stores the captured video data when the captured video data of the N cameras 111 and the M mobile cameras 121 is input.

  The analysis unit 141 detects a blind spot from the video data stored in the storage unit 131 and outputs a movement instruction signal to the mobile camera 121 so as to prevent the blind spot. The moving camera 121 moves according to the movement instruction signal, and continues shooting. The mobile camera 121 is, for example, a camera mounted on an unmanned flight type radio control, and is not limited to this as long as it can move a place and / or change a shooting direction.

  The analysis unit 141 can perform blind spot detection using, for example, SfM (Structure from Motion). SfM is a technique for restoring the three-dimensional shape of a subject from a plurality of videos captured from different positions, and is widely known as shape restoration from a motion that simultaneously estimates the subject shape and the camera position. For example, Non-Patent Document 1 proposes SfM suitable for sequential processing. Since SfM is described in detail in Non-Patent Document 1 and the like, description thereof is omitted here. The analysis unit 141 may use the SfM to restore the three-dimensional shape in the facility or the stadium from the video data stored in the storage unit 131, thereby detecting an unrecoverable area as a blind spot.

  When the position and the shooting direction of the camera 111 are fixed and the information on the position and the shooting direction is known, the SfM may be performed using the known information. When the position and the shooting direction of the mobile camera 121 can be acquired by the GPS, the angle sensor, and the like provided in the mobile camera 121, the information on the position and the shooting direction of the mobile camera 121 is transmitted to the analysis unit 141 and transmitted. SfM may be performed using the information on the set position and the shooting direction.

  The method of detecting the blind spot is not limited to the method using SfM described above. For example, any information that can detect the shooting area of each camera, such as detecting information such as the position of the moving camera, the shooting direction, and the zoom (or the shooting area included in the video data) from a marker disposed in the space to be shot. The blind spot can be detected by using the method described above. Alternatively, information such as a positional relationship between a plurality of imaging targets may be acquired from video data or the like, and an area in which a blind spot is likely to occur may be specified based on the acquired positional relationship. Such a method may be able to realize the blind spot detection with a smaller calculation amount when the calculation amount of the SfM is large or a delay occurs in the detection of the blind spot using SfM for other reasons. Useful in that respect.

  In addition, the blind spot may include not only a portion where an image does not exist in a region to be photographed but also a portion where image quality is lower than other portions, a portion where a predetermined image quality is not obtained, and the like. What kind of area is detected as a blind spot may be set according to the configuration or purpose of a monitoring system or a video distribution system, and an area identified by a reference different from the above example is detected as a blind spot. You may. For example, the required image quality may be set high for a specific subject in the space where the image is taken. Conversely, for a specific area in the shooting space, the required image quality may be set low, or the setting may be made so that a blind spot is not determined even when no video is shot.

  The image quality described above includes various information related to the video such as the area (for example, the number of pixels) occupied by the subject to be captured in the video and whether the subject to be captured is in focus. It is sufficient to judge whether or not it is a blind spot based on the information and the combination thereof.

  In the above description, the detection of the area that is actually a blind spot has been described. However, the area that needs to be detected in order to prevent the occurrence of a blind spot is not limited to the area that is actually a blind spot. For example, when there are a plurality of imaging targets and at least a part thereof is moving, a new blind spot may be generated when another imaging target enters between a certain imaging target and a camera. On the other hand, for example, the movement of a plurality of photographing targets is detected from photographed video data and the like, and a new blind spot is formed based on the detected movements of the plurality of photographing targets and the positional information of the camera 111 and the moving camera 121. It is also possible to estimate possible regions. In this case, the video information processing apparatus may be configured to transmit a movement instruction signal to the mobile camera 121 so as to capture an image of a region that may be a blind spot, thereby preventing occurrence of a blind spot.

  When there are a plurality of mobile cameras 121, it is necessary to select a mobile camera 121 that transmits a movement instruction signal in order to photograph a blind spot or an area that may be a blind spot. Further, when there are a plurality of mobile cameras 121 and a plurality of areas that may be blind spots or blind spots, each of the plurality of mobile cameras 121 determines which blind spot or area that may be blind spots is to be photographed. You need to decide. This determination may be made based on, for example, a blind spot or a blind spot area and a position of the area where each moving camera 121 is capturing an image. Good. Also, for each mobile camera 121, it is determined whether or not a new blind spot occurs when the video data currently being captured by the mobile camera 121 cannot be obtained. The mobile camera 121 that has been determined not to generate a blind spot may be selected.

  With the above configuration, the analysis unit detects the blind spot and outputs a movement instruction signal to the mobile camera so as to prevent the blind spot, thereby preventing the occurrence of the blind spot.

  An information processing apparatus according to one aspect of the present invention detects, from video data captured by a plurality of capturing devices, an area that is determined not to have been captured or has a low quality of a captured video, and detects the area. The indicated information is transmitted to a plurality of photographing devices or a photographing device different from the plurality of photographing devices. According to this configuration, the area where the imaging data is insufficient in the area to be imaged is transmitted to the imaging device, so that the imaging device can know the area where the imaging data is insufficient.

  <Modification-1>

  Next, a first modification of the first embodiment will be described with reference to FIG. The video information processing apparatus 101 shown in FIG. 2 is different from the video information processing apparatus 100 shown in FIG. 1 in that the analysis unit 141 is replaced with an analysis unit 142. The video information processing apparatus 101 receives, in addition to the video data captured by the N cameras, video data captured by the L user cameras. Each of the L user cameras 151 has a function of capturing a video and a function of transferring captured video data to the video information processing apparatus 101.

  Although a functional block diagram of the user camera 151 is not shown, the user camera 151 transmits an image capturing unit that captures an image and image data generated from the image captured by the image capturing unit to the image information processing apparatus 101. Unit. Further, the user camera 151 may include a display unit that displays an image being captured. Note that the video data generated by the user camera 151 is obtained by converting a video signal shot by the shooting unit into an HEVC or H.264 video signal. The data may be data encoded using an encoding method such as H.264, or may be unencoded data. When the video data is encoded data, the user camera 151 further includes an encoder that generates video data by encoding video captured by the capturing unit and outputs the video data to the transmission unit. Further, the user camera 151 controls the receiving unit that receives the direction instruction signal transmitted from the video information processing apparatus 101, and displays the instruction image on the display unit according to the direction instruction signal received by the receiving unit. A control unit is provided.

  When the captured video data of the N cameras 111 and the L user cameras 151 is input, the storage unit 131 stores the captured video data. The analysis unit 142 detects a blind spot in the shooting space from the video data stored in the storage unit 131, and outputs a direction instruction signal to the user camera 151 so as to prevent occurrence of a blind spot.

  FIG. 3 is an example of an instruction image displayed on the screen of the user camera 151 based on the direction instruction signal. FIG. 3A is an example of an image indicating a direction in which the user wants the user to point the camera, and FIG. 3B is another example of an image indicating the direction in which the user is desired to move. The instruction displayed on the screen of the user camera 151 is not limited to this, and may be, for example, indicating a moving route on a displayed map. The user camera 151 is, for example, a smartphone, a tablet terminal, a glasses-type wearable terminal, or an HMD (Head Mounted Display) possessed by a spectator in the stadium or a security guard in the facility, and is not limited to this as long as it can capture an image. .

  The display terminal on which the instruction image is displayed based on the direction instruction signal does not need to be the same as the camera that captures the video data. For example, the camera may transmit a direction instruction signal or an instruction image to another display terminal associated in advance with the camera that captures the video, and the instruction image may be displayed on the other display terminal. . If the information of the display terminal corresponding to the camera that captures the video is registered in advance in the video information processing apparatus 101, the video information processing apparatus 101 sends a direction instruction signal directly to the display terminal corresponding to the camera. The instruction image may be transmitted and displayed on the display terminal.

  The analysis unit 142 performs blind spot detection using, for example, SfM. The analysis unit 142 uses SfM to restore the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131. Since the position and orientation of the user camera 151 that has captured the video data have been estimated by the SfM processing, the user camera that is capturing an area near the blind spot is identified, and the user camera is identified with respect to the identified user camera 151. The direction indication signal can be transmitted based on the area being photographed and the blind spot. If the video data captured by the user camera 151 is not used in the SfM processing, the restored three-dimensional shape is matched with the captured video of the user camera 151, so that the user camera 151 It is also possible to determine which area in the stadium is being photographed and output a direction instruction signal. Note that the method of determining the photographing area and the method of generating the direction instruction signal are not limited thereto.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method that detect a blind spot and output a direction instruction signal to a user camera so as to prevent the blind spot, thereby preventing the blind spot. Can be.

  Further, as shown in FIG. 4, the video information processing apparatus 101 can be applied to a case where captured video data is input only from the user camera 151. Further, the video information processing apparatus 101 has a function of receiving video data from the mobile camera shown in FIG. 1 and transmitting a movement instruction signal to the mobile camera in addition to the user camera 151 and the camera 111. Is also good.

  <Variation 2>

  FIG. 5 is a diagram illustrating a second modification of the first embodiment. The video information processing apparatus 102 shown in FIG. 5 is different from the video information processing apparatus 100 shown in FIG. 1 in that the analysis unit 141 is replaced with an analysis unit 143 and a video distribution unit 144 is added.

  The analysis unit 143 restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 using, for example, SfM, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 143. The video distribution unit 144 reads out video information data corresponding to the view information (and / or viewpoint information) input from the video reception device 161 from the storage unit 131, and outputs the video information data to the video reception device 161. In addition, the video distribution unit 144 outputs the visual field information (and / or the viewpoint information) to the analysis unit 143.

  The analysis unit 143 detects the blind spot, and outputs a movement instruction signal to the mobile camera 121 so as to prevent the blind spot when the delivery video based on the visual field information (and / or the viewpoint information) has the blind spot. The moving camera 121 moves according to the movement instruction signal, and continues shooting.

  With the above configuration, the image processing apparatus detects the blind spot by the analysis unit and outputs a movement instruction signal to the moving camera so as to prevent the blind spot of the video delivered to the video receiving apparatus, thereby preventing the blind spot. , A video information processing method can be provided.

  <Modification Example-3>

  Note that the video information processing apparatus 101 shown in FIG. 2 may be replaced with the video information processing apparatus 103 shown in FIG. The video information processing apparatus 103 shown in FIG. 6 is different from the video information processing apparatus 101 shown in FIG. 2 in that the analysis unit 142 is replaced with an analysis unit 145 and a video distribution unit 144 is added.

  The analysis unit 145 restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 using, for example, SfM, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 145. The video distribution unit 144 reads out video information data corresponding to the view information (and / or viewpoint information) input from the video reception device 161 from the storage unit 131, and outputs the video information data to the video reception device 161. In addition, the video distribution unit 144 outputs the visual field information (and / or the viewpoint information) to the analysis unit 145.

  The analysis unit 145 detects the blind spot, and outputs a direction instruction signal to the user camera 151 so as to prevent the blind spot when the delivery video based on the view information (and / or the viewpoint information) has the blind spot. FIG. 3 is an example of an instruction displayed on the screen of the user camera 151 based on the direction instruction signal.

  With the above configuration, the image processing apparatus detects the blind spot by the analysis unit and outputs a direction instruction signal to the user camera so as to prevent the blind spot of the video delivered to the video receiving apparatus, thereby preventing the blind spot. , A video information processing method can be provided.

  Also, as shown in FIG. 7, the video information processing apparatus 103 can be applied to a case where captured video data is input only from the user camera 151.

  <Modification-4>

  Note that the video information processing apparatus 100 shown in FIG. 1 may be replaced with the video information processing apparatus 104 shown in FIG. The video information processing apparatus 104 shown in FIG. 8 is different from the video information processing apparatus 100 shown in FIG. 1 in that the analysis unit 141 is replaced by an analysis unit 146.

  When the captured video data of the N cameras 111 and the M mobile cameras 122 are input, the storage unit 131 stores the captured video data. The analysis unit 146 analyzes the video data stored in the storage unit 131 and outputs a shooting prohibition notification signal to the mobile camera 122 when the mobile camera 122 is shooting a shooting prohibited area. The mobile camera 122 stops shooting while receiving the shooting prohibition notification signal.

  The analysis unit 146 may determine, for example, whether or not the shooting prohibited area is shot using SfM. The analysis unit 146 restores the three-dimensional shape in the facility or the stadium from the video data stored in the storage unit 131 using SfM and matches the captured image with the video captured by the mobile camera 122 so that the mobile camera 122 It may be determined whether or not the shooting prohibited area is being shot, and the shooting prohibited notification signal may be output. Note that the method of determining whether the shooting prohibited area is being shot and the method of generating the shooting prohibition notification signal are not limited thereto. Examples of the shooting prohibited area include a toilet in a facility or a stadium.

  With the above-described configuration, a video information processing device that prohibits shooting of a certain area by outputting a shooting prohibition notification signal to a moving camera so that the analysis unit determines shooting prohibition and stops shooting, A processing method can be provided.

  <Modification-5>

  Note that the video information processing device 104 shown in FIG. 8 may be replaced with the video information processing device 105 shown in FIG. The video information processing apparatus 105 shown in FIG. 9 is different from the video information processing apparatus 104 shown in FIG. 8 in that the analysis unit 146 is replaced by an analysis unit 147.

  When the captured video data of the N cameras 111 and the L user cameras 152 is input, the storage unit 131 stores the captured video data. The analysis unit 147 analyzes the video data stored in the storage unit 131, and outputs a shooting prohibition notification signal to the user camera 152 when the user camera 152 is shooting a shooting prohibited area.

  FIG. 10 is an example of a notification displayed on the screen of the user camera 152 based on the shooting prohibition notification signal. The instruction displayed on the screen of the user camera 152 is not limited to this, and may be, for example, indicating a shooting prohibited area and a current shooting area on a displayed map.

  With the above configuration, the analysis unit determines whether or not shooting is prohibited, and outputs a shooting prohibition notification signal to the user camera so as to stop the shooting, thereby prohibiting shooting of a certain area. A processing method can be provided.

  Also, as shown in FIG. 11, the video information processing apparatus 105 can be applied to a case where captured video data is input only from the user camera 152.

  (Embodiment 2)

  FIG. 12 is a diagram showing a configuration of a video information processing device 200 according to Embodiment 2 of the present invention. The same components as those of the video information processing apparatus according to the first embodiment are denoted by the same reference numerals, and description thereof is omitted.

  In the present embodiment, a description will be given of a video information processing apparatus that distributes a video at a free or low fee to a user who has transferred a captured video. Also, a description will be given of a video information processing apparatus that gives points to a user who has transferred a captured video and gives particularly high points to a user who has transferred a captured video in a field of view (and / or viewpoint) with a large number of requests. I do. The video information processing apparatus according to the present embodiment is particularly applicable to a video distribution system and the like.

  The video information processing apparatus 200 shown in FIG. 12 is different from the video information processing apparatus 100 shown in FIG. 6 in that the analysis unit 145 and the video distribution unit 144 are replaced with the analysis unit 241 and the video distribution unit 242, respectively, and the charging management unit 243 is replaced. This is the added configuration.

  When photographed video data and user information are input from the L user cameras 251, the storage unit 131 stores them. When photographed video data is input from the N cameras 111, the storage unit 131 stores them.

  The analysis unit 241 restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 using, for example, SfM, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 241. The video distribution unit 242 reads video information data corresponding to the view information (and / or viewpoint information) input from the video reception device 161 from the storage unit 131 and outputs the video information data to the video reception device 161.

  When the captured video is transferred from the user camera 251 of the same user as the video receiving device 161, the charging management unit 243 distributes the video information data free of charge. Alternatively, the video information data may be distributed at a lower rate than the user who has not transferred the captured video. Note that the function of the video receiving device 161 may be included in the user camera 251.

  With the above configuration, the charging management unit performs charging management based on whether or not the user has transferred the captured video, and the video information for delivering the video to the user who has transferred the captured video at a free or low rate. A processing device and a video information processing method can be provided.

  Also, as shown in FIG. 13, the video information processing apparatus 200 can be applied to a case where captured video data is input only from the user camera 251.

  <Modification-1>

  FIG. 14 is a diagram illustrating a first modification of the second embodiment. The video information processing apparatus 201 shown in FIG. 14 is different from the video information processing apparatus 200 shown in FIG. 12 in that the charging management section 243 is replaced with a point management section 244.

  The point management unit 244 gives points to the user who has transferred the captured video, and gives particularly high points to the user who has transferred the captured video of a field of view (and / or viewpoint) with a large number of requests. Alternatively, points may be given only to a user who has transferred a photographed video having a field of view (and / or viewpoint) with a large number of requests. The user can use the given points to receive the paid distribution free of charge with the video receiving device 161. The method of using the given points is not limited to this.

  With the above configuration, the point management unit performs the point management, so that it is possible to provide a video information processing apparatus and a video information processing method for giving a point to a user who has transferred a captured video.

  Also, as shown in FIG. 15, the video information processing apparatus 201 can be applied to a case where captured video data is input only from the user camera 251.

(Embodiment 3)
FIG. 16 is a diagram showing a configuration of a video information processing device 300 according to Embodiment 3 of the present invention. The same components as those of the video information processing apparatus described in the first and second embodiments are denoted by the same reference numerals, and description thereof is omitted. In the present embodiment, a description will be given of a video information processing apparatus which provides a user in a shooting location with additional information and gives a sense of realism. The video information processing apparatus according to the present embodiment can be applied to a monitoring system, a video distribution system, and the like.

  The video information processing apparatus 300 shown in FIG. 16 is different from the video information processing apparatus 101 shown in FIG. 2 in that the analysis unit 142 is replaced with an analysis unit 341 and an additional information generation unit 342 is added.

  When the captured video data of the N cameras 111 and the L user cameras 351 is input, the storage unit 131 stores the captured video data.

  The analysis unit 341 restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 using, for example, SfM, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 341. In addition, the analysis unit 341 may determine which area in the facility or stadium the user camera 351 is capturing by matching the free viewpoint video with the video captured by the user camera 351. Note that the method of determining a shooting region is not limited to this, and the various methods of determining a shooting region described in the above embodiments and other methods of determining a shooting region can be used.

  The additional information generation unit 342 outputs additional information to the user camera 351 based on the analysis result of the analysis unit 341. The user camera 351 superimposes the additional information on the captured video and displays it on the screen.

  FIG. 17 is an example of a screen of the user camera 351 when soccer is played in the stadium. FIG. 17A shows an example of a screen before superimposing additional information, and FIG. 17B shows an example of a screen after superimposing additional information. An example is shown in which a player name is superimposed as additional information. A screen viewed by a user in the stadium using the user camera 351 is transferred as captured video data to the storage unit 131 of the video information processing apparatus 300, and at the same time, the analysis unit 341 and the additional information generation are performed based on the captured video data. Since the additional information generated by the unit 342 is superimposed on the screen viewed by the user, it is possible to give the user in the shooting location a sense of realism. In particular, when the user camera 351 is a glasses-type wearable terminal or an HMD, the user can feel great realism. The player name is an example of the additional information, and is not limited to this. Further, not only in the video distribution system but also in the surveillance system, by superimposing, for example, information of a person of interest on the user camera 351 possessed by the security guard in the facility, the accuracy of the surveillance system can be improved.

  With the above-described configuration, an analysis unit and an additional information generation unit generate additional information based on captured video data of a user, thereby providing a video information processing apparatus and a video information processing method for giving a sense of reality to a user in a shooting location. can do.

  Also, as shown in FIG. 18, the video information processing apparatus 300 can be applied to a case where captured video data is input only from the user camera 351.

  <Modification-1>

  FIG. 19 is a diagram illustrating a first modification of the third embodiment. The video information processing apparatus 301 illustrated in FIG. 19 is different from the video information processing apparatus 300 illustrated in FIG. 16 in that the analysis unit 341 and the additional information generation unit 342 are replaced with an analysis unit 343 and a past video distribution unit 344, respectively. .

  The analysis unit 343 restores the three-dimensional shape in the facility or the stadium from the video data stored in the storage unit 131 using, for example, SfM, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 343. The analysis unit 343 may determine which area in the facility or stadium the user camera 352 is capturing by matching the free viewpoint video with the video captured by the user camera 352. Note that the method of determining a shooting region is not limited to this, and the various methods of determining a shooting region described in the above embodiments and other methods of determining a shooting region can be used.

  The past video distribution unit 344 outputs a past video to the user camera 352 based on the analysis result of the analysis unit 343. The user camera 352 superimposes the past image on the photographed image or replaces the photographed image with the past image and displays the image on the screen.

  FIG. 20 is an example of a screen of the user camera 352 when soccer is being played in the stadium. FIG. 20A is an example of a screen before replacement with a past video, and FIG. 20B is an example of a screen after replacement with a past video. An example is shown in which a past video is replaced with a highlight scene of the first half during half time. A screen viewed by a user in the stadium using the user camera 352 is transferred to the storage unit 131 of the video information processing apparatus 301 as captured video data, and at the same time, based on the captured video data on the screen viewed by the user. Thus, the image is replaced with the past video generated by the analysis unit 343 and the past video distribution unit 344, so that a sense of realism can be given to the user in the shooting location. In particular, when the user camera 352 is a glasses-type wearable terminal or an HMD, the user can feel more realistic.

  Thus, the user can enjoy the highlight scene of the first half as a video in the direction in which he or she is watching during the half time. Note that the past video is not limited to the highlight scene of the first half, but may be a highlight scene of a past game played at the stadium. The timing at which the video information processing apparatus 301 distributes the past video is not limited to the half-time, and may be, for example, after the end of the match or during the match. Especially during a match, the video information processing apparatus 301 may distribute a scene considered important by the user based on the analysis result of the analysis unit 343. Further, the video information processing apparatus 301 may distribute the past video only when requested by the user, or may distribute a distribution permission message before distributing the past video.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method that give a user a sense of realism in a shooting location by distributing a past video based on the shot video data of the user.

  Also, as shown in FIG. 21, the video information processing apparatus 301 can be applied to a case where captured video data is input only from the user camera 352.

  <Variation 2>

  FIG. 22 is a diagram illustrating a second modification of the third embodiment. The video information processing apparatus 302 shown in FIG. 22 is different from the video information processing apparatus 301 shown in FIG. 19 in that the past video distribution unit 344 is replaced with an advertisement information distribution unit 345.

  The advertisement information distribution unit 345 outputs advertisement information to the user camera 352 based on the analysis result of the analysis unit 343. The user camera 352 superimposes the advertisement information on the captured video and displays it on the screen.

  FIG. 23 is an example of a screen of the user camera 352 when a soccer game is played in the stadium. FIG. 23A is an example of a screen before the advertisement information is superimposed, and FIG. 23B is an example of the screen after the advertisement information is superimposed, which is a case of a drink advertisement. The advertisement information may be distributed, for example, during the half-time or immediately before the distribution of the past video after the match, as shown in Modification Example-1. As a result, the distributor can obtain the advertisement fee from the advertiser, and can provide the user with an inexpensive or free video distribution service. In addition, the video information processing apparatus 302 may distribute an advertisement distribution permission message immediately before distribution of the advertisement information, may provide a service free of charge only when the user views the advertisement, or may not distribute the advertisement. The service may be provided at a lower cost.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method capable of distributing advertisement information based on photographed video data of a user.

  Also, as shown in FIG. 24, the video information processing apparatus 302 can be applied to a case where captured video data is input only from the user camera 352.

(Embodiment 4)
FIG. 25 is a diagram showing a configuration of a video information processing device 400 according to Embodiment 4 of the present invention. The same components as those of the video information processing apparatuses according to Embodiments 1 to 3 are denoted by the same reference numerals, and description thereof is omitted. In the present embodiment, a description will be given of a video information processing apparatus that gives a sense of realism to a user in a shooting location. The video information processing apparatus according to the present embodiment can be applied to a monitoring system, a video distribution system, and the like.

  The video information processing apparatus 400 shown in FIG. 25 is different from the video information processing apparatus 301 shown in FIG. 21 in that the analysis unit 343 and the past video distribution unit 344 are replaced with an analysis unit 441 and a video distribution unit 442, respectively.

  When photographed video data and viewpoint information are input from the L user cameras 451, the storage unit 131 stores them. When photographed video data is input from the N cameras 111, the storage unit 131 stores them.

  The analysis unit 441 restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 using, for example, SfM, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 441. In addition, the analysis unit 441 may determine which area in the facility or stadium the user camera 451 is capturing by matching the free viewpoint video with the video captured by the user camera 451. Note that the method of determining the shooting area is not limited to this.

  FIG. 26 is an example of a method for generating viewpoint information in the user camera 451. FIG. 26A is an example of a screen of the user camera 351 when a soccer game is played in the stadium. When the user performs a swipe operation in the direction of an arrow, for example, the user camera 451 moves the camera in that direction and outputs the viewpoint information. Generate. The video distribution unit 442 reads from the storage unit 131 the video information data moved from the shooting area of the user camera 451 determined by the analysis unit 441 by the amount of the viewpoint information, and starts outputting the video information data to the user camera 451. Then, the user camera 451 displays not the shot video but the video distributed from the video distribution unit 442. The display screen is as shown in FIG. As described above, even after the start of the video distribution, the video distribution unit 442 reads the video information data shifted from the distribution video area by the amount of the viewpoint information from the storage unit 131 and outputs the video information data to the video reception device 161 by the screen swipe operation by the user. .

  As described above, a user in a facility or a stadium can view a video from a desired viewpoint with a simple operation such as a swipe of a screen. For example, a spectator watching a game on the third base side of a baseball stadium can view a video from the viewpoint of the first base side. In addition to the video distribution system, in the surveillance system, security personnel in the facility can view the video from any viewpoint with a simple operation such as swiping the screen, improving the accuracy of the surveillance system be able to.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method that provide a user with a sense of realism in a facility or a stadium by providing a video from a favorite viewpoint.

  In addition, distribution of video to users in a facility or a stadium is also effective, for example, when there is an invisible area between a camera and a shooting target due to an obstacle. In this case, the video of a part of the camera shooting area including the obstacle may be switched from the shot video to the video distributed from the video distribution unit 442 and displayed, or the entire screen may be displayed from the captured video to the video. The image may be switched to the image distributed from the distribution unit 442 and displayed. Further, the captured video and the video distributed from the video distribution unit 442 may be combined to display a video in which the viewing target is visible through an obstacle. According to this configuration, even when the photographing target cannot be seen from the position of the user due to the influence of the obstacle, the video distributed from the video distribution unit 442 can be viewed, so that the influence of the obstacle can be reduced. .

  When displaying a video distributed from the video distribution unit 442 for an area that cannot be seen due to an obstacle, display switching control different from display switching control according to input processing by the user, such as the above-described screen swipe, is performed. May go. For example, when it is determined that an obstacle is included in the shooting area based on information on the movement and shooting direction of the camera and the position information of the obstacle obtained in advance, the image is distributed from the video distribution unit 442 from the captured video. Switching of the display to the video may be performed automatically. Further, when it is determined from the analysis of the captured video data that an obstacle that is not a capturing target is reflected, the display may be automatically switched from the captured video to the video distributed from the video distribution unit 442. . Further, when the area of an obstacle (for example, the number of pixels) included in the captured image exceeds a predetermined threshold value or when the ratio of the area of the obstacle to the area of the imaging target exceeds a predetermined ratio, The display may be automatically switched to the video distributed from the video distribution unit 442.

  Of course, in accordance with the input processing of the user, the display is switched from the captured video to the video distributed from the video distribution unit 442 and the display is switched from the video distributed from the video distribution unit 442 to the captured video. Needless to say, this may be done.

  Also, as shown in FIG. 27, the video information processing apparatus 400 can be applied to a case where captured video data is input only from the user camera 451.

(Embodiment 5)
FIG. 28 is a diagram illustrating a configuration of a video information processing apparatus 500 according to Embodiment 5 of the present invention. The same components as those of the video information processing apparatus according to Embodiment 1-4 are denoted by the same reference numerals, and description thereof will be omitted. In the present embodiment, a description will be given of a video information processing apparatus that instructs the speed at which video data is transferred to the video information processing apparatus based on the importance of video data captured by each camera. The video information processing apparatus according to the present embodiment can be applied to a monitoring system, a video distribution system, and the like.

  The video information processing apparatus 500 shown in FIG. 28 is different from the video information processing apparatus 101 shown in FIG. 2 in that the analysis unit 142 is replaced with an analysis unit 541 and a communication speed management unit 542 is added.

  When the captured video data of the N cameras 511 and the L user cameras 551 is input, the storage unit 131 stores the captured video data. The analysis unit 541 determines the video data stored in the storage unit 131 or the importance of the camera that captured the video data. The determination of the importance here is performed based on information such as the number of people and the number of moving objects included in the video, the image quality of the video data, and a combination thereof.

  Further, the determination of the importance of the video data may be based on the position of the camera where the video data is captured or the area where the video data is captured. In this case, a camera that has a plurality of cameras that are shooting nearby, a camera that has a plurality of cameras that are shooting the same area even if the cameras are far apart, or the importance of video data shot by the camera Can be reduced. Further, the determination of the importance of the video data may be performed based on the number of requests in the video distribution service described in the second embodiment. The method of determining importance is not limited to those described above and combinations thereof, and a method of determining importance according to the configuration or purpose of the monitoring system or the video distribution system may be used.

  Further, the determination of the importance may not be based on the captured video data. For example, the importance of a camera that transfers video data to a terminal other than the video information processing device 500 may be determined to be high. Conversely, the importance of a camera that transfers video data to a terminal other than the video information processing device 500 may be determined to be low. As a result, for example, when a plurality of services that require transmission of video data share a communication band, the degree of freedom in controlling the communication band according to the purpose and characteristics of each service increases, and the required video There is a possibility that the quality of each service can be prevented from deteriorating due to the inability to obtain data.

  The analysis unit 541 restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 using, for example, SfM, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 541. The analyzing unit 541 may determine the importance of the video data using the free viewpoint video and the video captured by the camera 511 and the user camera 551.

  The communication speed management unit 542 outputs a wired communication speed instruction signal to the camera 511 and outputs a wireless communication speed instruction signal to the user camera 551 based on the determination result of the importance performed by the analysis unit 541. The communication speed management unit 542 may, for example, instruct a high communication speed to the camera 511 and the user camera 551 that are capturing images with high importance. This allows efficient communication within the facility or the entire stadium. When the camera 511 performs wireless communication, the communication speed management unit 542 outputs a wireless communication speed instruction signal, and when the user camera 551 performs wired communication, the communication speed management unit 542 outputs a wired communication speed instruction signal. Is output. Further, the communication speed management unit 542 may be configured to control only one of the wired communication and the wireless communication and transmit only one of the wired communication speed instruction signal and the wireless communication speed instruction signal.

  The camera 511 and the user camera 551 transfer photographed video data to the video information processing device 500 according to a wired communication speed instruction signal or a wireless communication speed instruction signal. If the camera 511 and the user camera 551 have failed to retransmit a predetermined number of times, the retransmission of the captured video data may be stopped and the transfer of the next captured video data may be started. Accordingly, communication within the facility or the entire stadium can be efficiently performed, and it is possible to contribute to speeding up of processing in the analysis unit 541.

  If the communication speed assigned to each of the cameras 511 and the user camera 551 is not enough to transfer the captured video data, the camera 511 and the user camera 551 convert the captured video data into video data of a transferable bit rate. Then, the converted video data may be transmitted. In addition, the camera 511 and the user camera 551 may stop the transfer of the video data when the assigned communication speed is not a band enough to transfer the captured video data.

  When video data is used to prevent the occurrence of blind spots described in the first and second embodiments, only some of the shooting areas included in the shot video data have blind spots. May be needed to fill. In this case, extracted video data may be generated by extracting at least a part of the video including an area required to prevent the occurrence of blind spots, and the generated extracted video data may be transferred. According to this configuration, the occurrence of blind spots can be prevented with a smaller communication band.

  Further, for example, when performing the superimposed display of the additional information described in the third embodiment or the video distribution described in the fourth embodiment, it is necessary to transmit camera position information and shooting direction information to the video information processing apparatus. There is. In this case, a camera to which only a band that is not sufficient to transfer the video data may be allocated may transfer only the position information and the shooting direction information detected by the camera. When estimating the position information of the camera and the information of the shooting direction in the video information processing apparatus 500, the video data is converted into a resolution required for estimating the position information of the camera and the information of the shooting direction, and the converted video data is converted. May be transferred. According to this configuration, it is possible to provide a service of superimposed display of additional information and a video distribution service even to a camera to which only a small communication band is allocated. In addition, according to this configuration, since information about the area photographed by each camera can be obtained for more cameras, the area photographed by the camera for the purpose of, for example, detecting the area of interest is detected. It is also effective when using information.

  Note that the switching of the video data transfer process according to the allocated communication band described above may be determined by the camera based on the notified communication band, or the video information processing apparatus 500 The operation may be determined, and each camera may be notified as a control signal indicating the determined operation. Thereby, it is possible to appropriately perform the sharing of the processing in accordance with the amount of calculation necessary for determining the switching of the operation, the processing capability of the control unit of the camera, the required communication band, and the like.

  With the above configuration, the analysis unit determines the importance of the captured video data, and instructs the transfer speed based on the importance, thereby efficiently performing communication within the facility or the entire stadium, the video information processing apparatus, and the video information. A processing method can be provided.

  Also, as shown in FIG. 29, the video information processing apparatus 500 can be applied to a case where captured video data is input only from the user camera 551.

  <Modification-1>

  Note that the video information processing apparatus 500 shown in FIG. 28 may be replaced with the video information processing apparatus 501 shown in FIG. The video information processing apparatus 501 shown in FIG. 30 is different from the video information processing apparatus 500 shown in FIG. 28 in that the analysis unit 541 is replaced with an analysis unit 543 and a video distribution unit 545 is added.

  The analysis unit 543 restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 using, for example, SfM, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 543. The video distribution unit 545 reads video information data corresponding to the view information (and / or viewpoint information) input from the video reception device 161 from the storage unit 131, and outputs the video information data to the video reception device 161. In addition, the video distribution unit 545 outputs visual field information (and / or viewpoint information) to the analysis unit 543.

  The analysis unit 543 may determine the importance of the video data based on the visual field information (and / or the viewpoint information). For example, the degree of importance of captured video data including many regions indicated by the visual field information (and / or viewpoint information) is increased. In addition, the analysis unit 543 may determine the importance of the video data in consideration of the number of people, the number of moving objects, and the like included in the video. The method of determining the importance is not limited to this.

  With the above configuration, the analysis unit determines the importance of the captured video data, and instructs the transfer speed based on the importance, thereby efficiently performing communication within the facility or the entire stadium, the video information processing apparatus, and the video information. A processing method can be provided.

  Also, as shown in FIG. 31, the video information processing device 501 can be applied to a case where captured video data is input only from the user camera 551.

  Note that the communication control method described in the present embodiment does not necessarily need to be used in a system that reconstructs a three-dimensional shape from a plurality of video data. For example, in an environment where a plurality of cameras and user cameras are present, if video data is transmitted by wire communication and / or wireless communication selectively or with a different transmission speed, the communication control described in the present embodiment is used. The method may be effective.

(Embodiment 6)
FIG. 32 is a diagram showing a configuration of a video information processing apparatus 600 according to Embodiment 6 of the present invention. The same components as those in the video information processing apparatuses according to the first to fifth embodiments are denoted by the same reference numerals, and description thereof is omitted. The present embodiment relates to a video information processing apparatus that efficiently restores a three-dimensional shape from a plurality of pieces of photographing data in a facility or a stadium. The video information processing apparatus according to the present embodiment can be applied to a monitoring system, a video distribution system, and the like.

  The video information processing apparatus 600 shown in FIG. 32 is different from the video information processing apparatus 103 shown in FIG. 6 in that the video distribution unit 144 and the analysis unit 145 are replaced with a video distribution unit 641 and an analysis unit 642, respectively.

  When video data captured by each of the N cameras 111 and the L user cameras 651 is input, the storage unit 131 stores the video data.

  The analysis unit 642 restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 using, for example, SfM, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 642.

  When a distribution request is input from the video receiver 161, the video distributor 641 reads an overview video of the entire facility or stadium from the storage unit 131 and outputs the overview video to the video receiver 161. The overview video may have a long update interval (may have a low frame rate) and may have low image quality. The viewer gives an instruction by touching a desired part in the overview video displayed on the screen of the video receiving device 161, and the video receiving device 161 transmits the visual field information (and / or the viewpoint information) to the video distribution unit 641. Output.

  The video distribution unit 641 reads out video information data corresponding to the view information (and / or viewpoint information) from the storage unit 131, and outputs the video information data to the video receiving device 161. In addition, the video distribution unit 641 outputs view information (and / or viewpoint information) to the analysis unit 642.

  The analysis unit 642 preferentially performs the three-dimensional shape restoration on the region indicated by the view information (and / or the viewpoint information), and generates a free viewpoint image. The analysis unit 642 may make the update interval longer (may be a lower frame rate) and may lower the image quality, assuming that the three-dimensional shape restoration of the entire facility or stadium shows an overview. The storage unit 131 stores the free viewpoint video generated by the analysis unit 642. Thereby, the three-dimensional shape can be efficiently restored. As a result, it contributes to a higher frame rate and a higher image quality of a free viewpoint video in an area desired by the viewer.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method that efficiently perform three-dimensional shape restoration.

  Also, as shown in FIG. 33, the video information processing apparatus 600 can be applied to a case where captured video data is input only from the user camera 651.

  <Modification-1>

  The video information processing device 600 shown in FIG. 32 may be replaced with a video information processing device 601 shown in FIG. The video information processing device 601 illustrated in FIG. 34 includes a storage unit 131, a preliminary video storage unit 632, and an analysis unit 643.

  The preliminary image storage unit 632 previously stores, for example, three-dimensional shape restoration data of a facility or a stadium generated in advance as a preliminary image. Note that the preliminary image is not limited to this.

  The analysis unit 643 restores the three-dimensional shape in the facility or the stadium from the video data stored in the storage unit 131 using, for example, SfM, and generates a free viewpoint video. For example, when soccer is played in a stadium, the analysis unit 643 may generate a free viewpoint video by performing a three-dimensional shape restoration only for a player and a ball and combining the three-dimensional shape with a preliminary video. Alternatively, the analysis unit 643 may perform the three-dimensional shape restoration by giving priority to the player and the ball. Thereby, the three-dimensional shape can be efficiently restored. As a result, it contributes to a higher frame rate and a higher image quality of the free viewpoint video relating to the player and the ball that the viewer pays attention to. When applied to a monitoring system, the analysis unit 643 may perform the three-dimensional shape restoration by limiting only to a person and a moving object or giving priority to them.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method for performing three-dimensional shape restoration more efficiently.

  As shown in FIG. 35, the video information processing device 601 can be applied to a case where captured video data is input only from the user camera 651.

  <Variation 2>

  Note that the video information processing apparatus 600 shown in FIG. 32 may be replaced with a video information processing apparatus 602 shown in FIG. The video information processing device 602 illustrated in FIG. 36 includes a storage unit 131 and an analysis unit 644.

  The analysis unit 644 restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 using, for example, SfM, and generates a free viewpoint video. The analysis unit 644 performs the three-dimensional shape restoration using only the data at a close time among the video data captured by the plurality of cameras 111 and the plurality of user cameras 651. The detection of the time is, for example, the time when the transfer of the captured video data to the storage unit is completed. The method of detecting the time is not limited to this. Thereby, the three-dimensional shape can be efficiently restored. As a result, it contributes to higher frame rate and higher image quality of the free viewpoint video.

  Alternatively, the analyzing unit 644 may perform the three-dimensional shape restoration by limiting only the high-quality data or preferentially using the high-quality data among the video data stored in the storage unit 131.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method that efficiently perform three-dimensional shape restoration.

  As shown in FIG. 37, the video information processing device 602 can be applied to a case where captured video data is input only from the user camera 651.

  <Modification Example-3>

  The video information processing device 600 shown in FIG. 32 may be replaced with a video information processing device 603 shown in FIG. The video information processing device 603 illustrated in FIG. 38 includes a storage unit 131 and an analysis unit 645.

  When the captured video data and the camera attribute information are input from the L user cameras 652 and the N cameras 612, the storage unit 131 stores them. The camera attribute information includes, for example, a shooting position, a shooting angle, a shooting time, a zoom, and the like, but is not limited thereto.

  The analysis unit 645 restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 using, for example, SfM, and generates a free viewpoint video. The analysis unit 645 performs three-dimensional shape restoration using the camera attribute information. Thereby, the three-dimensional shape can be efficiently restored. As a result, it contributes to higher frame rate and higher image quality of the free viewpoint video.

  A method of generating camera attribute information in the camera 612 and the user camera 652 will be described below, but this is an example, and the present invention is not limited to this.

  Three-dimensional coordinates are defined in the facility or stadium, and information on what coordinates the camera 612 and the user camera 652 took from which angle, how much zoom, and at what time are stored as camera attribute information together with the video. Then, it may be transferred to the video information processing device 603. When the camera 612 and the user camera 652 are activated, time information can be generated by synchronizing a clock on a communication network in a facility or a stadium with a clock in the camera.

  Regarding the specification of the coordinates, for example, the position / angle information may be acquired by adjusting the camera to a specific point in the facility or the stadium at the time of starting the user camera 652 or at an arbitrary timing. FIG. 39 is an example of a notification displayed on the screen of the user camera 652 when the user camera 652 is activated. When the user touches the display of the camera in accordance with this notification by adjusting the “+” at the center of the soccer ball in the advertisement on the north side of the stadium, the user camera 652 acquires vector information from the camera to the advertisement and obtains the camera position and angle. Identify criteria. After that, the camera coordinates and angle at each time are specified from the motion information of the camera.

  Regarding the specification of the coordinates, the camera 612 and the user camera 652 may use GPS, or may use radio waves of WiFi (wireless LAN). Alternatively, information indicating which base station in the facility or stadium has received the captured video data may be used.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method that efficiently perform three-dimensional shape restoration.

  Also, as shown in FIG. 40, the video information processing device 603 can be applied to a case where captured video data is input only from the user camera 652.

(Supplement)
The present invention is not limited to the contents described in the above embodiments, and can be implemented in any form for achieving the object of the present invention and the related or accompanying objects. For example, the following may be adopted. .

  (1) Embodiments 1 to 6 may be implemented in any combination.

  (2) In the image matching in the first, third, and fourth embodiments, the three-dimensional shape restoration data used may be the pre-image shown in the sixth embodiment.

  (3) The above embodiments may relate to implementation using hardware and software. The embodiments described above may be implemented or performed using a computing device (processor). The computing device or processor may be, for example, a main processor / general purpose processor, a digital signal processor (DSP), an ASIC (application specific integrated circuit), an FPGA (field programmable device), a logic device, and the like. May be. The above-described embodiments may be implemented or realized by combining these devices.

  (4) The above embodiments may be realized by a mechanism of a software module executed by a processor or directly by hardware. Also, a combination of software modules and hardware implementation is possible. The software modules may be stored on various types of computer readable storage media, for example, RAM, EPROM, EEPROM, flash memory, registers, hard disks, CD-ROM, DVD, and the like.

  The information processing apparatus, information processing method, camera, receiving apparatus, and receiving method according to the present invention can be applied to a monitoring system, a video distribution system, and the like.

100, 101, 102, 103, 104, 105, 200, 201, 300, 301, 302, 400, 500, 501, 600, 601, 602, 603 Video information processing devices 111, 511, 612 Cameras 121, 122 Moving cameras 131 storage units 141, 142, 143, 145, 146, 147, 241, 341, 343, 441, 541, 543, 642, 643, 644, 645 analysis units 144, 242, 442, 545, 641 video distribution unit 151, 152, 251, 351, 352, 451, 551, 651, 652 User camera 161 Video receiving device 243 Charging management unit 244 Point management unit 342 Additional information generation unit 344 Past video distribution unit 345 Advertising information distribution unit 542 Communication speed management unit 632 Advance video storage

Claims (4)

A storage unit for storing a plurality of shooting video data,
An analysis unit that detects a blind spot from the plurality of shot video data stored in the storage unit, generates an instruction signal to prevent the blind spot, and outputs the instruction signal to a camera that generates the shot video data. Information processing device. The information processing device according to claim 1, wherein the analysis unit generates, as the instruction signal, a signal that instructs movement of the camera and / or a signal that instructs a shooting direction of the camera. The information processing apparatus further includes a video distribution unit,
The analysis unit generates a free viewpoint video from the captured video data stored in the storage unit,
The storage unit stores the free viewpoint video generated by the analysis unit,
The video distribution unit outputs video information data corresponding to the input visual field information and / or viewpoint information to a video receiving device that generates the visual field information and / or viewpoint information,
The information processing apparatus according to claim 1, wherein the analysis unit generates an instruction signal to prevent a blind spot when the video information data output from the video distribution unit has a blind spot. . A storage step for storing the captured video data,
An analysis step of detecting a blind spot from the captured video data stored in the storage step, generating an instruction signal to prevent the blind spot, and outputting the instruction signal to a camera that generates the captured video data. Information processing method.

Images