JP2015204512A - Information processing apparatus, information processing method, camera, reception device, and reception method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent dead angles which are sometimes difficult to eliminate even if many cameras are installed, in a monitoring system.SOLUTION: A storage unit 131 stores photographic image data obtained by cameras 111 and mobile cameras 121. An analyzer 141 detects a dead angle from the photographic image data stored in the storage unit, generates a travel indication signal so as to prevent the occurrence of the dead angle, and outputs the travel indication signal to the mobile cameras 121. The mobile cameras 121 move in response to the travel indication signal and continue photographing.

Description

  The present invention relates to information processing technology, and more particularly to video technology that enables blind spot prevention, high presence, efficient communication, and efficient three-dimensional shape restoration, or related technology.

  In recent years, surveillance systems have become more intelligent and the monitoring space has become wider. With the widening of the monitoring target space, the number of cameras installed tends to increase in order to photograph the entire monitoring target space with fewer blind spots.

  Also, in video distribution systems such as television broadcasting, the number of cameras used for shooting tends to increase, especially for the purpose of shooting powerful scenes during sports broadcasts.

JP 2011-120283 A

Torii et al .: “Instant 3D reconstruction by sequential SfM and surface generation”, 19th Symposium on Image Sensing (SSII2013), pp. IS2-33-1-4, June 2013

  However, even if a large number of cameras are installed in the surveillance system, it may be difficult to completely eliminate the blind spots at all times.

  Also, in a video distribution system such as a television broadcast, even if a large number of cameras are installed, it may be difficult to shoot without missing a powerful scene by completely eliminating the blind spots at all times.

  To solve this problem, studies are being made on a free viewpoint video system in which a large number of fixed cameras are installed to reproduce video at an arbitrary position. For example, Patent Document 1 discloses an information processing method that does not cause an excessive amount of information processing in a transmission-side system when generating a video of an arbitrary viewpoint based on a plurality of camera videos.

  By the way, if it is a facility such as a stadium, it is possible to install a large number of fixed cameras in various places, but for example, for outdoor events, the installation place is limited, and it is difficult to generate a free viewpoint video. .

  Furthermore, the conventional free viewpoint video system is assumed to be viewed by a user outside the shooting location, and the user inside the shooting location cannot receive the benefits of the free viewpoint video.

  This application is used in a video processing device, a video shooting device such as a camera, a video receiving device, and those devices that are effective in a system that handles a plurality of cameras and a plurality of video data born in consideration of solving the above problems. The purpose is to provide a method.

  As an example of a more specific purpose, an object is to provide an information processing apparatus and an information processing method for preventing blind spots or prohibiting photographing of a specific area. Another object of the present invention is to provide an information processing apparatus and an information processing method for distributing video for free or at a low price to a user who has transferred a captured video. Another object of the present invention is to provide an information processing apparatus and an information processing method that give a user a sense of reality in a shooting location. As another example, an object is to provide an information processing apparatus and an information processing method that efficiently perform communication within a facility or the entire stadium. Another object of the present invention is to provide an information processing apparatus and an information processing method that efficiently perform three-dimensional shape restoration.

  It is another object of the present invention to provide a camera, a video receiving device, and a video receiving method that can be connected to the information processing device and the information processing method.

  One aspect of the present invention that achieves one of the above objects is an information processing apparatus, a storage unit that stores captured video data, a blind spot detected from the captured video data stored in the storage unit, and a blind spot An analysis unit that generates an instruction signal so as to prevent the image and outputs the instruction video data to a camera that generates the captured video data.

  According to the above invention, it is possible to provide an information processing apparatus that prevents the blind spot by detecting the blind spot and outputting the instruction signal so as to prevent the blind spot.

FIG. 3 is a diagram illustrating a configuration of the information processing apparatus 100 according to the first embodiment. FIG. 3 illustrates a configuration of the information processing apparatus 101 according to the first embodiment. FIG. 9 is a diagram illustrating an example of an instruction displayed on the screen of the user camera 151 based on a direction instruction signal in the first embodiment. FIG. 3 illustrates a configuration of the information processing apparatus 101 according to the first embodiment. FIG. 3 illustrates a configuration of the information processing apparatus 102 according to the first embodiment. FIG. 3 illustrates a configuration of the information processing apparatus 103 according to Embodiment 1. FIG. 3 illustrates a configuration of the information processing apparatus 103 according to Embodiment 1. FIG. 3 is a diagram illustrating a configuration of the information processing device 104 according to the first embodiment. FIG. 3 is a diagram illustrating a configuration of the information processing device 105 according to the first embodiment. FIG. 10 is a diagram illustrating an example of a notification displayed on the screen of the user camera 152 based on a shooting prohibition notification signal in the second embodiment. FIG. 3 is a diagram illustrating a configuration of the information processing device 105 according to the first embodiment. The figure which shows the structure of the information processing apparatus 200 in Embodiment 2. FIG. The figure which shows the structure of the information processing apparatus 200 in Embodiment 2. FIG. FIG. 10 is a diagram illustrating a configuration of the information processing apparatus 201 according to the second embodiment. FIG. 10 is a diagram illustrating a configuration of the information processing apparatus 201 according to the second embodiment. FIG. 10 illustrates a configuration of an information processing device 300 according to Embodiment 3. In Embodiment 3, it is a figure which shows an example of the screen of the user camera 351 when soccer is performed in a stadium. FIG. 10 illustrates a configuration of an information processing device 300 according to Embodiment 3. FIG. 9 illustrates a configuration of an information processing device 301 according to Embodiment 3. In Embodiment 3, it is a figure which shows an example of the screen of the user camera 352 when soccer is performed in a stadium FIG. 9 illustrates a configuration of an information processing device 301 according to Embodiment 3. FIG. 9 illustrates a configuration of an information processing device 302 according to Embodiment 3. In Embodiment 3, it is a figure which shows an example of the screen of the user camera 352 when soccer is performed in a stadium FIG. 9 illustrates a configuration of an information processing device 302 according to Embodiment 3. FIG. 10 is a diagram illustrating a configuration of an information processing device 400 according to Embodiment 4. In Embodiment 4, it is a figure which shows an example of the method of producing | generating viewpoint information in the user camera 451 FIG. 10 is a diagram illustrating a configuration of an information processing device 400 according to Embodiment 4. FIG. 10 is a diagram illustrating a configuration of an information processing device 500 according to a fifth embodiment. FIG. 10 is a diagram illustrating a configuration of an information processing device 500 according to a fifth embodiment. FIG. 10 illustrates a configuration of an information processing device 501 in Embodiment 5. FIG. 10 illustrates a configuration of an information processing device 501 in Embodiment 5. FIG. 10 is a diagram illustrating a configuration of an information processing device 600 according to a sixth embodiment. FIG. 10 is a diagram illustrating a configuration of an information processing device 600 according to a sixth embodiment. FIG. 10 illustrates a configuration of an information processing device 601 according to a sixth embodiment. FIG. 10 illustrates a configuration of an information processing device 601 according to a sixth embodiment. FIG. 10 illustrates a configuration of an information processing device 602 according to a sixth embodiment. FIG. 10 illustrates a configuration of an information processing device 602 according to a sixth embodiment. FIG. 10 illustrates a configuration of an information processing device 603 according to Embodiment 6. FIG. 18 is a diagram illustrating an example of a notification displayed on the screen of the user camera 652 when the user camera 652 is activated in the sixth embodiment. FIG. 10 illustrates a configuration of an information processing device 603 according to Embodiment 6.

  Hereinafter, prior to the description of each embodiment of the present invention, a part of the aspects of the present invention included in the following description of the embodiment and the effects thereof will be described.

  The invention of claim 1 of the present application is an information processing apparatus, a storage unit for storing captured video data, and an instruction signal for detecting a blind spot from the captured video data stored in the storage unit and preventing the blind spot And an analysis unit that outputs the captured video data to a camera that generates the captured video data.

  According to a second aspect of the present invention, in the information processing apparatus according to the first aspect, the analysis unit receives, as the instruction signal, a signal for instructing movement of the camera and / or a signal for instructing a photographing direction of the camera. It is characterized by generating.

  The invention according to claim 3 of the present application is the information processing apparatus according to claim 1, wherein the information processing apparatus further includes a video distribution unit, and the analysis unit uses the captured video data stored in the storage unit. A free viewpoint video is generated, the storage unit stores the free viewpoint video generated by the analysis unit, and the video distribution unit stores video information data corresponding to input visual field information and / or viewpoint information. The visual field information and / or viewpoint information is read out and output to the video reception device, and the analysis unit prevents the blind spot when the video information data output from the video distribution unit has a blind spot. Thus, the instruction signal is generated as described above.

  The invention of claim 4 of the present application is a camera, characterized in that it moves based on the instruction signal output from the information processing apparatus of claim 1 and / or displays the instruction content on the screen. It is.

  The invention according to claim 5 of the present application is an information processing apparatus, and detects whether or not a shooting prohibited area is shot from a storage unit that stores captured video data and the captured video data stored in the storage unit In addition, it is characterized by having an analysis unit that generates a notification signal so as to prohibit shooting when shooting, and outputs the signal to the camera that generates the shot video data.

  The information processing apparatus according to claim 6 of the present application is the information processing apparatus according to claim 5, wherein the analysis unit uses, as the notification signal, a signal for instructing prohibition of photographing of the camera and / or a signal for informing of prohibition of photographing of the camera. Is generated.

  The invention of claim 7 of the present application is a camera, characterized in that, based on a notification signal output from the information processing apparatus of claim 5, shooting is stopped and / or notification content is displayed on the screen. Is.

  The invention of claim 8 of the present application is an information processing apparatus, a storage unit that stores captured video data, an analysis unit that generates a free viewpoint video from the captured video data stored in the storage unit, and video information data And a charge management unit that manages charging, the storage unit stores the free viewpoint video generated by the analysis unit, and the video distribution unit includes: Video information data corresponding to the viewpoint information is read from the storage unit and output to the video reception device that generates the visual field information and / or the viewpoint information. The billing management unit is the same user as the video reception device. The video information data output from the video distribution unit is charged based on whether or not the photographed video data has been transferred from the camera.

  The invention of claim 9 of the present application is an information processing apparatus, a storage unit for storing captured video data, an analysis unit for generating a free viewpoint video from the captured video data stored in the storage unit, and the captured video It is characterized by having a point management part which manages the point with respect to the user of the camera which produces | generates data.

  The invention of claim 10 of the present application is an information processing apparatus, wherein a storage unit that stores captured video data, an analysis unit that analyzes the captured video data stored in the storage unit, and an analysis result of the analysis unit And an additional information generating unit that generates additional information to be superimposed on the captured video data and outputs the additional information to a camera that generates the captured video data.

  The invention according to claim 11 of the present application is the information processing apparatus according to claim 10, wherein the additional information is advertisement information.

  A twelfth aspect of the present invention is a camera, wherein the additional information output from the information processing apparatus according to the tenth aspect is displayed superimposed on the captured video data.

  A thirteenth aspect of the present invention is an information processing apparatus, a storage unit that stores captured video data, an analysis unit that generates a free viewpoint video from the captured video data stored in the storage unit, and video information data A past video distribution unit that outputs, the storage unit stores the free viewpoint video generated by the analysis unit, the analysis unit analyzes the captured video data stored in the storage unit, The past video distribution unit reads past video information data from the storage unit based on the analysis result of the analysis unit and outputs the data to a camera that generates the captured video data.

  The invention of claim 14 of the present application is a camera, and is characterized by displaying past video information data output from the information processing apparatus of claim 13.

  The invention of claim 15 of the present application is an information processing apparatus, a storage unit that stores captured video data and viewpoint information, an analysis unit that generates a free viewpoint video from the captured video data stored in the storage unit, A video distribution unit that outputs video information data, wherein the video distribution unit reads out video information data corresponding to the viewpoint information from a storage unit and outputs the video information data to the camera that generates the captured video data and the viewpoint information; It is characterized by.

  The invention of claim 16 of the present application is a camera, and is characterized by displaying video information data output from the information processing apparatus of claim 15.

  The invention of claim 17 of the present application is an information processing apparatus, a storage unit for storing captured video data, an analysis unit for analyzing the captured video data stored in the storage unit, and a communication speed for managing a communication speed A communication unit that generates an instruction signal indicating a communication speed of the camera that generates the captured video data based on the analysis result of the analysis unit, and outputs the instruction signal to the camera. It is a feature.

  According to claim 18 of the present application, in the information processing apparatus according to claim 17, the information processing apparatus further includes a video distribution unit, and the analysis unit is configured to use the captured video data stored in the storage unit. A free viewpoint video is generated, the storage unit stores the free viewpoint video generated by the analysis unit, and the video distribution unit stores video information data corresponding to input visual field information and / or viewpoint information. Read out from the unit, and outputs the visual field information and / or viewpoint information to a video receiving device, the analysis unit determines the importance of the captured video data based on the visual field information and / or viewpoint information, The communication speed management unit generates the instruction signal based on the importance.

  A nineteenth aspect of the present invention is a camera, which is characterized in that, based on an instruction signal output from the information processing apparatus according to the seventeenth aspect, photographed video data is transferred to the information processing apparatus.

  The invention of claim 20 of the present application is an information processing apparatus, a storage unit for storing captured video data, an analysis unit for generating a free viewpoint video from the captured video data stored in the storage unit, and video information data And the storage unit stores the free viewpoint video generated by the analysis unit, and the video distribution unit includes video information corresponding to input visual field information and / or viewpoint information. Data is read from the storage unit and output to the video reception device that generates the visual field information and / or viewpoint information, and the analysis unit preferentially performs the region indicated by the visual field information and / or viewpoint information. It is characterized by generating a free viewpoint video.

  The invention of claim 21 of the present application is a video receiving apparatus, characterized in that the video information data output from the information processing apparatus of claim 20 is displayed.

  The invention of claim 22 of the present application is an information processing apparatus, wherein a storage unit that stores captured video data, a pre-video storage unit that stores a pre-generated free viewpoint video, and the storage unit stored in the storage unit It has an analysis part which updates and produces | generates a free viewpoint image | video from the picked-up image data and the said free viewpoint image | video preserve | saved at the said prior | preceding image | video preservation | save part.

  The invention of claim 23 of the present application is an information processing apparatus, comprising: a storage unit that stores captured video data; and an analysis unit that generates a free viewpoint video from the captured video data stored in the storage unit, The storage unit stores the free viewpoint video generated by the analysis unit, and the analysis unit uses only data at a close time in the captured video data and / or is limited to high-quality data only or high The image quality data is preferentially used.

  The invention of claim 24 of the present application is an information processing apparatus, a storage unit for storing captured video data and camera attribute information, the captured video data stored in the storage unit, and free viewpoint video from the camera attribute information. The camera attribute information is at least one of a photographing position, a photographing angle, a photographing time, and a zoom.

  The invention of claim 25 of the present application is a camera, characterized in that camera attribute information used by the information processing apparatus of claim 24 is generated.

  According to a twenty-sixth aspect of the present invention, in the camera according to the twenty-fifth aspect, the camera attribute information is generated by matching a photographing position with a reference point when the camera is activated.

  According to a twenty-seventh aspect of the present invention, in the camera according to the twenty-fifth aspect, as the camera attribute information, a shooting position is generated using at least one of information on GPS, WiFi, and a base station. It is a feature.

  According to a twenty-eighth aspect of the present invention, in the camera according to the twenty-fifth aspect, as the camera attribute information, a photographing time is generated by synchronizing a clock on a communication network and a clock in the camera. It is.

  The invention of claim 29 of the present application is an information processing method, a storage step for storing captured video data, and an instruction signal for detecting a blind spot from the captured video data stored in the storage step and preventing the blind spot. And an analysis step of outputting the captured video data to a camera that generates the captured video data.

  The invention of claim 30 of the present application is an information processing method for detecting whether or not a shooting prohibited area is shot from a saving step for saving shot video data and the shot video data saved in the saving step. In addition, it is characterized by having an analysis step of generating a notification signal so as to prohibit shooting when shooting, and outputting the shot video data to the camera.

  The invention of claim 31 of the present application is an information processing method, a storage step of storing captured video data, an analysis step of generating a free viewpoint video from the captured video data stored in the storage step, and video information data A video distribution step for outputting the charge, and a charge management step for managing the charge, wherein the storage step stores the free viewpoint video generated in the analysis step, and the video distribution step includes: The video information data corresponding to the viewpoint information is read from the storage step, and is output to the video reception device that generates the visual field information and / or the viewpoint information. The charging management step is performed by the same user as the video reception device. The video information data output in the video distribution step is based on whether the captured video data has been transferred from the camera. It is characterized in that to perform data accounting management.

  The invention of claim 32 of the present application is an information processing method, a storage step of storing captured video data, an analysis step of generating a free viewpoint video from the captured video data stored in the storage step, and the captured video It has a point management step for managing points for a camera user who generates data.

  The invention of claim 33 of the present application is an information processing method, comprising: a storage step for storing captured video data; an analysis step for analyzing the captured video data stored in the storage step; and an analysis result of the analysis step. Based on the above, an additional information generation step of generating additional information to be superimposed on the captured video data and outputting the additional information to a camera that generates the captured video data is provided.

  The invention of claim 34 of the present application is an information processing method, a storage step for storing captured video data, an analysis step for generating a free viewpoint video from the captured video data stored in the storage step, and video information data A past video distribution step for outputting, wherein the storing step stores the free viewpoint video generated in the analyzing step, and the analyzing step analyzes the captured video data stored in the storing step, In the past video distribution step, past video information data is read from the storage step based on the analysis result of the analysis step, and is output to the camera that generates the captured video data.

  The invention of claim 35 of the present application is an information processing method, a storage step of storing captured video data and viewpoint information, an analysis step of generating a free viewpoint video from the captured video data stored in the storage step, A video distribution step for outputting video information data, wherein the video distribution step reads out the video information data corresponding to the viewpoint information from the storage step and outputs the video information data and the viewpoint information to a camera that generates the video information data; It is characterized by.

  The invention of claim 36 of the present application is an information processing method, a storage step for storing captured video data, an analysis step for analyzing the captured video data stored in the storage step, and a communication speed for managing a communication speed. A management step, wherein the communication speed management step generates an instruction signal indicating the communication speed of the camera that generates the captured video data based on the analysis result of the analysis step, and outputs the instruction signal to the camera It is a feature.

  The invention of claim 37 of the present application is an information processing method, a storage step of storing captured video data, an analysis step of generating a free viewpoint video from the captured video data stored in the storage step, and video information data A video distribution step for outputting the video, and the storage step stores the free viewpoint video generated in the analysis step, and the video distribution step includes video information corresponding to input visual field information and / or viewpoint information. Data is read from the storage step and output to the video reception device that generates the visual field information and / or viewpoint information, and the analysis step preferentially performs the region indicated by the visual field information and / or the viewpoint information. It is characterized by generating a free viewpoint video.

  The invention of claim 38 of the present application is an information processing method, a storage step of storing captured video data, a pre-video storage step of storing a free viewpoint video generated in advance, and the storage step stored in the storage step The method further comprises an analysis step of updating and generating a free viewpoint video from the captured video data and the free viewpoint video stored in the previous video storage step.

The invention of claim 39 of the present application is an information processing method, comprising: a storage step of storing captured video data; and an analysis step of generating a free viewpoint video from the captured video data stored in the storage step, The storage step stores the free viewpoint video generated in the analysis step, and the analysis step uses only data at a close time in the captured video data and / or is limited to high-quality data only or high The image quality data is preferentially used.
The invention of claim 40 of the present application is an information processing method, comprising: a storage step for storing captured video data and camera attribute information; the captured video data stored in the storage step; The camera attribute information is at least one of a shooting position, a shooting angle, a shooting time, and a zoom.

  According to the first aspect of the present invention, it is possible to provide an information processing apparatus that prevents a blind spot by detecting a blind spot and outputting an instruction signal so as to prevent the blind spot.

  According to the second aspect of the present invention, the blind spot is prevented by generating a signal for instructing the movement of the camera and / or a signal for instructing the photographing direction as the instruction signal. An information processing apparatus can be provided.

  According to the third aspect of the present invention, when there is a blind spot in the video information data output from the video distribution unit, the analysis unit generates the instruction signal so as to prevent the blind spot, thereby preventing the blind spot. An information processing apparatus can be provided.

  According to the fourth aspect of the present invention, it is possible to provide a camera that moves based on the instruction signal output from the information processing apparatus according to the first aspect and / or displays the instruction content on the screen.

  According to the fifth aspect of the present invention, there is provided an information processing apparatus that prohibits photographing in a certain region by the analysis unit determining prohibition of photographing and outputting a notification signal to the camera so as to prohibit photographing. Can be provided.

  According to the invention of claim 6, in the information processing apparatus for preventing blind spots in claim 5, in particular, the analysis unit generates a signal for instructing the camera prohibition and / or a signal for notification as the notification signal. An apparatus can be provided.

  According to the seventh aspect of the present invention, it is possible to provide a camera that stops shooting and / or displays the notification content on the screen based on the notification signal output from the information processing apparatus according to the fifth aspect. .

  Further, according to the eighth aspect of the invention, the charge management unit performs charge management based on whether or not the user has transferred the captured video, so that the user who has transferred the captured video is different from the user who is not. It is possible to provide an information processing apparatus that distributes video for a fee.

  According to the ninth aspect of the present invention, it is possible to provide an information processing apparatus that gives points to a user who has transferred a captured video by performing point management by the point management unit.

  According to the tenth aspect of the present invention, there is provided an information processing apparatus that gives a user a sense of presence in a shooting place by generating additional information based on the user's captured video data by the analyzing unit and the additional information generating unit. can do.

  According to the eleventh aspect of the present invention, the analysis unit and the additional information generation unit generate advertisement information based on the user's captured video data, so that the advertisement information can be distributed based on the user's captured video data. A processing device can be provided.

  According to the twelfth aspect of the present invention, a camera can be provided that displays the additional information output from the information processing apparatus according to the tenth aspect by superimposing it on the captured video data.

  According to the invention of claim 13, it is possible to provide an information processing apparatus that gives a user a sense of realism in a shooting location by distributing past video based on user's shot video data.

  According to the fourteenth aspect of the present invention, it is possible to provide a camera that displays past video information data output from the information processing apparatus according to the thirteenth aspect.

  According to the fifteenth aspect of the present invention, it is possible to provide an information processing apparatus that gives a sense of reality to users in a facility or stadium by providing an image from a favorite viewpoint.

  According to the sixteenth aspect of the present invention, it is possible to provide a camera for displaying video information data output from the information processing apparatus according to the fifteenth aspect.

  In addition, according to the invention described in claim 17, it is possible to provide an information processing apparatus that efficiently performs communication within the facility and the entire stadium.

  According to the invention of claim 18, the analysis unit determines the importance of the captured video data, and instructs the communication speed based on the importance, thereby efficiently communicating in the facility or the entire stadium. An information processing apparatus can be provided.

  According to the nineteenth aspect of the present invention, it is possible to provide a camera that transfers photographed video data to the information processing apparatus based on an instruction signal output from the information processing apparatus according to the seventeenth aspect.

  According to the twentieth aspect of the present invention, the analysis unit preferentially generates a free viewpoint video for the area indicated by the visual field information and / or the viewpoint information input from the video receiving device, thereby generating a three-dimensional shape. An information processing apparatus that efficiently performs restoration can be provided.

  According to the twenty-first aspect of the present invention, it is possible to provide a video receiving apparatus that displays the video information data output from the information processing apparatus according to the twentieth aspect.

  According to the twenty-second aspect of the present invention, it is possible to provide an information processing apparatus that efficiently restores a three-dimensional shape by using a free viewpoint video stored in advance in the preliminary video storage unit.

  According to the invention of claim 23, the analysis unit uses only data at a close time in the captured video data and / or restricts only to high-quality data or uses high-quality data preferentially, An information processing apparatus that efficiently performs three-dimensional shape restoration can be provided.

  According to the twenty-fourth aspect of the present invention, it is possible to provide an information processing apparatus that efficiently performs three-dimensional shape restoration by using the camera attribute information by the analysis unit.

  According to the invention of claim 25, a camera for generating camera attribute information used by the information processing apparatus of claim 24 can be provided.

  According to the twenty-sixth aspect of the present invention, as the camera attribute information used by the information processing apparatus according to the twenty-fourth aspect, it is possible to provide a camera that is generated by matching the shooting position to a certain reference point when the camera is activated.

  According to the twenty-seventh aspect of the present invention, as the camera attribute information used by the information processing apparatus according to the twenty-fourth aspect, the shooting position is generated by using at least one of GPS, WiFi, and base station information. A camera can be provided.

  According to the twenty-eighth aspect of the present invention, there is provided a camera that generates, as camera attribute information used by the information processing apparatus according to the twenty-fourth aspect, a shooting time by synchronizing a clock on a communication network and a clock in the camera. be able to.

  According to the invention of claim 29, it is possible to provide an information processing method for preventing the blind spot by detecting the blind spot and outputting the instruction signal so as to prevent the blind spot in the analyzing step.

  According to the invention of claim 30, there is provided an information processing method in which an analysis step determines whether to shoot, and outputs a notification signal to the camera so that shooting is prohibited, thereby prohibiting shooting of a certain area. Can be provided.

  According to the invention of claim 31, the charge management step performs charge management based on whether or not the user has transferred the photographed video, so that the user who has transferred the photographed video is different from the user who is not. It is possible to provide an information processing method for performing video distribution with charge.

  According to the invention of claim 32, it is possible to provide an information processing method for giving points to a user who has transferred a captured video by performing point management in the point management step.

  According to the 33rd aspect of the present invention, there is provided an information processing method for providing a sense of reality to a user in a shooting place by generating additional information in the analysis step and the additional information generation step based on the captured video data of the user. can do.

  According to the 34th aspect of the present invention, it is possible to provide an information processing method that gives a user a sense of realism in a shooting location by distributing past video based on user's shot video data.

  Furthermore, according to the invention of claim 35, it is possible to provide an information processing method that gives a user a sense of reality to a user in a facility or stadium by providing a video from a favorite viewpoint.

  In addition, according to the invention of claim 36, an information processing method for efficiently communicating in the facility or the entire stadium can be provided.

  According to the invention of claim 37, it is possible to provide an information processing method for efficiently communicating in the facility or the entire stadium.

  According to the invention of claim 38, it is possible to provide an information processing method that efficiently restores a three-dimensional shape by using the free viewpoint video stored in advance in the preliminary video storage step.

  According to the invention of claim 39, the analysis step uses only data at a close time in the captured video data and / or is limited to only high-quality data or preferentially uses high-quality data, An information processing method for efficiently performing three-dimensional shape restoration can be provided.

  According to the invention of claim 40, an information processing method for efficiently performing three-dimensional shape restoration can be provided by using the camera attribute information in the analysis step.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a diagram showing a configuration of a video information processing apparatus 100 according to Embodiment 1 of the present invention. In the present embodiment, a video information processing apparatus that prevents the generation of blind spots will be described. A video information processing apparatus that prohibits shooting of a specific area will also be described. The video information processing apparatus according to the present embodiment can be applied to a monitoring system, a video distribution system, and the like.

  The video information processing apparatus 100 illustrated in FIG. 1 includes a storage unit 131 and an analysis unit 141. Each of the N cameras 111 and the M mobile cameras 121 has a function of capturing a video and a function of transferring captured video data to the video information processing apparatus 100.

  Although the functional block diagrams of the camera 111 and the moving camera 121 are not shown, the camera 111 and the moving camera 121 are a video processing unit that captures video and video data generated from the video captured by the imaging unit. A transmission unit that transmits to the device 100. In addition, the camera 111 and the moving camera 121 may include a display unit that displays an image being captured. Note that the video data generated by the camera 111 and the moving camera 121 is obtained by converting the video signal captured by the imaging unit to HEVC or H.264. The data may be encoded using an encoding method such as H.264, or may be unencoded data. When the video data is encoded data, the camera 111 and the moving camera 121 further include an encoder that encodes the video captured by the imaging unit to generate video data and outputs the video data to the transmission unit. In addition, the mobile camera 121 changes a position and a shooting direction of the mobile camera itself according to the reception unit that receives the movement instruction signal transmitted from the video information processing apparatus 100 and the movement instruction signal received by the reception unit. The control part which controls is provided. Note that the control unit may have functions of transmitting video data, receiving a movement instruction signal, and controlling a screen displayed on the display unit.

  The storage unit 131 of the information processing apparatus 100 stores the captured video data when the captured video data of the N cameras 111 and the M mobile cameras 121 is input.

  The analysis unit 141 detects a blind spot from the video data stored in the storage unit 131 and outputs a movement instruction signal to the moving camera 121 so as to prevent the blind spot. The moving camera 121 moves according to the movement instruction signal and continues shooting. The moving camera 121 is, for example, a camera mounted on an unmanned flight type radio control, and is not limited to this as long as it can move a place and / or change a shooting direction.

  The analysis unit 141 can perform blind spot detection using, for example, SfM (Structure from Motion). SfM is a technique for restoring the three-dimensional shape of a subject from a plurality of images taken from different positions, and is widely known as shape restoration from a motion that simultaneously estimates the subject shape and the camera position. For example, Non-Patent Document 1 proposes SfM suitable for sequential processing. Since SfM is described in detail in Non-Patent Document 1 and the like, description thereof is omitted here. The analysis unit 141 may detect an area that cannot be restored as a blind spot by restoring the three-dimensional shape in the facility or the stadium from the video data stored in the storage unit 131 using SfM.

  In addition, when the position and shooting direction of the camera 111 are fixed and the position and shooting direction information is known, SfM may be performed using the known information. If the position and shooting direction of the moving camera 121 can be acquired by a GPS or an angle sensor provided in the moving camera 121, information on the position and shooting direction of the moving camera 121 is transmitted to the analysis unit 141 and transmitted. SfM may be performed using the information on the position and the shooting direction.

  The blind spot detection method is not limited to the method using SfM described above. For example, it is possible to detect the shooting area of each camera by detecting information such as the position of the moving camera, the shooting direction, and zoom (or the shooting area included in the video data) from a marker arranged in the space to be shot. Using this method, it is possible to detect the blind spot. In addition, information such as a positional relationship between a plurality of imaging targets may be acquired from video data or the like, and an area where a blind spot is likely to occur may be specified based on the acquired positional relationship. Such a method has a possibility that the blind spot detection can be realized with a smaller calculation amount when the calculation amount of the SfM is large or a delay occurs in the detection of the blind spot using the SfM for other reasons. Useful in terms.

  Note that the blind spot may include not only a portion where an image does not exist in a region to be photographed, but also a portion having a poor image quality compared to other portions, a portion where a predetermined image quality is not obtained, and the like. In addition, what kind of area is detected as a blind spot may be set according to the configuration and purpose of the monitoring system or the video distribution system, and an area identified based on a reference different from the above example is detected as a blind spot. May be. For example, the required image quality may be set high for a specific subject in the space to be photographed. Conversely, for a specific area in the shooting space, the required image quality may be set low, or may be set so as not to be determined as a blind spot even if no video is shot.

  The above-mentioned image quality includes various information related to the video such as the area occupied by the subject to be photographed in the video (for example, the number of pixels) and whether the subject to be photographed is in focus. It may be determined whether or not it is a blind spot based on the information and the combination thereof.

  In the above description, detection of a region that is actually a blind spot has been described. However, a region that needs to be detected in order to prevent the generation of a blind spot is not limited to a region that is actually a blind spot. For example, when there are a plurality of shooting targets and at least some of them are moving, there is a possibility that a new blind spot may be generated when another shooting target enters between a shooting target and the camera. On the other hand, for example, movements of a plurality of shooting targets are detected from captured video data, and a new blind spot is obtained based on the detected movements of the shooting targets and position information of the camera 111 and the moving camera 121. It is also possible to estimate possible areas. In this case, the video information processing apparatus may be configured to transmit a movement instruction signal to the moving camera 121 so as to capture an area that may become a blind spot, thereby preventing the generation of a blind spot.

  When there are a plurality of moving cameras 121, it is necessary to select the moving camera 121 that transmits a movement instruction signal in order to capture a blind spot or an area that may become a blind spot. In addition, when there are a plurality of moving cameras 121 and blind spots, or areas that may become blind spots, which dead spots or areas that may become blind spots are to be captured for each of the plurality of moving cameras 121. It is necessary to decide. For example, this determination may be performed by selecting a blind camera or a moving camera closest to the blind spot based on the position of the blind spot or an area that may be a blind spot and the position of each moving camera 121 that is being photographed. Good. Further, for each moving camera 121, it is determined whether or not a blind spot is newly generated when the moving camera 121 cannot obtain the video data currently being shot, and even if the video data currently being shot cannot be obtained. You may make it select the moving camera 121 judged that a blind spot does not generate | occur | produce.

  With the above configuration, the analysis unit detects the blind spot and outputs a movement instruction signal to the moving camera so as to prevent the blind spot, thereby preventing the generation of the blind spot.

  An information processing apparatus according to one aspect of the present invention detects an area that is determined not to be captured or has a low quality of captured video from video data captured by a plurality of imaging devices, and The information shown is transmitted to a plurality of photographing devices or photographing devices different from the plurality of photographing devices. According to this configuration, since the region where the shooting data is insufficient in the region to be shot is transmitted to the shooting device, the shooting device can grasp the region where the shooting data is insufficient.

  <Modification-1>

  Next, Modification Example 1 of Embodiment 1 will be described with reference to FIG. The video information processing apparatus 101 illustrated in FIG. 2 has a configuration in which the analysis unit 141 is replaced with an analysis unit 142 as compared to the video information processing apparatus 100 illustrated in FIG. In addition to the video data captured by N cameras, video data captured by L user cameras is input to the video information processing apparatus 101. Each of the L user cameras 151 has a function of capturing a video and a function of transferring the captured video data to the video information processing apparatus 101.

  Although a functional block diagram of the user camera 151 is not illustrated, the user camera 151 transmits a video capturing unit that captures video and video data generated from the video captured by the video capturing unit to the video information processing apparatus 101. A part. In addition, the user camera 151 may include a display unit that displays an image being shot. Note that the video data generated by the user camera 151 is obtained by converting the video signal captured by the imaging unit to HEVC or H.264. The data may be encoded using an encoding method such as H.264, or may be unencoded data. When the video data is encoded data, the user camera 151 further includes an encoder that encodes the video captured by the imaging unit to generate video data and outputs the video data to the transmission unit. Further, the user camera 151 controls to display a direction image on the display unit according to the receiving unit that receives the direction indicating signal transmitted from the video information processing apparatus 101 and the direction indicating signal received by the receiving unit. A control unit is provided.

  When the captured video data of the N cameras 111 and the L user cameras 151 are input, the storage unit 131 stores the captured video data. The analysis unit 142 detects a blind spot in the shooting space from the video data stored in the storage unit 131 and outputs a direction instruction signal to the user camera 151 so as to prevent the generation of the blind spot.

  FIG. 3 is an example of an instruction image displayed on the screen of the user camera 151 based on the direction instruction signal. FIG. 3A is an example of an image indicating the direction in which the user wants the camera to point, and FIG. 3B is another example of an image indicating the direction in which the user wants to move. The instruction displayed on the screen of the user camera 151 is not limited to this. For example, the moving route may be indicated on the displayed map. The user camera 151 is, for example, a smartphone, a tablet-type terminal, a glasses-type wearable terminal, or a HMD (Head Mounted Display) held by a spectator in the stadium or a guard in the facility, and is not limited to this as long as it can shoot video. .

  Further, the display terminal on which the instruction image is displayed based on the direction instruction signal does not have to be the same as the camera that captures the video data. For example, the camera may transmit a direction instruction signal or an instruction image to another display terminal previously associated with the camera that captures the video, and the instruction image may be displayed on the other display terminal. . In addition, when information on a display terminal corresponding to a camera that captures a video is registered in the video information processing apparatus 101 in advance, the video processing apparatus 101 directly transmits a direction indication signal to the display terminal corresponding to the camera. Then, the instruction image may be displayed on the display terminal.

  For example, the analysis unit 142 performs blind spot detection using SfM. The analysis unit 142 uses SfM to restore the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131. Since the position and orientation of the user camera 151 that has captured the video data are estimated by the SfM processing, the user camera that is capturing an area close to the blind spot is identified, and the user camera is identified with respect to the identified user camera 151. A direction indication signal can be transmitted based on the area being shot and the blind spot. When the video data captured by the user data 151 is not used for the SfM processing, the user camera 151 can be used in the facility by matching the restored three-dimensional shape with the captured video of the user camera 151. It may be determined which area in the stadium is being photographed and a direction indication signal may be output. Note that the imaging region determination method and the direction instruction signal generation method are not limited thereto.

  To provide a video information processing apparatus and a video information processing method for preventing a blind spot by detecting a blind spot and outputting a direction instruction signal to a user camera so as to prevent the blind spot with the above configuration. Can do.

  As shown in FIG. 4, the video information processing apparatus 101 can also be applied to the case where captured video data from only the user camera 151 is input. In addition to the user camera 151 and the camera 111, the video information processing apparatus 101 has a function of receiving video data from the moving camera shown in FIG. 1 and transmitting a movement instruction signal to the moving camera. Also good.

  <Modification-2>

  FIG. 5 is a diagram illustrating a modified example-2 of the first embodiment. Compared with the video information processing apparatus 100 shown in FIG. 1, the video information processing apparatus 102 shown in FIG. 5 has a configuration in which the analysis unit 141 is replaced with an analysis unit 143 and a video distribution unit 144 is added.

  The analysis unit 143 uses, for example, SfM, restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 143. The video distribution unit 144 reads video information data corresponding to the visual field information (and / or viewpoint information) input from the video receiving device 161 from the storage unit 131 and outputs the video information data to the video receiving device 161. In addition, the video distribution unit 144 outputs visual field information (and / or viewpoint information) to the analysis unit 143.

  The analysis unit 143 detects the blind spot, and outputs a movement instruction signal to the mobile camera 121 so as to prevent the blind spot when there is a blind spot in the distribution video based on the visual field information (and / or viewpoint information). The moving camera 121 moves according to the movement instruction signal and continues shooting.

  With the above configuration, a video information processing apparatus that prevents a blind spot by detecting a blind spot and outputting a movement instruction signal to a moving camera so as to prevent a blind spot of a video distributed to a video receiving apparatus. A video information processing method can be provided.

  <Modification-3>

  Note that the video information processing apparatus 101 shown in FIG. 2 may be replaced with the video information processing apparatus 103 shown in FIG. Compared with the video information processing apparatus 101 shown in FIG. 2, the video information processing apparatus 103 shown in FIG. 6 has a configuration in which the analysis unit 142 is replaced with an analysis unit 145 and a video distribution unit 144 is added.

  The analysis unit 145 uses SfM, for example, to restore the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 and generate a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 145. The video distribution unit 144 reads video information data corresponding to the visual field information (and / or viewpoint information) input from the video receiving device 161 from the storage unit 131 and outputs the video information data to the video receiving device 161. In addition, the video distribution unit 144 outputs visual field information (and / or viewpoint information) to the analysis unit 145.

  The analysis unit 145 detects the blind spot, and outputs a direction indication signal to the user camera 151 so as to prevent the blind spot when the delivery video based on the visual field information (and / or the viewpoint information) has a blind spot. FIG. 3 is an example of an instruction displayed on the screen of the user camera 151 based on the direction instruction signal.

  With the above configuration, a video information processing apparatus that prevents a blind spot by detecting a blind spot and outputting a direction instruction signal to a user camera so as to prevent a blind spot of a video distributed to a video receiving apparatus. A video information processing method can be provided.

  Further, as shown in FIG. 7, the video information processing apparatus 103 can also be applied to the case where captured video data from only the user camera 151 is input.

  <Modification-4>

  Note that the video information processing apparatus 100 shown in FIG. 1 may be replaced with the video information processing apparatus 104 shown in FIG. Compared with the video information processing apparatus 100 shown in FIG. 1, the video information processing apparatus 104 shown in FIG. 8 has a configuration in which the analysis unit 141 is replaced with an analysis unit 146.

  When the captured video data of the N cameras 122 and the M moving cameras 122 are input, the storage unit 131 stores the captured video data. The analysis unit 146 analyzes the video data stored in the storage unit 131, and outputs a shooting prohibition notification signal to the mobile camera 122 when the mobile camera 122 is shooting a shooting prohibited region. The mobile camera 122 stops shooting while receiving the shooting prohibition notification signal.

  For example, the analysis unit 146 may determine whether or not the photographing prohibited area is photographed using SfM. The analysis unit 146 uses SfM to restore the three-dimensional shape in the facility or the stadium from the video data stored in the storage unit 131 and matches the captured video of the mobile camera 122 so that the mobile camera 122 It may be determined whether or not the photographing prohibited area is being photographed and a photographing prohibition notification signal may be output. Note that the method for determining whether the photographing prohibited area is being photographed and the method for generating the photographing prohibition notification signal are not limited thereto. Examples of the photographing prohibited area include toilets in facilities and stadiums.

  With the above configuration, the image processing information processing apparatus and the video information for prohibiting the shooting of a certain region by the shooting of the analysis unit determining that the shooting is prohibited and outputting a shooting prohibition notification signal to the moving camera so as to stop the shooting. A processing method can be provided.

  <Modification-5>

  Note that the video information processing apparatus 104 shown in FIG. 8 may be replaced with the video information processing apparatus 105 shown in FIG. Compared with the video information processing apparatus 104 shown in FIG. 8, the video information processing apparatus 105 shown in FIG. 9 has a configuration in which the analysis unit 146 is replaced with an analysis unit 147.

  When the captured video data of the N cameras 111 and the L user cameras 152 are input, the storage unit 131 stores the captured video data. The analysis unit 147 analyzes the video data stored in the storage unit 131 and outputs a shooting prohibition notification signal to the user camera 152 when the user camera 152 is shooting a shooting prohibited region.

  FIG. 10 is an example of a notification displayed on the screen of the user camera 152 based on the shooting prohibition notification signal. The instruction displayed on the screen of the user camera 152 is not limited to this. For example, the shooting prohibited area and the current shooting area may be indicated on the displayed map.

  With the above configuration, the image processing apparatus and the video information for prohibiting the shooting of an area by the analysis unit determining that the shooting is prohibited and outputting a shooting prohibition notification signal to the user camera so as to stop the shooting. A processing method can be provided.

  As shown in FIG. 11, the video information processing apparatus 105 can also be applied to the case of inputting captured video data from only the user camera 152.

  (Embodiment 2)

  FIG. 12 is a diagram showing a configuration of the video information processing apparatus 200 according to Embodiment 2 of the present invention. The same components as those of the video information processing apparatus of the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  In the present embodiment, a video information processing apparatus that delivers video for free or at a low price to a user who has transferred a captured video will be described. In addition, a description will be given of a video information processing apparatus that gives points to a user who has transferred a captured video, and gives particularly high points to a user who has transferred a captured video with a requested field of view (and / or viewpoint). To do. The video information processing apparatus according to the present embodiment can be applied particularly to a video distribution system.

  Compared with the video information processing apparatus 100 shown in FIG. 6, the video information processing apparatus 200 shown in FIG. 12 replaces the analysis unit 145 and the video distribution unit 144 with an analysis unit 241 and a video distribution unit 242, respectively, and replaces the charge management unit 243. This is an added configuration.

  When captured video data and user information are input from the L user cameras 251, the storage unit 131 stores them. Further, when photographed video data is input from the N cameras 111, the storage unit 131 stores them.

  The analysis unit 241 uses, for example, SfM, restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 241. The video distribution unit 242 reads video information data corresponding to the visual field information (and / or viewpoint information) input from the video receiving device 161 from the storage unit 131 and outputs the video information data to the video receiving device 161.

  The charge management unit 243 distributes video information data free of charge when a captured video is transferred from the user camera 251 of the same user as the video receiving device 161. Alternatively, the video information data may be distributed at a lower price than the user who has not transferred the captured video. Note that the function of the video reception device 161 may be included in the user camera 251.

  With the above configuration, video information is distributed free of charge or at a low price to the user who transferred the shot video by the charge management unit performing the charge management based on whether or not the user transferred the shot video. A processing apparatus and a video information processing method can be provided.

  As shown in FIG. 13, the video information processing apparatus 200 can also be applied to the case of inputting captured video data from only the user camera 251.

  <Modification-1>

  FIG. 14 is a diagram illustrating a modified example 1 of the second embodiment. Compared with the video information processing apparatus 200 shown in FIG. 12, the video information processing apparatus 201 shown in FIG. 14 has a configuration in which the charge management unit 243 is replaced with a point management unit 244.

  The point management unit 244 gives points to a user who has transferred a captured video, and gives particularly high points to a user who has transferred a captured video of a visual field (and / or viewpoint) with many requests. Alternatively, points may be given only to a user who has transferred a captured image of a field of view (and / or viewpoint) with many requests. The user can receive a paid distribution free of charge at the video reception device 161 using the given points. The method of using the given points is not limited to this.

  With the above configuration, the point management unit performs point management, whereby a video information processing apparatus and a video information processing method for giving points to a user who has transferred a captured video can be provided.

  As shown in FIG. 15, the video information processing apparatus 201 can also be applied to the case where captured video data from only the user camera 251 is input.

(Embodiment 3)
FIG. 16 is a diagram showing a configuration of a video information processing apparatus 300 according to Embodiment 3 of the present invention. The same components as those of the video information processing apparatus described in Embodiments 1 and 2 are denoted by the same reference numerals, and description thereof is omitted. In the present embodiment, a video information processing apparatus that provides additional information and gives a sense of reality to a user in a shooting location will be described. The video information processing apparatus according to the present embodiment can be applied to a monitoring system, a video distribution system, and the like.

  Compared with the video information processing apparatus 101 shown in FIG. 2, the video information processing apparatus 300 shown in FIG. 16 has a configuration in which the analysis unit 142 is replaced with an analysis unit 341 and an additional information generation unit 342 is added.

  When the captured video data of the N cameras 111 and the L user cameras 351 are input, the storage unit 131 stores the captured video data.

  The analysis unit 341 uses, for example, SfM, restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 341. The analysis unit 341 may determine which area in the facility or stadium the user camera 351 is capturing by matching the free viewpoint image and the captured image of the user camera 351. Note that the imaging region determination method is not limited to this, and various imaging region determination methods and other imaging region determination methods described in the above-described embodiments can be used.

  The additional information generation unit 342 outputs additional information to the user camera 351 based on the analysis result of the analysis unit 341. The user camera 351 superimposes additional information on the captured video and displays it on the screen.

  FIG. 17 shows an example of a screen of the user camera 351 when soccer is being played in the stadium. FIG. 17A shows an example of a screen before superimposing additional information, and FIG. 17B shows an example of a screen after superimposing additional information. As an example of additional information, a player name is superimposed. A screen viewed by the user in the stadium using the user camera 351 is transferred to the storage unit 131 of the video information processing apparatus 300 as captured video data, and at the same time, the analysis unit 341 and additional information generation based on the captured video data Since the additional information generated by the unit 342 is superimposed on the screen that the user is viewing, it is possible to give the user a sense of reality in the shooting location. In particular, when the user camera 351 is a glasses-type wearable terminal or an HMD, the user can feel a great sense of realism. The player name is an example of additional information and is not limited thereto. In addition to the video distribution system, in the monitoring system, for example, supervision of the information on the caution person can be achieved on the user camera 351 held by the security guard in the facility, so that the accuracy of the monitoring system can be improved.

  With the above configuration, an information processing apparatus and a video information processing method are provided that give the user a sense of realism in the shooting location by generating additional information based on the user's captured video data. can do.

  As shown in FIG. 18, the video information processing apparatus 300 can also be applied to the case where captured video data from only the user camera 351 is input.

  <Modification-1>

  FIG. 19 is a diagram illustrating a modified example 1 of the third embodiment. Compared with the video information processing apparatus 300 shown in FIG. 16, the video information processing apparatus 301 shown in FIG. 19 has a configuration in which the analysis unit 341 and the additional information generation unit 342 are replaced with an analysis unit 343 and a past video distribution unit 344, respectively. .

  The analysis unit 343 uses, for example, SfM, restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 343. The analysis unit 343 may determine which region in the facility or stadium the user camera 352 is shooting by matching the free viewpoint video and the video shot by the user camera 352. Note that the imaging region determination method is not limited to this, and various imaging region determination methods and other imaging region determination methods described in the above-described embodiments can be used.

  The past video distribution unit 344 outputs the past video to the user camera 352 based on the analysis result of the analysis unit 343. The user camera 352 displays the past video on the screen by superimposing the past video on the captured video or replacing the captured video with the past video.

  FIG. 20 is an example of a screen of the user camera 352 when soccer is being performed in the stadium. FIG. 20A shows an example of a screen before replacement with a past video, and FIG. 20B shows an example of a screen after replacement with a past video. As an example of the past video, the highlight scene in the first half is replaced during the halftime. A screen viewed by the user in the stadium using the user camera 352 is transferred to the storage unit 131 of the video information processing apparatus 301 as captured video data, and at the same time, based on the captured video data on the screen viewed by the user. Thus, the past video generated by the analysis unit 343 and the past video distribution unit 344 is replaced, so that it is possible to give the user a sense of reality in the shooting location. In particular, when the user camera 352 is a glasses-type wearable terminal or an HMD, the user can feel more realistic.

  Accordingly, the user can enjoy the highlight scene of the first half as a video in the direction in which he / she is viewing during the halftime. The past video is not limited to the highlight scene in the first half, but may be a highlight scene of a past game held at the stadium. Moreover, the timing at which the video information processing apparatus 301 distributes the past video is not limited to half time, and may be, for example, after the game is over or during the game. Particularly in the case of a game, the video information processing apparatus 301 may deliver an important scene that the user has missed based on the analysis result of the analysis unit 343. Further, the video information processing apparatus 301 may distribute a past video only when a request is received from a user, or may distribute a distribution permission message before the past video is distributed.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method that provide a user with a sense of reality by distributing past video based on user video data.

  As shown in FIG. 21, the video information processing apparatus 301 can also be applied when inputting captured video data from only the user camera 352.

  <Modification-2>

  FIG. 22 is a diagram illustrating a modification 2 of the third embodiment. The video information processing apparatus 302 illustrated in FIG. 22 has a configuration in which the past video distribution unit 344 is replaced with an advertisement information distribution unit 345 as compared with the video information processing apparatus 301 illustrated in FIG.

  The advertisement information distribution unit 345 outputs advertisement information to the user camera 352 based on the analysis result of the analysis unit 343. The user camera 352 superimposes advertisement information on the captured video and displays it on the screen.

  FIG. 23 is an example of a screen of the user camera 352 when soccer is played in the stadium. FIG. 23A is an example of a screen before the advertisement information is superimposed, and FIG. 23B is an example of the screen after the advertisement information is superimposed, which is a case of a drink advertisement. For example, the advertisement information may be distributed immediately before the past video distribution during the halftime or after the match, as shown in Modification-1. Accordingly, the distributor can obtain an advertisement fee from the advertiser, and can provide a video distribution service to the user at a low cost or free of charge. Further, the video information processing apparatus 302 may distribute an advertisement distribution permission message immediately before distribution of the advertisement information, may provide a service for free only when the user views the advertisement, or does not view the advertisement. Services may be provided at a lower cost.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method that can distribute advertisement information based on user-captured video data.

  As shown in FIG. 24, the video information processing apparatus 302 can also be applied to the case of inputting captured video data from only the user camera 352.

(Embodiment 4)
FIG. 25 is a diagram showing the configuration of the video information processing apparatus 400 according to Embodiment 4 of the present invention. The same constituent elements as those of the video information processing apparatus according to the first to third embodiments are denoted by the same reference numerals, and description thereof is omitted. In the present embodiment, a video information processing apparatus that gives a user a sense of reality in a shooting location will be described. The video information processing apparatus according to the present embodiment can be applied to a monitoring system, a video distribution system, and the like.

  Compared with the video information processing apparatus 301 shown in FIG. 21, the video information processing apparatus 400 shown in FIG. 25 has a configuration in which the analysis unit 343 and the past video distribution unit 344 are replaced with an analysis unit 441 and a video distribution unit 442, respectively.

  When captured video data and viewpoint information are input from the L user cameras 451, the storage unit 131 stores them. Further, when photographed video data is input from the N cameras 111, the storage unit 131 stores them.

  The analysis unit 441 uses, for example, SfM, restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 441. The analysis unit 441 may determine which area in the facility or stadium the user camera 451 is capturing by matching the free viewpoint video and the video captured by the user camera 451. Note that the method for determining the imaging region is not limited to this.

  FIG. 26 shows an example of a method for generating viewpoint information in the user camera 451. FIG. 26A shows an example of the screen of the user camera 351 when soccer is played in the stadium. When the user performs a swipe operation in the direction of an arrow, for example, the user camera 451 displays viewpoint information as movement in that direction. Generate. The video distribution unit 442 reads the video information data moved by the viewpoint information from the shooting area of the user camera 451 determined by the analysis unit 441 from the storage unit 131 and starts output to the user camera 451. The user camera 451 displays the video distributed from the video distribution unit 442 instead of the captured video. The display screen is as shown in FIG. Thus, even after the start of video distribution, the video distribution unit 442 reads the video information data moved from the distribution video area by the amount of the viewpoint information from the storage unit 131 and outputs it to the user camera 161 by the screen swipe operation by the user.

  As described above, the users in the facility and the stadium can view the video from a favorite viewpoint with a simple operation like a screen swipe. For example, a spectator watching on the third base side of a baseball field can view a video from the first base side viewpoint. In addition to the video distribution system, in the surveillance system, the security guards in the facility can watch the video from their favorite viewpoint with a simple operation like a screen swipe, and the surveillance system is highly accurate. be able to.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method that provide a user with a sense of reality by providing a video from a favorite viewpoint.

  In addition, distribution of video to users in facilities or stadiums is also effective when, for example, there is an area that cannot be seen due to an obstacle between the camera and the object to be photographed. In this case, the video of a part of the camera shooting area including the obstacle may be switched from the shot video to the video delivered from the video delivery unit 442, or the entire screen may be displayed from the shot video. It may be switched to the video distributed from the distribution unit 442 and displayed. Alternatively, the captured video and the video distributed from the video distribution unit 442 may be combined to display a video that allows an object to be seen through the obstacle. According to this configuration, even when the shooting target cannot be seen from the position of the user due to the influence of the obstacle, the video delivered from the video delivery unit 442 can be viewed, so that the influence of the obstacle can be reduced. .

  Further, when displaying the video delivered from the video delivery unit 442 for an area that cannot be seen due to an obstacle, display switching control different from the display switching control according to the input processing by the user, such as the screen swipe described above. You may go. For example, when it is determined that an obstacle is included in the shooting area based on information on camera movement, shooting direction, and obstacle position information obtained in advance, the video delivery unit 442 delivers the shot video. The display switching to the video may be automatically performed. In addition, when it is determined by the analysis of the captured video data that an obstacle that is not a target for imaging is reflected, display switching from the captured video to the video distributed from the video distribution unit 442 may be automatically performed. . In addition, when the area of the obstacle (for example, the number of pixels) included in the photographed image exceeds a predetermined threshold or when the ratio of the area of the obstacle to the area to be photographed exceeds a predetermined ratio, The display may be automatically switched to the video distributed from the video distribution unit 442.

  Of course, the display is switched from the captured video to the video distributed from the video distribution unit 442 and the display from the video distributed from the video distribution unit 442 to the captured video is switched according to the user input processing. Needless to say.

  As shown in FIG. 27, the video information processing apparatus 400 can also be applied to the case where captured video data from only the user camera 451 is input.

(Embodiment 5)
FIG. 28 is a diagram showing a configuration of a video information processing apparatus 500 according to Embodiment 5 of the present invention. The same components as those of the video information processing apparatus according to Embodiment 1-4 are denoted by the same reference numerals, and description thereof is omitted. In this embodiment, a video information processing apparatus that instructs a speed at which video data is transferred to the video information processing apparatus based on the importance of the video data captured by each camera will be described. The video information processing apparatus according to the present embodiment can be applied to a monitoring system, a video distribution system, and the like.

  Compared with the video information processing apparatus 101 shown in FIG. 2, the video information processing apparatus 500 shown in FIG. 28 has a configuration in which the analysis unit 142 is replaced with an analysis unit 541 and a communication speed management unit 542 is added.

  When the captured video data of the N cameras 511 and the L user cameras 551 are input, the storage unit 131 stores the captured video data. The analysis unit 541 determines the importance of the video data stored in the storage unit 131 or the camera that captured the video data. The determination of the importance here is performed based on information such as the number of people included in the video, the number of moving objects, the image quality of the video data, and the combination thereof.

  Further, the determination of the importance of the video data may be based on the position of the camera where the video data is shot and the area where the video data is shot. In this case, the importance of the camera in which there are multiple cameras being photographed nearby, the camera in which multiple cameras are photographing the same area even if the camera is far away, or the video data captured by the camera It is conceivable to lower the value. The importance of video data may be determined based on the number of requests in the video distribution service described in the second embodiment. Note that the importance determination method is not limited to the above-described method or a combination thereof, and an importance determination method corresponding to the configuration and purpose of the monitoring system or the video distribution system may be used.

  Further, the determination of the importance may not be based on the captured video data. For example, the importance of a camera that transfers video data to a terminal other than the video information processing apparatus 500 may be determined to be high. Conversely, the importance of a camera that transfers video data to a terminal other than the video information processing apparatus 500 may be determined to be low. As a result, for example, when multiple services that require transmission of video data share a communication band, the degree of freedom in controlling the communication band according to the purpose and characteristics of each service is increased, and the necessary video There is a possibility that deterioration of the quality of each service due to the lack of data can be prevented.

  The analysis unit 541 uses SfM, for example, to restore the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 and generate a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 541. The analysis unit 541 may determine the importance of the video data using the free viewpoint video and the video shot by the camera 511 and the user camera 551.

  The communication speed management unit 542 outputs a wired communication speed instruction signal to the camera 511 and outputs a wireless communication speed instruction signal to the user camera 551 based on the importance determination result performed by the analysis unit 541. For example, the communication speed management unit 542 may instruct a high communication speed to the camera 511 and the user camera 551 that are capturing images with high importance. As a result, communication within the facility and the entire stadium can be performed efficiently. When the camera 511 performs wireless communication, the communication speed management unit 542 outputs a wireless communication speed instruction signal. When the user camera 551 performs wired communication, the communication speed management unit 542 outputs a wired communication speed instruction signal. Is output. The communication speed management unit 542 may be configured to control only one of wired communication and wireless communication and transmit only one of the wired communication speed instruction signal and the wireless communication speed instruction signal.

  The camera 511 and the user camera 551 transfer the captured video data to the video information processing apparatus 500 in accordance with a wired communication speed instruction signal or a wireless communication speed instruction signal. Note that when the retransmission of the camera 511 and the user camera 551 fails a predetermined number of times, the retransmission of the captured video data may be stopped and the transfer of the next captured video data may be started. As a result, communication within the facility or the entire stadium can be performed efficiently, which can contribute to speeding up of processing in the analysis unit 541.

  In addition, when the communication speed assigned to each of the camera 511 and the user camera 551 is not sufficient to transfer the captured video data, the camera 511 and the user camera 551 convert the captured video data to video data of a transferable bit rate. Then, the converted video data may be transmitted. In addition, the camera 511 and the user camera 551 may stop the transfer of the video data when the allocated communication speed is not sufficient for transferring the captured video data.

  In addition, when using video data to prevent the generation of the blind spot described in the first and second embodiments, only a part of the shooting area included in the shot video data has a blind spot. May be needed to fill. In this case, it is also possible to generate extracted video data in which only a part of a video including an area necessary for preventing the generation of blind spots is generated, and to transfer the generated extracted video data. According to this configuration, it is possible to prevent blind spots from occurring with a smaller communication band.

  Further, for example, when the superimposed display of the additional information described in the third embodiment or the video distribution described in the fourth embodiment is performed, it is necessary to transmit the camera position information and the shooting direction information to the video information processing apparatus. There is. In this case, a camera that has been allocated only a bandwidth that is not sufficient for transferring video data may transfer only position information detected on the camera side and information on the shooting direction. In addition, when estimating the position information and shooting direction information of the camera in the video information processing apparatus 500, the video data is converted to a resolution necessary for estimating the position information of the camera and the shooting direction information, and the converted video data May be transferred. According to this configuration, it is possible to provide additional information superimposed display and video distribution services even for cameras to which only a small communication band is allocated. In addition, according to this configuration, since it is possible to acquire information on the area captured by each camera for more cameras, for example, the area captured by the camera for the purpose of detecting an area of interest This is also effective when information is used.

  Note that the switching of the video data transfer process according to the allocated communication band described above may be performed on the camera side based on the notified communication band. The operation may be determined and each camera may be notified as a control signal indicating the determined operation. Accordingly, it is possible to appropriately share the processing according to the calculation amount necessary for determining the switching of the operation, the processing capability of the camera control unit, the necessary communication band, and the like.

  With the above configuration, the analysis unit determines the importance of the captured video data, and instructs the transfer rate based on the importance, thereby efficiently communicating the entire facility or stadium. A processing method can be provided.

  Also, as shown in FIG. 29, the video information processing apparatus 500 can be applied to the case where the captured video data from only the user camera 551 is input.

  <Modification-1>

  Note that the video information processing apparatus 500 shown in FIG. 28 may be replaced with the video information processing apparatus 501 shown in FIG. Compared with the video information processing apparatus 500 shown in FIG. 28, the video information processing apparatus 501 shown in FIG. 30 has a configuration in which the analysis unit 541 is replaced with an analysis unit 543 and a video distribution unit 545 is added.

  The analysis unit 543 uses, for example, SfM, restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 543. The video distribution unit 545 reads video information data corresponding to the visual field information (and / or viewpoint information) input from the video receiving device 161 from the storage unit 131 and outputs the video information data to the video receiving device 161. In addition, the video distribution unit 545 outputs visual field information (and / or viewpoint information) to the analysis unit 543.

  The analysis unit 543 may determine the importance of the video data based on the visual field information (and / or viewpoint information). For example, the importance of captured video data including a large area indicated by the visual field information (and / or viewpoint information) is increased. Further, the analysis unit 543 may determine the importance of the video data in consideration of the number of people included in the video, the number of moving objects, and the like. Note that the importance determination method is not limited to this.

  With the above configuration, the analysis unit determines the importance of the captured video data, and instructs the transfer rate based on the importance, thereby efficiently communicating the entire facility or stadium. A processing method can be provided.

  As shown in FIG. 31, the video information processing apparatus 501 can also be applied to the case where captured video data from only the user camera 551 is input.

  Note that the communication control method described in the present embodiment is not necessarily used in a system that reconstructs a three-dimensional shape from a plurality of video data. For example, in the environment where there are a plurality of cameras and user cameras, if video data is transmitted selectively or with a difference in transmission speed by wired communication and / or wireless communication, the communication control described in this embodiment The method may be effective.

(Embodiment 6)
FIG. 32 is a diagram showing a configuration of a video information processing apparatus 600 according to Embodiment 6 of the present invention. The same components as those of the video information processing apparatuses of the first to fifth embodiments are denoted by the same reference numerals, and the description thereof is omitted. The present embodiment relates to a video information processing apparatus that efficiently restores a three-dimensional shape from a plurality of shooting data in a facility or a stadium. The video information processing apparatus according to the present embodiment can be applied to a monitoring system, a video distribution system, and the like.

  32 has a configuration in which the video distribution unit 144 and the analysis unit 145 are replaced with a video distribution unit 641 and an analysis unit 642, respectively, as compared to the video information processing device 103 illustrated in FIG.

  When the video data captured by each of the N cameras 111 and the L user cameras 651 is input, the storage unit 131 stores the video data.

  The analysis unit 642 uses, for example, SfM, restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131, and generates a free viewpoint video. The storage unit 131 stores the free viewpoint video generated by the analysis unit 642.

  When a distribution request is input from the video reception device 161, the video distribution unit 641 reads out an overview video of the entire facility or stadium from the storage unit 131 and outputs it to the video reception device 161. The overview video may have a long update interval (may be a low frame rate) and may have a low image quality. The viewer gives an instruction by touching a portion of the overview video displayed on the screen of the video reception device 161 by touching the part to be viewed, and the video reception device 161 sends visual field information (and / or viewpoint information) to the video distribution unit 641. Output.

  The video delivery unit 641 reads video information data corresponding to the visual field information (and / or viewpoint information) from the storage unit 131 and outputs the video information data to the video reception device 161. In addition, the video distribution unit 641 outputs visual field information (and / or viewpoint information) to the analysis unit 642.

  The analysis unit 642 preferentially restores the three-dimensional shape to the region indicated by the visual field information (and / or viewpoint information), and generates a free viewpoint video. The analysis unit 642 may extend the update interval (may be a low frame rate) or may lower the image quality, assuming that the three-dimensional shape restoration of the entire facility or stadium shows an overview. The storage unit 131 stores the free viewpoint video generated by the analysis unit 642. Thereby, three-dimensional shape restoration can be performed efficiently. As a result, it contributes to a higher frame rate and higher image quality of the free viewpoint video in the area the viewer wants to see.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method that efficiently perform three-dimensional shape restoration.

  As shown in FIG. 33, the video information processing apparatus 600 can also be applied to the case where the captured video data from only the user camera 651 is input.

  <Modification-1>

  Note that the video information processing apparatus 600 shown in FIG. 32 may be replaced with the video information processing apparatus 601 shown in FIG. 34 includes a storage unit 131, a pre-video storage unit 632, and an analysis unit 643.

  For example, the preliminary image storage unit 632 stores, in advance, pre-generated three-dimensional shape restoration data of a facility or a stadium as a preliminary image. Note that the preliminary image is not limited to this.

  The analysis unit 643 uses SfM, for example, to restore the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 and generate a free viewpoint video. For example, when soccer is being played in a stadium, the analysis unit 643 may generate a free viewpoint video by performing three-dimensional shape restoration only for the players and the ball and combining it with the previous video. Alternatively, the analysis unit 643 may perform the three-dimensional shape restoration by giving priority to the player and the ball. Thereby, three-dimensional shape restoration can be performed efficiently. As a result, it contributes to a higher frame rate and higher image quality of the free viewpoint video regarding the player and the ball that the viewer pays attention to. When applied to the monitoring system, the analysis unit 643 may perform the three-dimensional shape restoration by limiting to only a person and a moving object, or giving priority to them.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method that more efficiently restore a three-dimensional shape.

  Also, as shown in FIG. 35, the video information processing apparatus 601 can be applied to the case of inputting captured video data from only the user camera 651.

  <Modification-2>

  Note that the video information processing apparatus 600 shown in FIG. 32 may be replaced with the video information processing apparatus 602 shown in FIG. A video information processing apparatus 602 illustrated in FIG. 36 includes a storage unit 131 and an analysis unit 644.

  The analysis unit 644 restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131 using, for example, SfM, and generates a free viewpoint video. The analysis unit 644 restores the three-dimensional shape using only data at close times among the captured video data of the plurality of cameras 111 and the plurality of user cameras 651. The detection of the time is, for example, the time when the captured video data has been transferred to the storage unit. The time detection method is not limited to this. Thereby, three-dimensional shape restoration can be performed efficiently. As a result, it contributes to higher frame rate and higher image quality of free viewpoint video.

  Alternatively, the analysis unit 644 may perform the three-dimensional shape restoration by limiting only the high-quality data among the video data stored in the storage unit 131 or preferentially using the high-quality data.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method that efficiently perform three-dimensional shape restoration.

  As shown in FIG. 37, the video information processing apparatus 602 can also be applied to the case where captured video data from only the user camera 651 is input.

  <Modification-3>

  Note that the video information processing apparatus 600 shown in FIG. 32 may be replaced with the video information processing apparatus 603 shown in FIG. The video information processing apparatus 603 illustrated in FIG. 38 includes a storage unit 131 and an analysis unit 645.

  When the captured video data and camera attribute information are input from the L user cameras 652 and the N cameras 612, the storage unit 131 stores them. Examples of camera attribute information include, but are not limited to, a shooting position, a shooting angle, a shooting time, and a zoom.

  The analysis unit 645 uses, for example, SfM, restores the three-dimensional shape in the facility or stadium from the video data stored in the storage unit 131, and generates a free viewpoint video. The analysis unit 645 performs three-dimensional shape restoration using the camera attribute information. Thereby, three-dimensional shape restoration can be performed efficiently. As a result, it contributes to higher frame rate and higher image quality of free viewpoint video.

  A method for generating camera attribute information in the camera 612 and the user camera 652 will be described below, but is an example, and is not limited thereto.

  Define three-dimensional coordinates in the facility or stadium, and store information about which coordinates the camera 612 and user camera 652 took from which angle, how much zoom, and at what time as camera attribute information Then, it may be transferred to the video information processing apparatus 603. When the camera 612 and the user camera 652 are activated, time information can be generated by synchronizing the clock on the communication network in the facility or stadium with the clock in the camera.

  Regarding the specification of the coordinates, for example, the position / angle information may be acquired by aligning the camera with a specific point in the facility or stadium when the user camera 652 is activated or at an arbitrary timing. FIG. 39 is an example of a notification displayed on the screen of the user camera 652 when the user camera 652 is activated. When the user follows this notification and puts the “+” at the center of the soccer ball in the advertisement on the north side of the stadium and touches the display of the camera, the user camera 652 acquires vector information up to the camera and advertisement, and acquires the camera position and angle. Identify criteria. After that, the camera coordinates and angle are identified from the camera motion information.

  Regarding the specification of coordinates, the camera 612 and the user camera 652 may use GPS or may use WiFi (wireless LAN) radio waves. Further, information on which base station in the facility or stadium the captured video data has arrived may be used.

  With the above configuration, it is possible to provide a video information processing apparatus and a video information processing method that efficiently perform three-dimensional shape restoration.

  As shown in FIG. 40, the video information processing apparatus 603 can also be applied to the case where captured video data from only the user camera 652 is input.

(Supplement)
The present invention is not limited to the contents described in the above embodiment, and can be implemented in any form for achieving the object of the present invention and the object related thereto or incidental thereto. .

  (1) The first to sixth embodiments may be implemented in any combination.

  (2) In the image matching in the first, third, and fourth embodiments, the three-dimensional shape restoration data to be used may be the preliminary image shown in the sixth embodiment.

  (3) The above embodiment may relate to implementation using hardware and software. The above-described embodiments may be implemented or executed using a computing device (processor). The computing device or processor is, for example, a main processor / general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable device such as a FPGA (field programmable gate, etc.). It may be. The above embodiments may be executed or realized by combining these devices.

  (4) The above embodiments may be realized by a mechanism of a software module that is executed by a processor or directly by hardware. A combination of software modules and hardware implementation is also possible. The software modules may be stored on various types of computer readable storage media, such as RAM, EPROM, EEPROM, flash memory, registers, hard disk, CD-ROM, DVD, etc.

  The information processing apparatus, information processing method, camera, receiving apparatus, and receiving method according to the present invention can be applied to a monitoring system, a video distribution system, and the like.

100, 101, 102, 103, 104, 105, 200, 201, 300, 301, 302, 400, 500, 501, 600, 601, 602, 603 Video information processing apparatus 111, 511, 612 Camera 121, 122 Moving camera 131 Storage unit 141, 142, 143, 145, 146, 147, 241, 341, 343, 441, 541, 543, 642, 643, 644, 645 Analysis unit 144, 242, 442, 545, 641 Video distribution unit 151, 152, 251, 351, 352, 451, 551, 651, 652 User camera 161 Video reception device 243 Charge management unit 244 Point management unit 342 Additional information generation unit 344 Past video distribution unit 345 Advertisement information distribution unit 542 Communication speed management unit 632 Advance video storage

Claims (40)

A storage unit for storing a plurality of captured video data;
An analysis unit that detects a blind spot from the plurality of captured video data stored in the storage unit, generates an instruction signal so as to prevent the blind spot, and outputs the instruction signal to a camera that generates the captured video data. Information processing apparatus. The information processing apparatus according to claim 1, wherein the analysis unit generates a signal instructing movement of the camera and / or a signal instructing a shooting direction of the camera as the instruction signal. The information processing apparatus further includes a video distribution unit,
The analysis unit generates a free viewpoint video from the captured video data stored in the storage unit,
The storage unit stores the free viewpoint video generated by the analysis unit,
The video distribution unit outputs video information data corresponding to input visual field information and / or viewpoint information to a video receiving device that generates the visual field information and / or viewpoint information,
The information processing apparatus according to claim 1, wherein the analysis unit generates an instruction signal so as to prevent a blind spot when the video information data output from the video distribution unit has a blind spot. A camera which moves based on an instruction signal output from the information processing apparatus according to claim 1 and / or displays instruction contents on a screen. A storage unit for storing captured video data;
From the captured video data stored in the storage unit, it is detected whether or not a capturing prohibited area is captured, and a notification signal is generated so as to prohibit capturing when capturing, and the captured video An information processing apparatus comprising: an analysis unit that outputs data to a camera that generates data. The information processing apparatus according to claim 5, wherein the analysis unit generates, as the notification signal, a signal instructing prohibition of photographing by the camera and / or a signal informing prohibition of photographing of the camera. 6. A camera that stops shooting and / or displays notification contents on a screen based on a notification signal output from the information processing apparatus according to claim 5. A storage unit for storing captured video data;
An analysis unit for generating a free viewpoint video from the captured video data stored in the storage unit;
A video distribution unit for outputting video information data;
A billing management unit for managing billing;
The storage unit stores the free viewpoint video generated by the analysis unit,
The video distribution unit reads video information data corresponding to input visual field information and / or viewpoint information from the storage unit, and outputs the video information data to the video receiving device that generates the visual field information and / or viewpoint information,
The billing management unit performs billing management of the video information data output from the video distribution unit based on whether or not the photographed video data has been transferred from the camera of the same user as the video receiving device. Information processing apparatus. A storage unit for storing captured video data;
An analysis unit for generating a free viewpoint video from the captured video data stored in the storage unit;
An information processing apparatus comprising: a point management unit that manages points for a camera user who generates the captured video data. A storage unit for storing captured video data;
An analysis unit for analyzing the captured video data stored in the storage unit;
An information processing apparatus comprising: an additional information generation unit that generates additional information to be superimposed on the captured video data based on an analysis result of the analysis unit and outputs the additional information to a camera that generates the captured video data. The information processing apparatus according to claim 10, wherein the additional information is advertisement information. An additional information output from the information processing apparatus according to claim 10 is displayed by being superimposed on the captured video data. A storage unit for storing captured video data;
An analysis unit for generating a free viewpoint video from the captured video data stored in the storage unit;
It has a past video distribution unit that outputs video information data,
The storage unit stores the free viewpoint video generated by the analysis unit,
The analysis unit analyzes the captured video data stored in the storage unit,
The past video distribution unit reads past video information data from the storage unit based on the analysis result of the analysis unit and outputs the data to a camera that generates the captured video data. A camera that displays past video information data output from the information processing apparatus according to claim 13. A storage unit for storing captured video data and viewpoint information;
An analysis unit for generating a free viewpoint video from the captured video data stored in the storage unit;
It has a video distribution unit that outputs video information data,
The information processing apparatus, wherein the video distribution unit reads video information data corresponding to the viewpoint information from a storage unit and outputs the video information data to a camera that generates the captured video data and viewpoint information. The camera characterized by displaying the video information data output from the information processing apparatus of Claim 15. A storage unit for storing captured video data;
An analysis unit for analyzing the captured video data stored in the storage unit;
It has a communication speed management unit that manages the communication speed,
The communication speed management unit generates an instruction signal instructing a communication speed of a camera that generates the captured video data based on an analysis result of the analysis unit, and outputs the instruction signal to the camera. . The information processing apparatus further includes a video distribution unit,
The analysis unit generates a free viewpoint video from the captured video data stored in the storage unit,
The storage unit stores the free viewpoint video generated by the analysis unit,
The video distribution unit reads video information data corresponding to input visual field information and / or viewpoint information from a storage unit, and outputs the video information data to a video reception device that generates the visual field information and / or viewpoint information,
The analysis unit determines the importance of the captured video data based on the visual field information and / or viewpoint information,
The information processing apparatus according to claim 17, wherein the communication speed management unit generates the instruction signal based on the importance. 18. A camera that transfers captured video data to the information processing device based on an instruction signal output from the information processing device according to claim 17. A storage unit for storing captured video data;
An analysis unit for generating a free viewpoint video from the captured video data stored in the storage unit;
It has a video distribution unit that outputs video information data,
The storage unit stores the free viewpoint video generated by the analysis unit,
The video distribution unit reads video information data corresponding to input visual field information and / or viewpoint information from the storage unit, and outputs the video information data to the video receiving device that generates the visual field information and / or viewpoint information,
The information processing apparatus, wherein the analysis unit preferentially generates a free viewpoint video for an area indicated by the visual field information and / or viewpoint information. 21. A video receiving apparatus, wherein the video information data output from the information processing apparatus according to claim 20 is displayed. A storage unit for storing captured video data;
A pre-image storage unit for storing pre-generated free viewpoint images;
An information processing apparatus comprising: an analysis unit that updates and generates a free viewpoint video from the captured video data stored in the storage unit and the free viewpoint video stored in the pre-video storage unit . A storage unit for storing captured video data;
An analysis unit that generates a free viewpoint video from the captured video data stored in the storage unit;
The storage unit stores the free viewpoint video generated by the analysis unit,
The information processing apparatus, wherein the analysis unit uses only data at a close time in the captured video data and / or restricts to only high-quality data or uses high-quality data preferentially. A storage unit for storing captured video data and camera attribute information;
An analysis unit that generates a free viewpoint video from the captured video data stored in the storage unit and the camera attribute information from the captured video data;
The information processing apparatus, wherein the camera attribute information is at least one of a shooting position, a shooting angle, a shooting time, and a zoom. A camera attribute information used by the information processing apparatus according to claim 24 is generated. 26. The camera according to claim 25, wherein the camera attribute information is generated by matching a shooting position with a reference point when the camera is activated. 26. The camera according to claim 25, wherein the camera position information is generated by using at least one of information on GPS, WiFi, and a base station as the camera attribute information. 26. The camera according to claim 25, wherein the camera attribute information is generated by synchronizing a shooting time with a clock on a communication network and a clock in the camera. A storage step for storing the captured video data;
An analysis step of detecting a blind spot from the captured video data stored in the storage step, generating an instruction signal so as to prevent the blind spot, and outputting the instruction signal to a camera that generates the captured video data. Information processing method. A storage step for storing the captured video data;
From the captured video data stored in the storing step, it is detected whether or not a capturing prohibited area is captured, and if it is captured, a notification signal is generated to prohibit capturing, and the captured video An information processing method comprising an analysis step of outputting to a camera that generates data. A storage step for storing the captured video data;
An analysis step of generating a free viewpoint video from the captured video data stored in the storage step;
A video distribution step for outputting video information data;
A billing management step for managing billing;
The storing step stores the free viewpoint video generated in the analyzing step,
The video distribution step reads video information data corresponding to input visual field information and / or viewpoint information from the storage step, and outputs the video information data to the video receiver that generates the visual field information and / or viewpoint information,
The charge management step performs charge management of the video information data output in the video distribution step based on whether or not the captured video data has been transferred from the camera of the same user as the video receiving device. Information processing method. A storage step for storing the captured video data;
An analysis step of generating a free viewpoint video from the captured video data stored in the storage step;
An information processing method comprising: a point management step of managing points for a camera user who generates the captured video data. A storage step for storing the captured video data;
An analysis step of analyzing the captured video data stored in the storage step;
An information processing method comprising: an additional information generation step of generating additional information to be superimposed on the captured video data based on an analysis result of the analysis step and outputting the additional information to a camera that generates the captured video data. A storage step for storing the captured video data;
An analysis step of generating a free viewpoint video from the captured video data stored in the storage step;
A past video distribution step of outputting video information data;
The storing step stores the free viewpoint video generated in the analyzing step,
The analysis step analyzes the captured video data stored in the storage step,
The information processing method according to claim 1, wherein the past video distribution step reads past video information data from the storage step based on the analysis result of the analysis step and outputs the data to the camera that generates the captured video data. A storage step for storing captured video data and viewpoint information;
An analysis step of generating a free viewpoint video from the captured video data stored in the storage step;
A video distribution step for outputting video information data;
In the video distribution step, the video information data corresponding to the viewpoint information is read from the storage step and output to the camera that generates the captured video data and the viewpoint information. A storage step for storing the captured video data;
An analysis step of analyzing the captured video data stored in the storage step;
A communication speed management step for managing the communication speed;
The communication speed management step generates an instruction signal instructing a communication speed of a camera that generates the captured video data based on an analysis result of the analysis step, and outputs the instruction signal to the camera. . A storage step for storing the captured video data;
An analysis step of generating a free viewpoint video from the captured video data stored in the storage step;
A video distribution step for outputting video information data;
The storing step stores the free viewpoint video generated in the analyzing step,
The video distribution step reads video information data corresponding to input visual field information and / or viewpoint information from the storage step, and outputs the video information data to the video receiver that generates the visual field information and / or viewpoint information,
In the information processing method, the analysis step generates a free viewpoint video preferentially for the area indicated by the visual field information and / or the viewpoint information. A storage step for storing the captured video data;
A pre-image storage step for storing a pre-generated free viewpoint image;
An information processing method comprising: an analysis step of updating and generating a free viewpoint video from the captured video data saved in the saving step and the free viewpoint video saved in the pre-video saving step . A storage step for storing the captured video data;
An analysis step of generating a free viewpoint video from the captured video data stored in the storage step;
The storing step stores the free viewpoint video generated in the analyzing step,
The information processing method characterized in that the analysis step uses only data at a close time in the photographed video data and / or limits to only high-quality data or preferentially uses high-quality data. A storage step for storing captured video data and camera attribute information;
An analysis step of generating a free viewpoint video from the photographed video data saved in the saving step and the camera attribute information from the shooting video data;
The information processing method, wherein the camera attribute information is at least one of a shooting position, a shooting angle, a shooting time, and a zoom.

Images